Friday, August 27, 2010
The Global Justice XML Data Model
THE PROBLEM
The Global Justice XML Data Model - (GJXDM - Wikipedia Entry) is an XML interchange format used by law enforcement and other justice agencies in the United States.
It's a replete standard - it contains over 400 complex types and around 150 simple types with a total of around 2,000 associations (properties). Almost half of these focus on the Activity area (such as an Arrest) and on Personal Details.
Actual adoption of the model will vary depending on the context. This is due to expected variances in implementation, but also to the context of the application. GJXDM is implemented across a diverse range of institutions at different levels of government, each with different concerns and underlying objectives. The GJXDM has also been adopted in geographies outside the United States. Naturally these implementations have requirements that go beyond the available concepts, but also find large sections of the model inapplicable for their region.
The challenge is how to adhere to the entrenched standard whilst also accommodating the necessary variations on specific implementations, and keeping in sync with any updates to the standard over time.
HOW GRAFT HELPS
We believe Graft offers a lot to users of GJXDM. The first benefit is that it allows users to visually navigate GJDM as a domain model.
A key feature of Graft is the ability to extend other data models - we allow this extension in two modes: Active or Passive.
All of the elements of a passively extended model appear "greyed out" or "ghosted" in the modeling tool. You can then opt to selectively bring each of these elements into your model, even potentially over multiple releases of your implementation. To get an idea, you can extend the GJXDM Model yourself and experiment with drawing forward the parts of the model that are of relevance to you. Your modifications remain private until you explicitly choose to make your extended model public.
This allows you to cherry-pick the elements you need as they are implemented. Instead of handing out a schema with thousands of elements, you can produce a schema (by visiting the export tab of your model) that shows the elements and relationships actually being used, whilst still remaining consistent with the source schema.
An ancillary side-effect on this is performance. Large schemas can introduce a parsing bottleneck; reducing schemas to only include elements actually in use can make a big difference.
GRAFT KEEPS CUSTOMIZATIONS IN SYNC
Another key benefit of Graft is the control over extensions and modifications to the model. If you actively extend the source of GJXDM or passively extend the source of GJXDM, you are not left stranded on a standalone branch of the model. Your model is actually kept as a delta from the source model, allowing you to easily upgrade to future standard changes.
As an example of extensions and modifications being kept in sync with changes to the source GJXDM model, you might rename an element like LocationPostalCodeID - particularly if you're implementing the model in a very specific geography. This name change is stored as a delta, you can then update the underlying source GJXDM model and this change would still apply whilst still enabling you to get all of the updates to the underlying model.
Changes that consumers of GJXDM make can even be re-incorporated to the original model by the original model's administrators. A future update may transparently include the updates from derived models.
This is a very powerful outcome. In many modeling exercises tools encourage you to take a model, customize it, and effectively create an island. Graft encourages a different approach - Instead of grabbing a model and morphing it - Take an existing model, only use the pieces you need, focus on the changes you need for this release alone, and then keep taking the advantage over time.
YOU CAN LEVERAGE FROM MULTIPLE MODELS
Graft doesn't limit you to extending just one model. You can extend and integrate any number of models. It's unlikely that GJXDM is the only model you need to use. There may be other local and international standards, as well as in-house and bespoke representations for areas that are perhaps not covered or not covered appropriately for your needs in GJXDM. Graft lets you bring all of these models together, integrating and leveraging elements wherever required.
The Specify Tool in Graft lets you specify any number of Active and Passive Extensions, you can even modify these "on-the-fly" - for example, by replacing an existing Active Extension with a later release, or even with a different implementation.
We'll be working a lot more in and around Industry Standards such as the Global Justice XML Data Model. If you're working in this space, we'd love to hear your thoughts, comments and feedback.
Thanks,
jon@jodoro.com
Sunday, August 22, 2010
Business Process Modeling - What's Your Purpose?
When taking on a Business Process Modeling exercise, it's good to outline the purpose up-front. Some of the key perspectives that you want to consider are:
- Business Process Re-Engineering (BPR)
You want to examine the Business Process to make improvements - for example, increase automation, reduce duplication, streamline and parallelize.
- Execution
Intent is to take the Business Process and execute it on a technology platform, such as IBM WebSphere Process Server. Often this is part and parcel of increasing automation, or streamlining the technology.
- Instrumentation
Intent in this case is to design around measuring and monitoring a process. You might want certain customer orders or interactions resolved in a timeframe. Or you may want certain items to be escalated after a critical time period has elapsed. Or you might want to be recording data points that can be accumulated and mined for patterns after-the-fact.
- User Interaction
Aim here is to model the user interactions with a process. Central focus in these models is naturally the users, their team structures, skills and locations. There are a number of reasons for doing this; skills and role realignment. Building and refining escalation structures. Ensuring privacy and clearance compliance. Optimising team structures. Optimising and perhaps consolidating locations. Undertaking outsourcing or offshoring.
If you draw a business process from one of these perspectives, most of the time they will look completely different from the others. This can present a potential pitfall for process models. For anything of any complexity it's nigh on impossible to get a model that incorporates all of these concerns adequately. Conversely, if your problem is simple, then these approaches are probably overkill.
An example might help position this better -- If your aim is BPR, you might put each human task in sequence, even though these are effectively done by a single person all at once. The reason you put them in sequence is that you have data that tells you how long each individual piece takes, and you also know that they are usually done together. When you simulate, you get an accurate picture of how long the macro pieces take, and where critical paths exist in the process. You also get some measures of complexity - a major one being the number of unique paths through the process.
(As an anecdotal aside - I heard of a major utility that mapped their provisioning process - and the number of unique paths exceeded the total number of customers)
So, someone processing an order might check the customer's credit, validate their address, check the shipping costs, check stock levels, enter the order and then submit it to be fulfiled by the warehouse. However, it's unlikely that anyone will do those tasks in that exact order. There might be dozens of reasons of this, a common one simply being the order of papers in a pile.
This is a trivial case, but it could be significantly more complex with something like processing a mortgage application, or a business loan, where there can be dozens (or indeed hundreds) of fragments of information.
The intent in this kind of process modelling is usually to uncover overlaps and efficiency opportunities -- in processing a new customer order, you might be validating a customer's address numerous times. This could ideally be reduced to once; or twice if you have a Quality Assurance stage.
The problem is this doesn't necessarily represent the process in a way suitable for other objectives - such as execution, user experience or instrumentation. I've seen this happen before. The process gets defined and then forces a user to do a sequence of tasks in a strict order -- when the reality is the user is sitting with a pile of paper in front of them and probably wants to do them in whatever order is convenient. Worst case scenario is this macro task gets formalised as numerous minor tasks that must be checked in and out of work queues, or end up as a horrendous sequential "wizard style" User Interface.
Since the original intent of the exercise was to re-engineer the process to make it better, this is a counter-intuitive outcome. However, without going into that detail and making those assumptions, you couldn't have assembled and simulated the process.
In a similar vein, this process implies that you can instrument the "validate address step", whereas the reality is that this step may well be embedded in person shuffling through some paperwork. It's not possible to get data around this individual step; not in any practical terms anyway. Going even further, all of this might be completely irrelevant from an instrumentation perspective -- the key KPI might be customer satisfaction, which is likely measured in a completely different way.
This is not to say that Business Process Modeling doesn't have significant value. Part of the issue is the hubris that surrounds Business Process Management (BPM) software, which really pushes this as a "new paradigm". The idea is that you sketch out a Business Process and then the software is capable of (magically) executing the process. However, this is really impractical. Tooling can help significantly, but it's a means to an ends. Mapping and understanding your process is the intrinsic value; software enhances or amplifies that.
An approach led by Business Process Modeling can be a significant advantage to the deliver of software projects. It's an excellent means of driving out requirements and outcomes. Just be clear about the purpose up-front, and don't get fixated on auto-magic tooling. Even if you sketch your process on paper, and then code it from scratch, you'll be getting many of the core advantages. Add tools and technology on top to maximize the advantage, not define it.
Jon
- Business Process Re-Engineering (BPR)
You want to examine the Business Process to make improvements - for example, increase automation, reduce duplication, streamline and parallelize.
- Execution
Intent is to take the Business Process and execute it on a technology platform, such as IBM WebSphere Process Server. Often this is part and parcel of increasing automation, or streamlining the technology.
- Instrumentation
Intent in this case is to design around measuring and monitoring a process. You might want certain customer orders or interactions resolved in a timeframe. Or you may want certain items to be escalated after a critical time period has elapsed. Or you might want to be recording data points that can be accumulated and mined for patterns after-the-fact.
- User Interaction
Aim here is to model the user interactions with a process. Central focus in these models is naturally the users, their team structures, skills and locations. There are a number of reasons for doing this; skills and role realignment. Building and refining escalation structures. Ensuring privacy and clearance compliance. Optimising team structures. Optimising and perhaps consolidating locations. Undertaking outsourcing or offshoring.
If you draw a business process from one of these perspectives, most of the time they will look completely different from the others. This can present a potential pitfall for process models. For anything of any complexity it's nigh on impossible to get a model that incorporates all of these concerns adequately. Conversely, if your problem is simple, then these approaches are probably overkill.
An example might help position this better -- If your aim is BPR, you might put each human task in sequence, even though these are effectively done by a single person all at once. The reason you put them in sequence is that you have data that tells you how long each individual piece takes, and you also know that they are usually done together. When you simulate, you get an accurate picture of how long the macro pieces take, and where critical paths exist in the process. You also get some measures of complexity - a major one being the number of unique paths through the process.
(As an anecdotal aside - I heard of a major utility that mapped their provisioning process - and the number of unique paths exceeded the total number of customers)
So, someone processing an order might check the customer's credit, validate their address, check the shipping costs, check stock levels, enter the order and then submit it to be fulfiled by the warehouse. However, it's unlikely that anyone will do those tasks in that exact order. There might be dozens of reasons of this, a common one simply being the order of papers in a pile.
This is a trivial case, but it could be significantly more complex with something like processing a mortgage application, or a business loan, where there can be dozens (or indeed hundreds) of fragments of information.
The intent in this kind of process modelling is usually to uncover overlaps and efficiency opportunities -- in processing a new customer order, you might be validating a customer's address numerous times. This could ideally be reduced to once; or twice if you have a Quality Assurance stage.
The problem is this doesn't necessarily represent the process in a way suitable for other objectives - such as execution, user experience or instrumentation. I've seen this happen before. The process gets defined and then forces a user to do a sequence of tasks in a strict order -- when the reality is the user is sitting with a pile of paper in front of them and probably wants to do them in whatever order is convenient. Worst case scenario is this macro task gets formalised as numerous minor tasks that must be checked in and out of work queues, or end up as a horrendous sequential "wizard style" User Interface.
Since the original intent of the exercise was to re-engineer the process to make it better, this is a counter-intuitive outcome. However, without going into that detail and making those assumptions, you couldn't have assembled and simulated the process.
In a similar vein, this process implies that you can instrument the "validate address step", whereas the reality is that this step may well be embedded in person shuffling through some paperwork. It's not possible to get data around this individual step; not in any practical terms anyway. Going even further, all of this might be completely irrelevant from an instrumentation perspective -- the key KPI might be customer satisfaction, which is likely measured in a completely different way.
This is not to say that Business Process Modeling doesn't have significant value. Part of the issue is the hubris that surrounds Business Process Management (BPM) software, which really pushes this as a "new paradigm". The idea is that you sketch out a Business Process and then the software is capable of (magically) executing the process. However, this is really impractical. Tooling can help significantly, but it's a means to an ends. Mapping and understanding your process is the intrinsic value; software enhances or amplifies that.
An approach led by Business Process Modeling can be a significant advantage to the deliver of software projects. It's an excellent means of driving out requirements and outcomes. Just be clear about the purpose up-front, and don't get fixated on auto-magic tooling. Even if you sketch your process on paper, and then code it from scratch, you'll be getting many of the core advantages. Add tools and technology on top to maximize the advantage, not define it.
Jon
Tuesday, July 20, 2010
New Graft Feature: XSD Exports
Graft now has the ability to generate XSDs for any model. Visit the export tab of your chosen model.
THE DECISION
We have chosen to generate a ComplexType and an accompanying element of the same name for each class that doesn't represent an xsd primitive type. Each class association is represented within the ComplexType definition as either an element or an attribute depending on whether or not the association's type is one of the xsd primitive types. We optionally support a namespace, but at least at this stage further external namespaces will need to be added to the schema post generation.
THE DEBATE
XSD generation is a feature that has caused a fair amount of debate at Jodoro. This is because the XSD 1.1 standard provides many different ways to represent the same concepts.
For example, the Global Justice Data Model defines all of its entities and their associations in ComplexTypes and then defines an element of the same name to take each type. Each ComplexType tends to contain a ComplexContent which in turn may extend an appropriate ComplexType and/or define associations to elements of the defined ComplexType. This model provides maximum flexibility because a valid xml schema could contain any chosen subset of the defined elements. This is particularly useful when many different types of software systems need to consistently communicate with each other.
In contrast the CellML and FieldML schemas define very few top level elements (in fact one), and tend to fully define associations between ComplexTypes within the ComplexTypes themselves. This approach allows for a more formally structured approach to "valid" xml structures, which can be helpful in sharing information between very similar systems. Even these two very similarly structured xsds differ as FieldML does not use any namespacing and treats everything as a ComplexType while CellML utilizes the "cellml" namespace and defines both ComplexTypes and SimpleTypes.
The Schools Interoperability Framework (SIF) defines an element for each ComplexType, but demonstrates yet another structurally different way to build an xsd by defining very few named ComplexTypes and creating many nameless ComplexTypes in their definitions. This directs the consumer's focus to the named ComplexTypes, but comes at a cost of comprehension of the ComplexType definitions (and results in many nameless classes in the domain model). Like CellML, the SIF standard also defines SimpleTypes and attributes differentiating them from ComplexTypes and elements by whether or not they extend the xsd primitive types.
Beyond these structural differences we also needed to contemplate whether and how we support concepts such as enumerations. The issue for us is that this concept blurs the borders between meta-data and data. Technically each enumeration value is one of the valid instances of the enumeration type. It is tempting (and very common) to define enumerations within xsd schemas, particularly when the values are unlikely to change. However we would argue that even if the values won't change a better approach is to define the enumeration as a code of type string or integer, and to store and maintain the valid values outside of the schema. This provides a cleaner separation between structure and business rules.
We also discussed and debated many other commonly used xsd concepts such as "pattern", "maxlength", "union", "choice", "key", "any" and "all". At this stage we have chosen to leave all of these out as they are defining concepts that we are not currently explicitly representing within our domain modelling tool, and because we feel that, like enumerations, many of their uses are often business rules and arguably shouldn't be defined in the schema. If you have suggestions or issues with our current approach, please email myself or support@jodoro.com.
Doug - @douglasenglish .
THE DECISION
We have chosen to generate a ComplexType and an accompanying element of the same name for each class that doesn't represent an xsd primitive type. Each class association is represented within the ComplexType definition as either an element or an attribute depending on whether or not the association's type is one of the xsd primitive types. We optionally support a namespace, but at least at this stage further external namespaces will need to be added to the schema post generation.
THE DEBATE
XSD generation is a feature that has caused a fair amount of debate at Jodoro. This is because the XSD 1.1 standard provides many different ways to represent the same concepts.
For example, the Global Justice Data Model defines all of its entities and their associations in ComplexTypes and then defines an element of the same name to take each type. Each ComplexType tends to contain a ComplexContent which in turn may extend an appropriate ComplexType and/or define associations to elements of the defined ComplexType. This model provides maximum flexibility because a valid xml schema could contain any chosen subset of the defined elements. This is particularly useful when many different types of software systems need to consistently communicate with each other.
In contrast the CellML and FieldML schemas define very few top level elements (in fact one), and tend to fully define associations between ComplexTypes within the ComplexTypes themselves. This approach allows for a more formally structured approach to "valid" xml structures, which can be helpful in sharing information between very similar systems. Even these two very similarly structured xsds differ as FieldML does not use any namespacing and treats everything as a ComplexType while CellML utilizes the "cellml" namespace and defines both ComplexTypes and SimpleTypes.
The Schools Interoperability Framework (SIF) defines an element for each ComplexType, but demonstrates yet another structurally different way to build an xsd by defining very few named ComplexTypes and creating many nameless ComplexTypes in their definitions. This directs the consumer's focus to the named ComplexTypes, but comes at a cost of comprehension of the ComplexType definitions (and results in many nameless classes in the domain model). Like CellML, the SIF standard also defines SimpleTypes and attributes differentiating them from ComplexTypes and elements by whether or not they extend the xsd primitive types.
Beyond these structural differences we also needed to contemplate whether and how we support concepts such as enumerations. The issue for us is that this concept blurs the borders between meta-data and data. Technically each enumeration value is one of the valid instances of the enumeration type. It is tempting (and very common) to define enumerations within xsd schemas, particularly when the values are unlikely to change. However we would argue that even if the values won't change a better approach is to define the enumeration as a code of type string or integer, and to store and maintain the valid values outside of the schema. This provides a cleaner separation between structure and business rules.
We also discussed and debated many other commonly used xsd concepts such as "pattern", "maxlength", "union", "choice", "key", "any" and "all". At this stage we have chosen to leave all of these out as they are defining concepts that we are not currently explicitly representing within our domain modelling tool, and because we feel that, like enumerations, many of their uses are often business rules and arguably shouldn't be defined in the schema. If you have suggestions or issues with our current approach, please email myself or support@jodoro.com.
Doug - @douglasenglish .
Thursday, July 15, 2010
Graft Rails Model Inheritance
We've pushed out a new feature for Graft.
Up until now, each exported class has directly translated across to a Ruby on Rails Model, without regard for inheritance in the models.
Exported Rails Models will now include all the primitive of the Graft class, as well as any superclasses. So if you have BMW inherit from Car, and Car has a "license" of type "String" - both Car and BMW will have the license property. The same applies to relationships.
We've put up an example model that should help illustrate. You can export it to Rails using the instructions listed under the Export tab.
It's fairly new, so if you've got any queries or strike any issues, drop us a line via Twitter @jodoro or email support@jodoro.com.
Right now all classes are generated as Models, so you can consider this the superset. Later releases will allow for finer-grained control, such as flattening (e.g Single Table Inheritance).
Jon - @jonathannen.
Up until now, each exported class has directly translated across to a Ruby on Rails Model, without regard for inheritance in the models.
Exported Rails Models will now include all the primitive of the Graft class, as well as any superclasses. So if you have BMW inherit from Car, and Car has a "license" of type "String" - both Car and BMW will have the license property. The same applies to relationships.
We've put up an example model that should help illustrate. You can export it to Rails using the instructions listed under the Export tab.
It's fairly new, so if you've got any queries or strike any issues, drop us a line via Twitter @jodoro or email support@jodoro.com.
Right now all classes are generated as Models, so you can consider this the superset. Later releases will allow for finer-grained control, such as flattening (e.g Single Table Inheritance).
Jon - @jonathannen.
Wednesday, July 14, 2010
Developing Business Process Models with Domain Models
Quite often Domain Models are developed around or in concert with Business Process Models. This often leads to a the question "how do they inter-relate?" (and subsequently, "who's in charge? what drives the definitions?").
It's important to relate Domain and Business Process Models, but the approach for doing this shouldn't be too onerous. At the most basic level, each Business Process step should have input and outputs that are driven off the Domain Model. If you are processing a shopping cart payment "Process Shopping Cart", the input may be "Shopping Cart" and the output "Invoice", both concepts in your Domain Model.
Generally this is applied at lower levels of a Business Process, L2 or L3, but there is nothing to stop you working top-down or bottom-up. For this type of exercise, I believe a top-down approach is the best fit. Bottom-up has a tendency to get bogged down in the detail (the wrong kind).
The obvious exceptions are cases where the low level processes are well understood. This can occur when the Business Process is already operating or established in another form. These existing processes often have associated pre-existing MI, data or other statistics - and often this information can be quite fine-grained. In these cases, bottom-up isn't a bad place to start.
Whatever the level, it's advantageous to elaborate on the details of the inputs and outputs. What elements of the Shopping Cart are required? Which are optional? For each domain concept, the key questions are "how is it used or applied?", "if it's optional, what are the rules?" and "what's the context?". Aim here should really be to reduce the inputs and outputs to what is absolutely necessary - this is for a few reasons:
That's just a quick summary for now. We'll dive into some of the advantages and pitfalls of models led by Business Process in a later article.
Jon
It's important to relate Domain and Business Process Models, but the approach for doing this shouldn't be too onerous. At the most basic level, each Business Process step should have input and outputs that are driven off the Domain Model. If you are processing a shopping cart payment "Process Shopping Cart", the input may be "Shopping Cart" and the output "Invoice", both concepts in your Domain Model.
Generally this is applied at lower levels of a Business Process, L2 or L3, but there is nothing to stop you working top-down or bottom-up. For this type of exercise, I believe a top-down approach is the best fit. Bottom-up has a tendency to get bogged down in the detail (the wrong kind).
The obvious exceptions are cases where the low level processes are well understood. This can occur when the Business Process is already operating or established in another form. These existing processes often have associated pre-existing MI, data or other statistics - and often this information can be quite fine-grained. In these cases, bottom-up isn't a bad place to start.
Whatever the level, it's advantageous to elaborate on the details of the inputs and outputs. What elements of the Shopping Cart are required? Which are optional? For each domain concept, the key questions are "how is it used or applied?", "if it's optional, what are the rules?" and "what's the context?". Aim here should really be to reduce the inputs and outputs to what is absolutely necessary - this is for a few reasons:
- It best informs your Domain Model.
- It best translates out to other representations, such as a technology implementation - Clarity here will have numerous downstream benefits, particularly in the implementation.
- The act of reduction/distilling itself is a good instrument to drive the exercise.
- It's easier to see what's missing. Blanket terms can hide conflicting points of view and assumptions.
- It greatly assists the construction and execution of tests and user-acceptance.
- Finally, in the heat of delivering a project, it can sometimes be hard to argue "why?" against business requirements. This is an ideal opportunity to ask.
That's just a quick summary for now. We'll dive into some of the advantages and pitfalls of models led by Business Process in a later article.
Jon
Tuesday, July 13, 2010
Workshop your Domain Models
The value of a Domain Model is often overlooked. Done properly it defines the structural integrity of your software system in a language and an entity relationship structure that can be understood unilaterally by the business domain experts and the technical specialists. As such it significantly reduces upfront requirement errors, and better structures the solution to support future change requests.
But a domain model must be owned by both the domain and technical experts on the project for it to realize this value. I have utilized the following facilitation technique, borrowed from Feature Driven Development, to achieve this ownership on several of my projects. It was originally designed for Domain Modeling, but the techniques would equally add value to any form of workshop.
Formalizing the domain model within a workshop process provides a very useful metric. For every week you spend in workshops defining the domain model you can expect a ramped up development team to spend three weeks building the content. So if it takes you four weeks to lock down a domain model for a project that needs to be finished the build phase within the next two months, you should probably take another look at the scope, or the time-line.
Doug
But a domain model must be owned by both the domain and technical experts on the project for it to realize this value. I have utilized the following facilitation technique, borrowed from Feature Driven Development, to achieve this ownership on several of my projects. It was originally designed for Domain Modeling, but the techniques would equally add value to any form of workshop.
- Identify a facilitator. Yes, cliche, but also amazingly too often overlooked. Choose the facilitator wisely. They should not get dragged into the technical debates, and shouldn't be afraid to call time. Keeping the meetings moving is critically important.
- Identify a documenter. Encourage all of the participants to jot down points throughout the workshop - it's the documenters job to collate and distribute these at the end of each day. Focus on decisions and actions, backed up with justifications. Include lots of photographs - it is amazing how unifying their inclusion makes the process. The documentation should be released by close of business on each workshop day.
- Keep the numbers balanced and low. It's of course important to ensure the appropriate stakeholders are represented in the workshop, but the key word is represented. Too many active participants in a workshop will grind its productivity to a halt. I've found that between 4 and 8 people tends to work well. Keep in mind that the workshop participants should also include appropriate representation of the domain experts and the key technical implementers. As close to 50:50 as you can orchestrate the better.
- Choose your location carefully. Pick spacious, quiet and light rooms. Come prepared with appropriate equipment. Ideally choose a location that's outside the team's usual working location. You want to encourage full participation without distractions.
- Define team norms as a team. (And make sure "have fun" is on the list!). Don't knock this one until you've tried it. Asking the team to define their own norms is an excellent way to ensure all team members adhere to the ground rules. Punctuality? Mobile phones? When is lunch?
- Take lots of planned breaks, and finish early. Done properly such workshops are amazingly exhausting. On top of this, the team bonding experience of the breaks is perhaps almost as important as the workshops themselves. Make sure you finish the workshops early each day. I often finish them at lunch time. It's important to ensure the participants have time at the end of each day to take care of business as usual activities, and that the documenter has time to compile the days results. Morning are usually ideal for modelling - people seem to be fresher.
- Reach consensus on issues. Yes in an ideal world the entire project team would immediately and unanimously agree to each decision, but this isn't an ideal world and humans are good at arguing. It's a good idea to give one person, perhaps the project's solution architect, the ultimate veto on decisions. Coming to consensus is a commitment from the team that, even if they don't 100% agree with the selected approach, that they'll at least live with it. Once a consensus is reached it's final. The last thing the project needs is for the same resolutions to be requestioned in a corridor three days later by a subset of the participants.
- Foster diversity. A commonly used Feature Driven Development technique is to break the larger group into two or three subgroups (ensuring domain and technical representatives remain in each subgroup) and ask each to model the same problem space concurrently. After a set amount of time each subgroup resents and the group as a whole drives towards a common single solution. This may mean accepting one of the subgroup solutions entirely, or it may mean merging components of each.
- Keep a "Parking Lot". I've seen the same technique referred to by many names. Essentially it's a team maintained list of topics that need to be closed out by the end of the workshops. It provides a mechanism to capture ideas as they're generated by the team without interrupting the currently flow of activities.
- Pulse check progress. At the end of each day ask for feedback. What's working, what could work better? Is the room too hot? Were the brainstorming sections long enough? This feedback loop can massively improve the experience and end result.
Formalizing the domain model within a workshop process provides a very useful metric. For every week you spend in workshops defining the domain model you can expect a ramped up development team to spend three weeks building the content. So if it takes you four weeks to lock down a domain model for a project that needs to be finished the build phase within the next two months, you should probably take another look at the scope, or the time-line.
Doug
Thursday, July 8, 2010
Draw a Real Picture
In a previous article I mentioned five tips for Domain Modeling. In this post I'm going to drill down further on one of those:
1. Draw a "Real Picture".
I've used Real Pictures as a technique for a number of years; not only in Domain Modeling, but in Architecture, Design, Development and Planning. It's a pretty simple exercise, but yields a lot of interesting information. In this description I talk in the context of running a project, because that's the most common case -- but that is not universal.
A Real Picture is foremost a conversation piece. It lets the stakeholders involved discuss and explore the domain and it's context. Besides just capturing raw information, a Real Picture also serves as a gentle introduction for participants that might not be familiar with formal diagramming approaches. Often a good first step.
To start, get the participants together. You don't need to mass-invite everyone, but make sure all the key representatives and stakeholders are covered. So if it's a software project, you should have sponsors, end-users, designers, developers, testers and so forth. If you need to economise on people, focus on those that have most affected or impacted by the end result of the exercise, rather than on pure expertise.
There isn't much to developing the picture itself -- Simply draw the major concepts and how they inter-relate. In my experience, this is enough to start the conversation flowing; just draw and re-draw the diagram as the "oh, and then there is" comments evolve.
As a rule of thumb, I keep Real Pictures to one page - A3 or a White Board usually. If it gets really messy, redraw the picture. If necessary, drop in prompters to keep the dialogue flowing. Pick a concept and drill down - focus on quantities, qualities, time, costs, constraints:
- How many of these items are there? e.g. How many employees are at that location?
- Does it vary over the day, are there seasonal variations? e.g. Do you have a rush at Christmas, End of Financial Year?
- Will this change or move soon? e.g. Are you changing network providers? When did you last open a new outlet?
- What would invalidate this element or relationship? e.g. What security issues would make a location unviable?
- What do competitors or other equivalents do for this function or outcome? e.g. Does your competitor do this differently? better?
- Are there constraints on the way things should be? e.g. Does the government enforce strict reporting on certain items?
It's important that you don't constrain the actual process and format too much. The objective of this exercise is to get all the stakeholders together and to articulate a landscape that everyone understands. The end result should also be something that all stakeholders reasonably grasps and agree to.
At this stage, try not to abstract too much - early abstraction can lead to some necessary detail or important corner-case being lost. If a specific location, or server, or piece of software is mentioned, it may have special significance.
In one particular instance I found a number of stakeholders would refer repeatedly to a specific printer - i.e. the actual device itself and where it was located. Most of the project team didn't see any significance of this, myself included. The model and design assumed it was a standard office printer, the same any printer we used in our day to day. When we eventually dug deeper, it turns out this specific instance was special - it enabled secure printing. The way the device functions is such that a human operator never gets to see the contents. If you've ever received a bank PIN in an special envelope, then it's probably from a pretty similar device.
Not only was the fact that this device was special important - but also many of the project participants hasn't realised the significance of the information being send by this device. The communications that ended up at this printer required an extra level of security; it has a very special significance. Up until that point most of the team hasn't considered that special, but clearly it was. Similarly, the end-users thought that information was obvious. Suffice to say, staying concrete at this stage usually is the best default - Abstractions will flow later.
A Real Picture sounds trivial, in fact you might be over-the-top to call it a technique. Either way, it's a low impact exercise with a lot of benefits. At the end of the exercise you have:
Jon
1. Draw a "Real Picture".
I've used Real Pictures as a technique for a number of years; not only in Domain Modeling, but in Architecture, Design, Development and Planning. It's a pretty simple exercise, but yields a lot of interesting information. In this description I talk in the context of running a project, because that's the most common case -- but that is not universal.
A Real Picture is foremost a conversation piece. It lets the stakeholders involved discuss and explore the domain and it's context. Besides just capturing raw information, a Real Picture also serves as a gentle introduction for participants that might not be familiar with formal diagramming approaches. Often a good first step.
To start, get the participants together. You don't need to mass-invite everyone, but make sure all the key representatives and stakeholders are covered. So if it's a software project, you should have sponsors, end-users, designers, developers, testers and so forth. If you need to economise on people, focus on those that have most affected or impacted by the end result of the exercise, rather than on pure expertise.
There isn't much to developing the picture itself -- Simply draw the major concepts and how they inter-relate. In my experience, this is enough to start the conversation flowing; just draw and re-draw the diagram as the "oh, and then there is" comments evolve.
As a rule of thumb, I keep Real Pictures to one page - A3 or a White Board usually. If it gets really messy, redraw the picture. If necessary, drop in prompters to keep the dialogue flowing. Pick a concept and drill down - focus on quantities, qualities, time, costs, constraints:
- How many of these items are there? e.g. How many employees are at that location?
- Does it vary over the day, are there seasonal variations? e.g. Do you have a rush at Christmas, End of Financial Year?
- Will this change or move soon? e.g. Are you changing network providers? When did you last open a new outlet?
- What would invalidate this element or relationship? e.g. What security issues would make a location unviable?
- What do competitors or other equivalents do for this function or outcome? e.g. Does your competitor do this differently? better?
- Are there constraints on the way things should be? e.g. Does the government enforce strict reporting on certain items?
It's important that you don't constrain the actual process and format too much. The objective of this exercise is to get all the stakeholders together and to articulate a landscape that everyone understands. The end result should also be something that all stakeholders reasonably grasps and agree to.
At this stage, try not to abstract too much - early abstraction can lead to some necessary detail or important corner-case being lost. If a specific location, or server, or piece of software is mentioned, it may have special significance.
In one particular instance I found a number of stakeholders would refer repeatedly to a specific printer - i.e. the actual device itself and where it was located. Most of the project team didn't see any significance of this, myself included. The model and design assumed it was a standard office printer, the same any printer we used in our day to day. When we eventually dug deeper, it turns out this specific instance was special - it enabled secure printing. The way the device functions is such that a human operator never gets to see the contents. If you've ever received a bank PIN in an special envelope, then it's probably from a pretty similar device.
Not only was the fact that this device was special important - but also many of the project participants hasn't realised the significance of the information being send by this device. The communications that ended up at this printer required an extra level of security; it has a very special significance. Up until that point most of the team hasn't considered that special, but clearly it was. Similarly, the end-users thought that information was obvious. Suffice to say, staying concrete at this stage usually is the best default - Abstractions will flow later.
A Real Picture sounds trivial, in fact you might be over-the-top to call it a technique. Either way, it's a low impact exercise with a lot of benefits. At the end of the exercise you have:
- A starter for the vocabulary that is in use. Perhaps more importantly, you've got a vocabulary that a variety participants are at least familiar with.
- An outline of scope. With the Real Picture you should be able to draw a line around what's "in" and what's "out". If something's ambiguous, explore that some more. If scope has been defined as part of a project, this can be a useful validation point.
- A fallback. If you hit a road-block or misunderstanding later, you might be down in some detail and a representation that some might not understand. If you have this, you can ask "but didn't we say these relate in that other diagram?". It's not the absolute source-of-truth, but it's a useful pivot.
- An understanding of the various perspectives of the stakeholders. Usually the Real Picture will help illustrate the natural focus and bias of the stakeholders.
- If the participants can't understand or arrive at a picture, then the composition or overall scope needs addressing. Every participant in the project or domain should at least be able to understand the A3 view of what you're trying to realise.
Jon
Wednesday, July 7, 2010
Customizing large industry standards
THE PROBLEM
Throughout our consulting engagements with corporates attempting to implement industry standards, two predominant issues keep recurring:
1. Customization of the standard means forking from the standard. Taking updates from this point is at best extremely labor intensive, and at worst impossible.
2. Very few organizations want to implement an entire standard from day one. A more typical usage is a phased approach with core services incrementally released over several years. Absorbing the entire standard up front results in service consumers being expected to understand and work with verbose and confusing generated artefacts (such as XSDs) filled with far more data fields than have actually been implemented. In fact I'd hazard a guess that there are very few cases where more than half of those data fields will ever be implemented.
WHY?
The integration of data and its structure is a major problem for almost all organizations - the bigger the organization the bigger the problem. We have seen many organizations look towards domain industry standards to provide a more structured blueprint for internal and external integration.
Business-to-business standards tend to be fairly effective because each organization involved has a strong financial incentive to adhere to the standard. An example is the AS2805 standard for Australian EFTPOS transactions, which itself was based on the ISO standard 8583.
Internal integration is, however, another story. If you've ever worked in the IT arm of a large organization you'll know what I'm talking about. Almost all software development is project based, and it's an all too common story to hear that the 'refactoring' was descoped to 'phase 2' which - surprise, surprise - never actually ends up being funded.
However, for those lucky enough (or perhaps unlucky enough) to participate in the rare sort of project that does attempt to implement the 'phase 2', you've probably also discovered that adhering to an industry standard is anything but easy. For starters there are few tools available for dealing with standards. Most open standards are available in XSD format accompanied by MicrosoftWord, and MicrosoftExcel documentation, and most proprietary standards are accompanied by equally proprietary vendor lock-in product sales. Beyond this, no standard is ever quite what you need, either missing large areas important to your business, or overly bloated due to a missing-the-point attempt to please all consumers all at once, and more commonly a little of both. Almost all approaches require the consumer to start with the entire standard and then customize for their needs.
SO WHAT?
As practitioners in this space, and still recoiling from the war wounds, these two problems were front of mind in the inception of http://graft.jodoro.com.
Customization
Every model in Graft is internally stored as tiny deltas. When you extend an existing model in Graft you simply add further deltas to the same base structures. Version control and management is baked into the framework of the application. As such, even if you completely rename an existing class, you can easily adopt any future associations to the base class within your extended model. When and what you adopt is controlled automatically via "releases" configured in the 'specify' application.
Templating
In Graft we provide users two distinct ways to extend a model:
1. The most commonly understood approach we call 'active' extension. This quite literally means you start with the entire base model in your scope. Graft then allows users to explicitly descope parts of the model they don't wish to use. Such descoped classes will remain in the default view, but grayed out. This allows users to see what they have removed, and provides the ability to reverse the removal at a later point. Although users can still see grayed out descoped items, such items will not be included in any exports. An example of an actively extended model can be found here: http://graft.jodoro.com/models/31881
2. We are particularly excited to introduce a new approach to extending that we call 'passive' extension. This allows users to begin with everything descoped, and to draw forward only what is of utility right now. This is the approach we strongly recommend be used when working with large industry standards. An example of a passively extended model can be found here: http://graft.jodoro.com/models/31893
Often organizations wish to borrow from more than one standard. Perhaps, for example, the street address structure is particularly weak in their chosen strategic industry standard. Graft allows any number and combination of active and passive extensions within the one custom model. This is again configured in the 'specify' application.
Feel free to extend these and any other public models within Graft. By default extensions of any models will remain private. You can however choose to make your extended models public too by visiting the admin tab.
Doug
Throughout our consulting engagements with corporates attempting to implement industry standards, two predominant issues keep recurring:
1. Customization of the standard means forking from the standard. Taking updates from this point is at best extremely labor intensive, and at worst impossible.
2. Very few organizations want to implement an entire standard from day one. A more typical usage is a phased approach with core services incrementally released over several years. Absorbing the entire standard up front results in service consumers being expected to understand and work with verbose and confusing generated artefacts (such as XSDs) filled with far more data fields than have actually been implemented. In fact I'd hazard a guess that there are very few cases where more than half of those data fields will ever be implemented.
WHY?
The integration of data and its structure is a major problem for almost all organizations - the bigger the organization the bigger the problem. We have seen many organizations look towards domain industry standards to provide a more structured blueprint for internal and external integration.
Business-to-business standards tend to be fairly effective because each organization involved has a strong financial incentive to adhere to the standard. An example is the AS2805 standard for Australian EFTPOS transactions, which itself was based on the ISO standard 8583.
Internal integration is, however, another story. If you've ever worked in the IT arm of a large organization you'll know what I'm talking about. Almost all software development is project based, and it's an all too common story to hear that the 'refactoring' was descoped to 'phase 2' which - surprise, surprise - never actually ends up being funded.
However, for those lucky enough (or perhaps unlucky enough) to participate in the rare sort of project that does attempt to implement the 'phase 2', you've probably also discovered that adhering to an industry standard is anything but easy. For starters there are few tools available for dealing with standards. Most open standards are available in XSD format accompanied by MicrosoftWord, and MicrosoftExcel documentation, and most proprietary standards are accompanied by equally proprietary vendor lock-in product sales. Beyond this, no standard is ever quite what you need, either missing large areas important to your business, or overly bloated due to a missing-the-point attempt to please all consumers all at once, and more commonly a little of both. Almost all approaches require the consumer to start with the entire standard and then customize for their needs.
SO WHAT?
As practitioners in this space, and still recoiling from the war wounds, these two problems were front of mind in the inception of http://graft.jodoro.com.
Customization
Every model in Graft is internally stored as tiny deltas. When you extend an existing model in Graft you simply add further deltas to the same base structures. Version control and management is baked into the framework of the application. As such, even if you completely rename an existing class, you can easily adopt any future associations to the base class within your extended model. When and what you adopt is controlled automatically via "releases" configured in the 'specify' application.
Templating
In Graft we provide users two distinct ways to extend a model:
1. The most commonly understood approach we call 'active' extension. This quite literally means you start with the entire base model in your scope. Graft then allows users to explicitly descope parts of the model they don't wish to use. Such descoped classes will remain in the default view, but grayed out. This allows users to see what they have removed, and provides the ability to reverse the removal at a later point. Although users can still see grayed out descoped items, such items will not be included in any exports. An example of an actively extended model can be found here: http://graft.jodoro.com/models/31881
2. We are particularly excited to introduce a new approach to extending that we call 'passive' extension. This allows users to begin with everything descoped, and to draw forward only what is of utility right now. This is the approach we strongly recommend be used when working with large industry standards. An example of a passively extended model can be found here: http://graft.jodoro.com/models/31893
Often organizations wish to borrow from more than one standard. Perhaps, for example, the street address structure is particularly weak in their chosen strategic industry standard. Graft allows any number and combination of active and passive extensions within the one custom model. This is again configured in the 'specify' application.
Feel free to extend these and any other public models within Graft. By default extensions of any models will remain private. You can however choose to make your extended models public too by visiting the admin tab.
Doug
Tuesday, July 6, 2010
5 Domain Modeling Tips
It's been a while since we've updated the blog. In the meantime, we've been working hard -- both consulting in the domain modeling space, as well as preparing the latest release of Graft. Given this, we're going to blog less about the technical implementation - and more about using Graft and the process of Domain Modeling.
To start we thought we'd discuss five key tips for building a domain model. We find these are generally applicable, but how and when you implement them will depend a lot on context. Key factors include the time at hand, pre-existing collateral, requirement maturity and the availability of Subject Matter Experts.
Here is a quick run down for now. We'll flesh these out over the coming weeks:
1. Draw a Real Picture
This has a lot of names - In this case it is just an informal sketch that every stakeholder will understand. Typically this is done on a white-board, but paper or online work just as well. Emphasis here isn't on building a refined model, it's about getting the landscape view, establishing an initial vocabulary and providing a basis of scope.
This picture doesn't have any formal semantics, it functions more as a conversation piece to illustrate and start grappling with the domain. Build this using whatever works for the participants. That said, if done right it can be an opportunity to gently start introducing some key modeling concepts and approaches.
2. Goal-Based Workshops
Many modeling exercises risk becoming esoteric. We encourage goal-based modeling that drives towards outcomes. Highlight the gaps, inconsistencies and pitfalls. Uncover the gnarly pieces early. You might not necessarily solve these immediately, but surfacing them is a key outcome.
Often the Goals themselves are simply driven out through workshops and consultation. In the ideal case you can derive Goals from sources such as strategic initiatives, requirements definitions, existing systems and business processes. If these aren't available or mature, focus on how the outcomes will be validated.
3. Deep-Dives, with a focus on Cohesion and Coupling
This is really where the heavy-lifting of modeling comes in. A deep-dive is a concentrated effort on an important aspect of the domain. Often the Goal-Based Workshops will help to drive out the topics.
The frequency and function of these will need to vary based upon the topic involved. Our ideal is a set of tight iterations on each topic. However, a common compromise is to run a deep-dive session, have the modeler refine the results offline and then a hold final session to reconfirm the outputs.
4. Revisit the Landscape
Deep-Dives generally uncover a lot of detail, however, before going to far down a rabbit-hole it's good to pause and check "is it really relevant?". The key question here is always "what's the context?".
Ideally you will have introduced Business Processes as a requirements driver earlier. However, if this hasn't occurred, this is a key opportunity to introduce them. The Business Process will let you ask a lot of key questions around model attributes and relationships - Are they used? When and what for? This can lead to surprising results, often for aspects the participants take for granted.
5. Iterate, Evolve & Refine
No model is ever "ultimately complete"; in fact it can be damaging to have this as a goal. It's important to plan for how a domain model can and should change over time. This can include small things - from notes and documentation on why decisions have been made and considerations for the future - through to a full roadmap on how the model needs to progress from here.
That's a very quick wrap for now. If you have any other tips, let us know and we'll see about exploring them over the coming weeks.
Jon
To start we thought we'd discuss five key tips for building a domain model. We find these are generally applicable, but how and when you implement them will depend a lot on context. Key factors include the time at hand, pre-existing collateral, requirement maturity and the availability of Subject Matter Experts.
Here is a quick run down for now. We'll flesh these out over the coming weeks:
1. Draw a Real Picture
This has a lot of names - In this case it is just an informal sketch that every stakeholder will understand. Typically this is done on a white-board, but paper or online work just as well. Emphasis here isn't on building a refined model, it's about getting the landscape view, establishing an initial vocabulary and providing a basis of scope.
This picture doesn't have any formal semantics, it functions more as a conversation piece to illustrate and start grappling with the domain. Build this using whatever works for the participants. That said, if done right it can be an opportunity to gently start introducing some key modeling concepts and approaches.
2. Goal-Based Workshops
Many modeling exercises risk becoming esoteric. We encourage goal-based modeling that drives towards outcomes. Highlight the gaps, inconsistencies and pitfalls. Uncover the gnarly pieces early. You might not necessarily solve these immediately, but surfacing them is a key outcome.
Often the Goals themselves are simply driven out through workshops and consultation. In the ideal case you can derive Goals from sources such as strategic initiatives, requirements definitions, existing systems and business processes. If these aren't available or mature, focus on how the outcomes will be validated.
3. Deep-Dives, with a focus on Cohesion and Coupling
This is really where the heavy-lifting of modeling comes in. A deep-dive is a concentrated effort on an important aspect of the domain. Often the Goal-Based Workshops will help to drive out the topics.
The frequency and function of these will need to vary based upon the topic involved. Our ideal is a set of tight iterations on each topic. However, a common compromise is to run a deep-dive session, have the modeler refine the results offline and then a hold final session to reconfirm the outputs.
4. Revisit the Landscape
Deep-Dives generally uncover a lot of detail, however, before going to far down a rabbit-hole it's good to pause and check "is it really relevant?". The key question here is always "what's the context?".
Ideally you will have introduced Business Processes as a requirements driver earlier. However, if this hasn't occurred, this is a key opportunity to introduce them. The Business Process will let you ask a lot of key questions around model attributes and relationships - Are they used? When and what for? This can lead to surprising results, often for aspects the participants take for granted.
5. Iterate, Evolve & Refine
No model is ever "ultimately complete"; in fact it can be damaging to have this as a goal. It's important to plan for how a domain model can and should change over time. This can include small things - from notes and documentation on why decisions have been made and considerations for the future - through to a full roadmap on how the model needs to progress from here.
That's a very quick wrap for now. If you have any other tips, let us know and we'll see about exploring them over the coming weeks.
Jon
Subscribe to:
Posts (Atom)