The Architecture For This Project Is Out Of Scope

not my monkey

It is fair to say that the project in question has not started well. A lengthy incubation period executed in isolation from business and delivery partners has seen resources consumed in exchange for very little demonstrable progress, and high-level commitment waning. Faced with challenges like these, project managers will often retreat into their defensive belief systems — the heavily-dogmatic linear methodologies. Under such circumstances, the project manager looked me in the eye and uttered the incredible words:

  • “The architecture for this project is out of scope for this project.”

This statement dumbfounded me, and made me think. Another literal expression had been made of the natural tensions between enterprise architects (and their broad-context thinking and striving for change-friendly capability delivery1) and project managers (and their project-constrained and limited-context viewpoint and striving to achieve traditionally-measured project success). Variously, while i am:

  • accepting that enterprise architecture does not have a strong track record in the successful application of its own heavyweight frameworks
  • holding strong personal convictions that well-executed P3M (Portfolio, Programme, and Project Management2) has transformative potential for good
  • noting signs that enterprise architecture and P3M practices can deliver better business outcomes working together harmoniously3

…i’m too often seeing struggling projects “remedied” through the application of portfolio-grade heavy management and governance techniques, with the primary effects being:

  • greater focus on time and budget, at the sacrifice of scope and benefit
  • more expensive (in terms of both effort, money, and time) project contols
  • blossoming of the infamous opportunities register

For this to change, the culture of projects, and the culture of enterprise architecture, needs to change, meld, and align — a new conversation is needed, needed to lift every project’s context for decision-making up to at least the portfolio level.

It was much later before i established the link with this item doing the rounds on Twitter:

  • “Not my problem” in Polish is “nie moj cyrk, nie moje malpy.” Literally “not my circus, not my monkey.”4

There it is, the crux of the execution-time tension: “not my circus, not my monkey“.

As we step down through the layers of P3M we find portfolios (selected circuses and selected monkeys), programmes (small numbers of circuses, small numbers of monkeys), and projects (one circus and some of its monkeys).

With its enduring concern for the broader context, enterprise architecture is concerned with the enterprise: with all the circuses, and with all the monkeys.

  1. Enterprise Architecture in 140 Characters, Brenda Michelson, Elemental Links, http://www.elementallinks.com/2010/02/17/enterprise-architecture-in-140-characters/
  2. The Project, Programme and Portfolio Management InfoKit from JISC http://www.jiscinfonet.ac.uk/p3m provides a good overview of P3M.
  3. Bittler, R.S. (2012) Best Practices for EA and PPM Integration Toward Improved Business Outcomes, Gartner Research, Article ID #G00237525, http://my.gartner.com/portal/server.pt?gr=dd&ref=shareSummary&resId=2173516 gives a pragmatic and actionable set of best-practice suggestions for aligning enterprise architecture and P3M teams and initiatives.
  4. @howardtaylor on Twitter at https://twitter.com/howardtayler/statuses/260944798167482368

Advertisements

Anti-Collaboration and the Hungry-Hippo Secret-Society

hungry hipposTrapped inside their plastic bubble-land, committed forever to competing for access to worthless inedible coloured plastic baubles that represent to them a measure of status, members of the hungry-hippo secret-society collaborate in isolation from the world around them, waiting for their plastic environments to be exhalted as wonderful contributions to the mission, the project, the policy, the business requirements, the strategy, or whatever it is these self-professed dippy maven-muppets believe they are achieving without consultation, without reflection, and without harnessing the power and good intentions of the collective.

These hungry hippos keep personal copies of their work in secretive locations, sharing only with the annointed few and the hierachico-politically-expected ones, those whose acknowledgement and annunciation they crave, and whose places they one day long to occupy by right of gift, devotion, and secret pledge.

Other hungry hippos that work this way are simply terrified, or wanna-be perfectionists, or incompetent.

From both camps, the disruptive and mischevious enterprise architects that manage to prise away DRAFT-watermarked embargoed final copies of the hippo’s work receive these gemmy gifts with form-letter strongly-worded warnings, such as:

  • please do not distribute this document any further, not to anybody else, as it is still under discussion by the team, and if this gets out before it’s been signed off then we are going to be punished heavily, especially you!

This is patent nonsense, and exactly the opposite of what’s needed: the sooner these works are distributed and visible widely, the better is the result for the initiative at hand, for the organisation involved, and for the World.

These hungry hippos must be stopped, counselled, and taught to collaborate using wiki, coauthoring, and socially-enabled tools.  As it stands, they’re routinely and repeatedly turning out expensive substandard frog-in-a-well constrained work as finished art, too late every time to undo or avoid.

Tear down the walls of the hungry-hippo secret societies!

Idempotence and the Dead-Letter Office

Dead Letter Office

“Dead Letter Office” is a lovely and hearty shiraz sourced from the McLaren Vale and Padthaway wine regions of South Australia.  While drinking a bottle of Dead Letter Office recently i started to sketch this post about asynchronous message-based integration architectures.  The dead-letter-office concept is an important form of exception-handling, though it contributes to complexity in delivering perfectly-reliable and predictable end-to-end integration services.

Asynchronous messaging has many benefits, not least the loose coupling it affords between message publishers and message subscribers.  Publishers are completely ignorant of their subscribers, and need only to honour the publication contract and behave well.  Between the publishers and the susbcribers usually are stateless mediators responsible for:

  1. message collection, dequeueing the message and validating its semantic correctness and that it is well-formed
  2. message filtering, discarding messages in which a particular subscriber is uninterested (e.g., for a subscriber that is interested only in messages relating to a particular site or a particular group of people or cannot process future-dated events)
  3. message transformation, changing the message to be appropriate in form and content for consumption by the subscriber (e.g., removing attributes the subscriber is not entitled to receive, masking attributes the subscriber is not allowed to receive in full fidelity, formatting or calculating attributes based upon other attributes in the message, or enriching the messages by making callouts to other systems)
  4. message delivery, delivering the message to the subscriber endpoint and negotiating the delivery outcome
  5. exception handling, responding appropriately by raising warnings or throwing errors when processing exceptions are encountered at any of the previous steps
  6. logging and auditing, to provide instance-tracking and to feed event-monitoring and alerting systems

If an exception occurs somewhere in that chain during message processing then the cause is either:

  • system exceptions, such as when a service is down or unreachable, or when the integration user cannot log into or has lost its permissions to perform the functions it requires in the target system — in general, messages encountering system errors will be delivered successfully on some subsequent attempt.

…or:

  • semantic exceptions, such as when a message schema is invalid or when the data contained in a message are invalid for the subscriber system (for referential-integrity, length, format, or other reasons) — in general, messages encountering semantic errors will never be delivered successfully until the data issues are corrected, and a human is often needed to make those corrections.

…and in either case there will be some exceptions that have been seen previously and some exceptions that are being encountered for the first time.  This boils down to a requirement for three message-queue buckets:

  • Dead Letter Office: there is something wrong with these messages, and they will never be delivered successfully as they are.  If there are referential-integrity issues with a subscriber system then the messages will generally need to be republished from the system of record.
  • Try Again Later: something environmental is wrong, so delivery of these messages should be attempted again, later.
  • Spooky Inspection: some messages cannot be delivered, but the reason is unknown, because it has never before been seen — these messages require inspection and analysis to categorise them either for the Dead Letter Office or for the Try Again Later queue-buckets.

Across the end-to-end message-processing implied by an exception causing an en-route message to be delivered into a queue-bucket exist basic quality-of-service requirements in the scope of the universe of messages for conditions to be honoured such as:

  • ordering
  • timeliness
  • security
  • transactions

…and achieving this implies that the message-delivery mediators must work with these exception queues in order to see, potentially, the:

  • Try Again Later queue-bucket emptied before taking fresh new messages from the publication-queue trunk
  • Spooky Inspection queue-bucket emptied before taking fresh new messages from the publication-queue trunk
  • Dead Letter Office queue-bucket either ignored forever (if the wider environment is sufficiently-well transactional) else constructed with micro-queues and feedback from the successfully-processed messages so that entities whose messages have been dead-messaged are released once the transactional safety of the item has been confirmed

These scenarios are complex.  A pragmatic and optimistic approach to dealing with them perhaps boils down to these two requirements of an message-based integration architecture:

  1. Idempotence is Mandatory: everything must be idempotent, so that the same message, or the same latest-state message, can be republished multiple times without harm.
  2. Staleness-Detection is Mandatory: subscribers must have some concept of stale messages, and if subscriber components do not have a capability to enforce a staleness concept then they must be prefaced by integration agents that can make staleness decisions on their behalf, effectively operating as a super-simple resequencer.

We’re working hard to get the data-integration scenarios working really well, while at the same time noting the industry drift toward higher-level integration solutions that operate at the application and the process level.  Getting the right patterns down and used (and reused) is the key to this.

Hohpe and Woolf’s Enterprise Integration Patterns is an indispensable guide in this regard, and it’s crucial those patterns are set down into reusable component-based recipes embedded into each organisation’s integration-architecture repositories and application-development frameworks.

For more details, refer to the Enterprise Integration Patterns website at www.enterpriseintegrationpatterns.com

First-Mover Advantage and BYOStuff

I still feel like a bit of a wanker pulling out my ipad in meetings…

…lamented the author of a recent tweet i found interesting because the stigma of the tablet has all but disappeared from my:

  • Work: where tablets are quite prevalent, with about as many being company-provided as are personally-owned devices, and the defining behaviour in my jaundiced eyes is not producing a tablet in a meeting or a workshop (that’s a good thing) but having it sitting on the desk as a beacon of status and having, or being fiddled with impotently, or being neglected as too hard in favour of a paper notebook, rather than being used for productive common good work.
  • Community: at conferences and other events the tablet ratio is much higher than at my work, but so too is the ratio of productive use in note-taking, micro-blogging for the benefit of remote attendees and colleagues back home, researching background information on the fly, fine-tuning presentations, and for sharing with other attendees.
  • Industry: Stephen Prentice, in his Gartner research note Technology Trends That Matter† that for media tablets:
    • First-mover advantage has already passed [and] it no longer garners the attention and buzz that it did in 2010, to the point where not using a tablet is likely to be more commented on.
  • Society: during my morning public-transport commute to work it’s become unusual not to see at least one passenger using a tablet, though they’re generally consuming news, social media streams, or ebooks rather than participating or being involved in content creation.

This tip-of-the-iceberg aspect of consumerisation-meets-the-enterprise in Bring-Your-Own-Device is quickly being echoed in non-physical domains, such as:

  • Bring Your Own Identity, whether that’s a governmental or commercial identity-proofing service, or whether that’s from something like Facebook or LinkedIn
  • Bring Your Own Productivity Software, from the likes of Google, Dropbox, and Evernote and from the as-a-service subscriptions to teamware such as Basecamp or code libraries on github.

With the first-mover advantage already well behind us on the device front, it’s likely we’ve already missed the boat in understanding (first priority) and providing guiding governance (second priority) how to get the best for everybody out of the BYOStuff stack.

† Prentice, S. (2011) Technology Trends That Matter, Gartner Research, Article G00212538

Christmastime: Horrible Usability and Horrible Process.

None of the fundamental kindnesses extended to normal web users are present in the “Request Absence” functionality in the PeopleSoft HRMS we’re using.  This mouse-intensive, navigationally-clunky, barely-usable interface would be rejected by any of our user-experience teams in the outside-customer-facing projects.  Three kinds of thing frustrate me most about the horrible experience of having to comply with a make-your-christmastime-plans-before-the-end-of-October ruling:

  • Experience: the entire experience seems to be geared up so the somebody with an extremely unusual absence-type and absence-take-type will be able to submit their request for approval.  However, catering for this one imaginary person and this one fascinatingly-unlikely combination of leave/absence conditions guarantees the 99% of cases (take some annual leave or report sick time) an extremely and needlessly difficult time, and mainly because of the usability shortcomings.
  • Usability: the request-absence functionality doesn’t do any useful defaulting, and lacks even the basic courtesy of defaulting the end date to the same month and year as the nominated start date.  For requests that fall due in the next calendar year, users are presented with a drop-down Year list that starts in 1900 and runs through until the year 2100!
  • Process: we have about 6,500 employees, and every one of them is having to look at their calendars and figure out which are the public holidays and which are University holidays and which need to be taken as annual leave and then pass successfully through the dreadful HR self-service environment — i’m struggling with the participation and compliance costs that must be associated with this exercise, and believe there is a Simply Better Way.

For all this, i’m nearly finished the task of loading up my leave for summer, and i really hope it matches with what my family wants me to be doing two months from now.

Watercoolers, Protectionism, Productivity, and Meatspace.

i’m providing up-to-half-of-full-time enterprise architecture consultancy to a major effort that will see the really-old PeopleSoft Student Administration 7.6 environment at my site upgraded to become the latest-available really-new PeopleSoft Campus Solutions 9.0, and it’s a large project with about sixty people working together in one spacious floor of a commercial building that housed previously a call centre, so there are lots of windows, and it’s quite open-plan, and there are network ports everywhere (but the network is really slow).

In some ways i’ve struggled with the transition from long-running core-relationship mode to project-context mode, and i’m wondering if some of the resistance from the wider project team to the mention of social networking tools like Twitter, Yammer, wiki, blogs, and Google Wave is more about the shorter-term scope of the monstrous and pressing tasks the team faces.

Where a community is unaccustomed or resistant to social networking (and the individual reasons range wildly from fear of recrimination for saying something silly, to not understanding anything of the value proposition, to feeling too busy to participate, to outright believing it’s a waste of time) i’m unable to rely upon the ambient intimacy that informs many/most of my relationships. This also means that core messages, and core advice, are being sought through and are able to be provided by only traditional channels such as email-mediated and meatspace-mediated interactions, neither one of which is:

  • social/public
  • challengeable/addressable
  • replayable/documentary
  • updateable/commentary

To address this, today i spent the first of my at-least-half-a-day-a-week working in-person on the floor, seeking to establish and engage in the watercooler interactions that are missing in this context. It’s a broad open-plan workspace, and it’s very different from the small and quiet and intense and ordered office of ten people where i normally perform deskwork. The experience was interesting:

  • the dividers between desks/cubicles are at least a foot lower than i’ve experienced before, so i found myself for the first hour or so working with my head forced down near my laptop, avoiding exposure and eye contact and trying to give others privacy
  • there’s a lot more noise than i’m accustomed to, though most of it is the sound of industry, and it felt kind of good to be exposed to it, though it felt simultaneously like i was trying to work in a cafe or an airport lounge
  • there’s a lot on my plate at the moment, and it’s hard to knuckle down and be productive and concentrate on and plough through pressing work (and, as a visitor placing myself in the environment in order to be more directly accessible, it wasn’t appropriate to do the headphones thing)
  • talking with people is very productive, and can resolve/explain/mentor/correct/demonstrate/challenge/assist much more quickly than with traditional remote interactions…
  • …but there’s nothing left behind, so behaviours such as the email-to-wiki pattern are invalidated, not least because we no longer enjoy a strong story-telling oral culture

i’m going to spend a lot more time up there on the floor within the big project team, and i’m left with the feeling that the limitations and downsides are chiefly my own, and it’s something of a personal challenge to overcome them, at least for my half-day-or-so each week in the environment!

Actionable Architecture and Other Matters

Welcome to the actionable architecture weblog.  Just like any other blog, it’s going to be a work in progress forever as what is merely a collection of one person’s opinions are described in articles that could be triggered by anything at all: work, life, updates on Twitter, test cricket, the weather, news, music, anything. However, what is intended to link all these articles together is that they are in some way concerned with:

  • actionability
  • enterprise architecture
  • patterns and standards
  • business process management
  • modelling
  • industry analysis and trends
  • business technologies
  • social networking and collaboration
  • methodologies for design and development
  • architecture frameworks
  • enterprise architecture as a profession
  • best practices for actionability

i’m a latecomer to the blogging, though a wiki and Twitter enthusiast, and the format shift will be at once interesting and challenging.