Change: Incrementally Making Things Better (Part 1)

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/change_incrementally_making_things_better_part_1.htm]

Part 1 (Intro)

Part 2

Part 3

Part 4

Part 5

Part 6

"You can be an entrepreneur inside a big company." anonymous coworker

A background on change

Carl stormed back to his desk, furious that the Jim the CIO blocked yet another profitable infrastructure enhancement. Carl had spent the last decade in a Fortune 500 company’s IT shop seeing brilliant minds and expensive consultants build a massive enterprise architecture that could handle hundreds of similar transactions a second. Carl eventually switched jobs, and hoped to bring his successful experiences with him. But Jim, obsessed with keeping the department alive during the current recession had no ears for anything Carl could say. Jim just wanted to keep the business sponsors happy, Carl just wanted to make things better - they were both professionals with good motives - but they often left meetings just angry at each other.

If you're a CIO, you're probably not reading this. So the real question is, what can Carl do?

Carl wants to create positive change. Changing a culture isn't just about "more" effort, rather it's about how that effort is applied. Who hasn't felt the frustration of spending a month of nights and weekends trying to boil the ocean only to have the initiative rejected with nothing to show for it? Often, a fraction of that effort could be applied more wisely, and to a more receptive initiative, such that you've changed (however slightly) the department for the better.

Three categories of changes

There is a big difference between business features and infrastructure/culture improvements. A business feature is simply some feature requested by the business - such as a new web page to enter data. The business doesn't care how that feature was implemented. An infrastructure change (or "process change") allows you to crank out those business features. Good infrastructure and culture means cranking out and updating more business features. The changes I'm talking about are infrastructure/cultural improvements - not business features.

 Over the years, I've seen three main categories of such changes, listed in order of impact from smallest to largest:

1. Individual - A developer makes a private change to simplify their job. It doesn't need any manager approval, nor does it change their "contract" with anyone else. For example, say every time you resolve a bug, you need to write a bloated email with source control and issue tracker information, and it takes you 20 inconvenient minutes every day. Say 95% of this email is tedious grunt work that is forced upon you because of some misguided process. You could sooner move Mount Everest than change this process, but life is what it is. So, you write some quick tool to query the source control and issue tracker databases, and generate most of that email. You're still sending out the exact same emails, so the bureaucracy overseers are satisfied, but now you're creating it in 60 seconds. This is an individual change.
2. Department: Private - The department (or team) has some thorn in their side, and as a team they can privately fix it "under the radar" - i.e. they don't need truckloads of cash, months of schedule time, other departments to change how they do things, or anything that would require the CEO's approval (the CIO may need to at least be kept in the loop). For example, perhaps they want to purchase a cheap tool, adapt a new convention, streamline their internal process, code generate their data layer, adapt a new development practice (OOP, unit testing, agile, etc...), start code reviews, refactor their code, make reusable blocks, etc... These are all types of initiatives that a non-micromanaged department can internally fix themselves.
3. Department: Public - The company's flagship application has just been hacked - some key developer five years ago forgot to do any client-side validation ("we don't have time to write the code re-validate on the server"), and everyone was so busy that they just copied & pasted that developer's code pattern - to all 200 web pages. The CEO herself holds an emergency meeting to tell IT that these security holes must be fixed, top priority, drop everything else. This change is still made by the developers in your department, but it's being tracked like an official business feature with status being reported directly to upper management in other departments.

For simplicity, let's call these cases #1, #2, and #3. Each has its pros and cons.

#1 changes are a good start, but by definition they don't directly benefit others. Private, individual improvements will no more change the course of a company than a dolphin flapping its fins will change the course of a battleship. Ideally, you'd either fix the root-cause with #2 at the department-level (say improve the process so that it no longer require those tedious emails), or you'd want to make an individual change easily promotable to #2 (say make your email tool a public utility that everyone can leverage).

 

Let’s skip #2 for now and jump straight to #3. These changes are great, but the sheer size of these makes them very rare and slow by definition. You maybe only get a handful of such initiatives a year, and they often need to go through layers of approval and red tape which makes them slow. These usually require CIO approval, so they're going to be very business-facing, and they usually take for granted the development infrastructure is already in place. (Think of developers smacking their head as some manager wonders out loud, "I just assumed that because you already did that dynamic rendering for one client, that we can easily apply it to all 500 clients - right?")

 

Back to #2 - it's these private department improvements that are the bread and butter of real company changes. #2 often mitigates the need for #1, and provides the necessary foundation for #3. Without #2, the individual changes of #1 become like building sandcastles on the seashore, and the huge changes of #3 get cut to pieces during actual implementation because the infrastructure can't handle its visionary goals.

 Because #2 is the key to real change, it's what this article will focus on.I'm thinking of smaller, gradual changes - adopt a new tool, automate a process, refactor critical code that everyone is scared of, empower some department so they can handle a task themselves without coming to you, create reusable code blocks, introduce a new quality control measure, add a new architectural component, bring in a consultant instead of building or researching a component yourself, etc... Each of these are small by themselves, but eventually they add up to greatness.
 

NEXT: Part 2

BOOK: Emotional Intelligence

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/book_emotional_intelligence.htm]

I have always admired those who have a firm grasp over their emotions, and I don’t mean the Spock-like characters that appear dead to their feelings. This eventually led me to read Daniel Goleman’s #1 bestseller Emotional Intelligence. While the book was filled with good concepts, two really stood out to me:

• Verbal Bias - Many “techy” people have a verbal bias in their communication. They think it is more “logical” and “objective” to just go with the words that were written or said, and not taint that with someone’s non-verbal inflections. But such a bias is not more logical, quite the opposite. The majority of someone’s communication is non-verbal, and it is illogical to ignore additional (and relevant) information, therefore it’s actually very illogical to have a verbal-bias that ignores - or doesn’t attempt to understand - someone non-verbal communication.
• Emotional Hijacking – I’ve seen coworkers freak out. Rands points out that this means that they care. Great, but I don’t want to be the one freaking out where the frustration of a bad project gets the better of me. It helps to know that most of us have some sort of trigger that will instantly hijack our emotional state and make us go into a state of uncontrollable rage. For example, don’t tell me that “I don’t have time to write unit tests”.

Good emotional exercises that came to mind as I read:

1. Think of every emotion you can. You fail if you can only list “happy” and “sad”. You also fail if you keep listing physical states (“tired”, “hungry”) or mental states (“frustrated”, “focused”). I failed on both accounts. Then I went to google for “list of emotions”.
2. Think of 20 coworkers. What emotions do you think each are feeling? “Rejected” because no on adopted their initiative? “Excited” because they’re on a hot project?

The book has a lot of good zingers:

1. “the brain has two memory systems, one for ordinary facts and one for emotionally charged one.” (pg. 21)
2. “people who cannot marshal some control over their emotional life fight inner battles that sabotage their ability for focused work and clear thought.” (pg. 36)
3. “many people with IQs of 160 work for people with IQs of 100.” (pg. 41)
4. 5 fields of emotional intelligence: Knowing one’s emotions, Managing emotions, Motivating oneself, Recognizing emotions in others, and Handling relationships (pg. 43)
5. “New solutions and fresh ways of seeing a problem do not typically come from worrying.” (pg. 67)
6. “People who are optimistic see a failure as due to something that can be changed so that they can succeed next time around” (pg. 88)
7. “People’s emotions are rarely put into words; far more often they are expressed through other cues.” (pg. 96)
8. There are at least three ways of displaying emotions: minimizing, exaggerating, and substituting (pg. 113)
9. “The two cardinal sins that almost always lead to rejection are trying to take the lead too soon and being out of synch with the frame of reference.” (pg. 123)
10. “Stress makes people stupid.” (pg. 149)
11. “Many things people do at work depend on their ability to call on a loose network of fellow workers” (pg. 161)
12. “[stars] do the work of building reliable networks before they actually need them. When they call someone for advice, stars almost always get a faster answer.” (pg. 162)
13. “There are no grades given in Self Science; life itself is the final exam.” (pg. 268)
14. “The emotional mind is far quicker than the rational mind” (pg. 291)

BOOK: Economics for Dummies

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/book_economics_for_dummies.htm]

 

I have never met a developer who said they had enough time to "properly" finish their project. Sure, everyone starts the project with dreams of how this will be momentous - somehow the stepping stone to curing cancer and world hunger - but then reality sinks in and the team scrambles to make the best of their limited time.

And that's where economics, the science of how people deal with scarcity, comes in. I had to take a micro and macro Econ course back in college for my engineering degree, but back then it was just an 8:00am commitment. When I was reading Joel on Software's blog, he picked my interest with economics again with his talk of compliments and supplements, vendor lock in, the chicken & egg problem, etc... So, I got a copy of "Economics for Dummies". I saw it as Part II of The Complete MBA for Dummies.

Living up to the "Dummies" genre, it was an easy read. The concepts of utility, marginal revenue, return on investment (ROI), consumer surplus, diminishing returns, and supply and demand are good things for any developer to know. Much of this may seem like common sense in today's world, but a book helps one to articulate what their head thinks is common but they can't find the words for.

Besides assisting with prioritizing features and making calculated risks, it helps a technical person continually appreciate the business side of things.

 

Dealing with an IT bureaucracy

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/dealing_with_an_it_bureaucracy.htm]

A background on bureaucracies

By definition, large companies employ a lot of people. Large companies also tend to become bureaucracies in order to manager all those employees. Therefore, lots of people are working in a bureaucracy.

At least in my experience, bureaucracies usually:

• Require a lot of red tape, which in turns hurts cross-team coordination
• Split people into specific roles and teams ("separation of duties"), so they must constantly coordinate with others
• Encourage escalation of problems instead of individuals finding immediate solutions.
• Require many people to approve a specific decision
• Regulate most activities
• Are very risk-adverse, and hence punish risk more than it awards innovation
• Are very big

Bureaucracies tend to have a negative connotation, sort of like living out a Dilbert comic strip. So, if bureaucracies have such a bad rap, then why would a company ever become one?

• As the company grows, it's an understandable way to govern masses of people.
• If a company employees micro-manager personalities, the bureaucracy is a natural consequence.
• Big companies can't afford risk: People will sue them, Hackers will attach the security of their systems, millions of users depend on their product, etc...  So bureaucracies use red-tape as a safety net.

To help appreciate the positive tips of how to deal with a bureaucracy, let's first explore some futile approaches.

• Whining sessions - Everyone will complain that it's a mess, but nothing will get done.
• Emotional appeals - Bart in IT feels your pain that your 1GB workstation is slow, but he's not allowed to give you an upgraded memory chip.
• Asking someone to do something outside their role - Ok, so maybe you get lucky and Martin, the local DBA, helps install a virtual machine, but in general you can't expect this.

Practical Tips

Besides common sense (be polite, do your homework, communicate clearly, etc...), here are 11 tips for dealing with a bureaucracy:

1. Know what each role should do. In a bureaucracy, each person has their role, and that is all you can expect them to do. It is not an entrepreneurial start-up where everyone does everything they can to make the team succeed. While you may get occasionally surprised, you can't expect someone to go above and beyond. For example, perhaps only the legal/procurement team can purchase tools, or only the security access team can give your user account rights to the Customer database, or only Support can view production data, or only the Graphics department is allowed to create the official icons used in the application (despite you could do it yourself with your 5 years of hobbyist Photoshop skills), etc... You can't expect a fish to fly. Each person has a role, and a bureaucracy beats people into fulfilling just their specific role.
2. Know how information is passed between departments. A major side-effect of a bureaucracy is that it takes many departments to do even simple things. This means that even the most mundane request could bounce through 5 departments like a ping-pong ball. How does the request get passed along? Is there some official ticket/issue software that tracks everything, are official emails sent, does it only happen in face-to-face meetings? For example, if nothing happens until "a ticket is opened", then learn how that ticket system works and be prepared to open tickets. Even if you go directly to your buddy in security access to help resolve something, he'll still need a ticket to track his time against. A dozen hallway conversations with the senior VP herself may have less impact than that one ticket you actually submit. You need to leverage these communication channels, else you're just screaming in the wind.
3. Allow time for requests to percolate through the system. Because even a simple request may need five signatures from five departments, requests can move slower than molasses. Therefore, when you're designing a solution, try to determine what ticket requests you'll have as early as you can, and then submit those as soon as you can. While those tickets are dripping through the system, you can flush out the internal details of your design. Sometimes, submitting the ticket "reserves your place in line", and you can update the ticket with more details as you find them (think of it as giving that department a "heads up"). The last thing you want to do is calculate every possible edge case, and then (two days before the deadline), submit a flood of ticket requests.
4. Let the person causing the pain feel the natural results of it. - Where possible, when someone is blocking the project due to some artificial rule, let them feel the natural consequence of that project being blocked. I.e., don't enable bad behavior. For example, if someone in sales keeps entering data the wrong way, consider not enabling them by writing an automated script to continue cleaning everything up. If you do, they'll essentially think that their current approach is working, and why would they ever change?
5. Distinguish between roles and titles. Say "managers don't have access to source control", but you (a new manager) really need access. If your company allows one person to play multiple roles, then perhaps you can pass muster with "I'm not asking you to change the security chart and give managers access to source control, rather I'm trying to (temporarily) contribute with the developer role, which needs access to source control."
6. Make red tape problems known to your manager. Don't be malicious or whiney, but simply report the facts to your manager. "I can't complete the report module until procurement gets the Amazingnator rendering engine." Don't just eat it yourself and try building your own Amazingnator rendering engine over the weekend. Sure, it may make you today's hero, but the continual burden of not getting the proper resources because another department is broken will leave you bitter and exhausted.
7. Know the people who approve the tickets - Even the biggest bureaucracy ultimately boils down to individual people. If a ticket is languishing in the Data Services department, it's beneficial to ask Marge from Data Services if she's heard anything about the ticket, and if she has any advice.
8. Where feasible, keep skills in your own team - Because cross-team coordination is often slow in a bureaucracy, having the skills in your own team such that you don't need to go to another team can be a good ace-up-your-sleeve. I remember a web consulting gig when .Net first came out where the manager split development into separate roles - C# guys and SQL guys. He thought this would be easier to staff ("you only need devs with one skill or the other"), and result in higher quality ("each dev is an expert in their niche"). The problem for applications at that time was that C# and SQL were so intertwined that one without the other was like trying to run with only one leg. Coincidentally, some sub-teams secretly had their own SQL guys, and those teams flew.
9. Stack the deck - You know when you ask the procurement department to purchase a tool, that they're going to want forms filled out - how much does it cost, what's the business justification, does your manager approve, are there alternatives, etc... Ask upfront what forms they need, and have those ready. Else, you may be "sent to the back of the line", and need to wait days (weeks!) to get your chance again.
10. Focus on what people can do without spending money. This is related to knowing what each role does. Maybe that department can't give you what you your request because it costs money (more staff, purchase a tool, additional hardware, etc ...), but they can do helpful things that cost nothing, such as: provide "temporary" security access so you can run a test yourself, let you borrow a resource that they're not currently using (such as a VM server you can log into), switch the order of two tasks that has no impact on them but simplifies your life, offer information such as who you should unofficially talk to "make it happen". Ironically, although time is money, it is often easier in a bureaucracy to get time than get approval to spend money.
11. Mentally prepare yourself that it is inefficient. People can psychologically deal with crap if they're in "I need to deal with crap mode". So, just set your expectations and prepare yourself that bureaucracies are slow and inefficient. If things do go well, then congratulations, but if not, at least you'll be psychologically prepared for it.

See Also:

Book: Making Things Happen, Book: Managing Humans

Most projects fail for non-technical reasons

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/most_projects_fail_for_nontechnical_reasons.htm]

I’ve seen projects fail, and it sucks. It sucks team moral, it sucks resources, and it sucks energy from other projects. Granted, there are degrees of failure, but generally a project is considered a failure when it is significantly over schedule, over budget, under quality, ships with too many bugs, or simply never even ships at all. I wouldn’t consider a project to be a failure if afterwards you find a more optimal way, or management throws the completed project away because of new business direction – in that case the project itself still succeeded.

It’s well known, and well experienced by developers, that many software projects fail. In the 80’s or 90’s, insufficient technical skill often contributed to project failure – the mere act of programming was complicated, the frameworks still young, even finding the right syntax was challenging. Today, there are powerful frameworks, open source projects to pull from, tools to assist almost any technical problem, Google, and years of precedent for most types of projects. Some may say that mere “coding” has become so easy that it’s as if platform companies like Microsoft are trying to make all developers dumb, or at least lower the bar so that anyone can develop.

Of course, projects can still fail today due to insufficient technical skills, but most of the time these days, they seem to fail for non-technical reasons: constantly changing requirements, poor communication among teams (because today’s complicated projects require lots of cross-team coordination), scope creep, bad estimation not allowing the team enough time to do it right, insufficient development infrastructure not allowing the dev team to actually build and deploy code, bureaucratic red-tape that prevents the team from procuring the right tools, poor team chemistry that results in internal conflicts, poor project management, lack of user input, etc…

Ironically, even if the project fails for these non-technical reasons, it still shows up on the technical folk’s desk. Ultimately, some manager or business sponsor hammers the developers with “Why couldn’t you build this?” Granted, a star technical team has a much better chance to handle the rapidly changing requirements, do more work with less time, or build their own tools and infrastructure “under the radar”.

The point is to be a star dev, you must push through successful projects. A dev who only does “moderate” technology on a profitable project will be viewed as far more successful than a dev who does “cool” technology on a failed project. These days, because most projects fail for non-technical (i.e. “soft”) reasons, developers who want to be stars should invest something in their soft skills.

Three cautions with mocking frameworks

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/three_cautions_with_mocking_frameworks.htm]

I'm a big fan of unit testing. I think in many cases, it's faster to developer with unit tests than without.

Perhaps the biggest problem for writing unit tests is how to handle dependencies - especially in legacy code. For example, say you have a method that calls the database or file system. How do you write a unit test for such a method?

One approach is dependency injection - where you inject the dependency into the method (via some seam like a parameter or instantiate it from a config file). This is powerful, but could require rewriting the code you want to test.

Another approach is using mock or "isolation" framework, like TypeMock or RhinoMock. TypeMock lets you isolate an embedded method call and replace it with something else (the "mock"). For example, you could replace a database call with a mock method that simply returns an object for your test. This is powerful; this changes the rules of the game. It's great to assist a team in adopting unit testing because it guarantees that they always have a way to test even that difficult code. However, as Spiderman taught us, "With great power comes great responsibility". TypeMock is fire. A developer can do amazing things with it, but they can also burn themselves. If abused, TypeMock could:

1. Enable developers to continue to write "spaghetti" code. You can write the most tangled, dependent code ever (with no seams), the kind of thing that would get zero test coverage, and TypeMock will rescue it. One of the key points of unit testing is that by writing testable code, you are writing fundamentally better code.
2. Allow developers to get high test coverage by simply mocking every line. The problem is that if everything is mocked, then there's nothing real left that is actually tested.
3. Make it harder to refactor because the method is no longer encapsulated. For example, say a spaghetti method has a call to a private database method, so the developer uses TypeMock to mock out that private call. Later, a developer refactors that code by simply changing the name of a private method (or splits a big private method into two smaller ones). It will break the related unit tests. This is the opposite of what you want - encapsulated code means you can change the private implementation without breaking anything, and unit tests are supposed to give confidence to refactoring.

TypeMock can work magic, but it must be used properly.

Developer balance of power

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/developer_balance_of_power.htm]

You need people to get the project done, but people are eventually error prone. Just like in government there are "separation of powers", software projects can also benefit from such separation. As a general rule, for production code, the same person should not both:

• Code and Review - The reviewer checks the code quality (It's too easy to give a free pass, or have bias, to your own code)
• Develop and Test - The tester checks the developer. (The dev already thinks their code works fine)
• Build and Deploy - Having someone else deploy what the developer built encourages easier and objective deployment, and helps invalidate the it works on my machine.