The Power of No

I was reminded of this recent event reading 5 Project Management Skills Every Developer Should Have.

My coworker (I’ll use “C” for their name) and I were recently asked by the project manager (for context, he was a very new hire, but that doesn’t imply he was new to the field of project management) assigned to our project, “Can you and C put due dates on all of the tasks for this project?”

My one line answer. “No”

The silence was deafening.

After the pregnant silence gave birth, the obvious question “Why not???” was asked.

Well, because:

  1. Our daily activities include a variety of other unpredictable tasks that are constantly shifting in priority (aside – such is the life in a small company. Isn’t that the definition of Agile? Laugh)
  2. We are working with undocumented verbal specifications where new information is provided every week in the weekly meeting with the client and often previous requirements change slightly. (aside – we’re an Agile team, right?)
  3. The nature of the work requires interfacing with third party API’s that are finicky and difficult to map their data responses into something we understand how to map to our fields. (aside – everyone is Agile nowadays, right?)
  4. Your own (the client’s) dataset doesn’t have all the information we need and we’re waiting for you to update your datasets. (aside – are THEY Agile???)
  5. To put a due date on something, yes, we can estimate the number of hours, on average, per day that we can work on the project, but a due date means figuring out how many hours the task will take, and we’re dealing with some unknowns that make that impossible at the moment. (Agile!) Once we have removed those unknowns, it may become possible to predict the hours.

Of course, the senior project manager started off the whole conversation with the typical Dilbert-esque management speak: “I am here to facilitate — if you need something from the client, let me know and I’ll make it happen.” I’ve been around the block enough times to know what utter BS that is.

So the manager decided that what his male ego needed was a daily 30 minute conference call with moi and C to review, each and every day (except weekends) the status of each ticket. Riiiight. So we complained to our direct manager, who “managed” – managed to get that stopped. I mean really, the guy can just look at the ticket to see the status, right?

The irony is that this project manager went from being a bull in a China shop to a mouse – no facilitation, no responses to our emails when we actually need some information from the client that he could “facilitate”, in fact, no communication at all except an hour before the scheduled weekly meetings “Are you guys ready?”

It’s amazing, the Power of No (apologies to Eckhart Tolle)

COVID is Becoming a Runaway Train

Putting a variety of things together, it seems like COVID is becoming a runaway train in many states.  We have:

  1. Increasing # of positive cases and infection rates
  2. Hospitals are starting the hit capacity
  3. The increased # of positive cases is making contact tracing harder and harder
  4. Testing is becoming delayed, now 2 to 5 day turnaround
  5. The test delays make contact tracing all but useless

That is a witch’s brew of disaster.

Lockdown and “Stay at Home” Doesn’t Work. Testing Works

All graphs are from Novel Coronavirus (COVIS-19) Infection Map as of March 28 2020.

South Korea

SouthKorea

As indicated by the 2 purple circles I’ve drawn on the graph:

  • On Feb 21, South Korea reported an 143 active cases.
  • On March 11, the peak of the active cases was 7869, which is where the yellow line starts sloping down.

In 19 days (less than 3 weeks) the rate of infections began decreasing.

Regarding South Korea, the following bullet points are all quotes from a 3/13/2020 article on Forbes:

  • South Korea Sees Coronavirus Slowdown—Without A Lockdown, But With Nearly 250,000 Tests
  • Seo Eun-young, the director of foreign press relations in the Ministry of Foreign Affairs, said aggressive testing has been the key to containing the coronavirus.
  • Unlike other countries—like the United States, where only people showing symptoms are recommended to be tested—South Korea tests anyone who had been in contact with a confirmed case, and tracks down by credit card activity, surveillance camera footage and mobile phone tracking those who are potentially exposed, a measure that has proved effective but has raised questions about privacy.
  •  The U.S. has reportedly tested fewer than 14,000 people, a statistic Director of the National Institute of Allergies and Infectious Diseases Anthony Fauci told congress was “a failing” on Thursday. In contrast, South Korea has reported they have the capacity to test 15,000 patients per day.

Hubei China

Hubei China

  • Jan 27, 1302 active cases
  • Feb 18, the peak of the active cases was 50620, which is where the yellow line starts sloping down.

A time span of three weeks and one day (22 days).

Italy

Italy

  • Feb 29, 1049 active cases
  • March 27, 66414 active cases

A time span of 4 weeks with no sign of the daily number of new active cases decreasing.

  • Italy has been in lockdown for 3 weeks now.

When I look at these graphs, the conclusion I draw is that a blanket lockdown does not work.  What does work is:

  1. extensive testing
  2. quarantining if infected
  3. tracking and testing the people with which the infected person has been in contact

New York

New York

  • March 18, 1635 active cases
  • March 27, 44116 active cases

If we look at the active confirmed reports for New York:

  • March 23: 20718
  • March 24: 25455 (an increase of 4737 from the day before)
  • March 25: 30526 (an increase of 5081 from the day before)
  • March 26: 36873 (an increase of 6347 from the day before)
  • March 27: 44116 (an increase of 7243 from the day before)

New York has been under a “stay at home” order for 13 days now (initiated at 8 PM on March 15).  The rate of new cases daily continues to increase.  “Stay at home” isn’t working.  If we were to achieve what South Korea and Hubei achieved, reversing the increase in daily new infections in 3 weeks, we should start seeing this curve decreases, well, about now, as we have one week left to go since the stay at home order at which point there would be somewhere around 100,000 active cases.  I seriously doubt this will be the case.

Why doesn’t lockdown and “stay at home” work?  My personal opinion is:

  1. I suspect this relates to “essential businesses” that must stay open.  This doesn’t just include hospitals, walk in clinics, pharmacies, gas stations, grocery stores, etc.  This includes the entire healthcare, production and delivery system for the goods and medicines that these essential businesses require.  We have to eat, people have to feed their pets, take their medications, go the the hospital, be treated, etc.
  2. Many people don’t know they are infected as they are asymptomatic but continue to spread the disease to others.
  3. Support staff required to keep these essential businesses open and the supply chain “supplying”: drivers, packers, distribution centers, administrative workers that must be on site, etc.

March 26:

Many COVID-19 cases are thought to be mild, and infected individuals with mild or no reported symptoms are still contagious and capable of spreading the virus. Plus, the virus has a long incubation period, with many people not showing symptoms for an average of five days after infection. Together, these two factors result in a lot of people who are infected and spreading the virus without knowing it….In a study published in the journal Science earlier this month, Shaman and his colleagues found that undocumented COVID-19 cases were responsible for 86 percent of the spread of the disease in China before the country enacted travel restrictions on January 23, 2020…“The fact that there may be some silent transmission for COVID-19 makes it very difficult to contain,” says Meyers. That’s why people worldwide are now taking such extreme social distancing measures to try to get the outbreak under control.  ssource

And yet South Korea did contain the outbreak.  Through testing.

Again, regarding testing:

March 13:

Despite the fact that last week, Vice President Mike Pence promised that “roughly 1.5 million tests” would soon be available, an ongoing Atlantic investigation can confirm only that 13,953 tests have been conducted nationally. New York, which has shut down Broadway and has at least 328 coronavirus cases, is still failing to test patients who have worrying symptoms. source

March 20:

Los Angeles County health officials advised doctors to give up on testing patients in the hope of containing the coronavirus outbreak, instructing them to test patients only if a positive result could change how they would be treated.  The guidance, sent by the Los Angeles County Department of Public Health to doctors on Thursday, was prompted by a crush of patients and shortage of tests, and could make it difficult to ever know precisely how many people in L.A. County contracted the virus. – source

March 21:

New York City health officials have directed medical providers to stop testing patients for the coronavirus, except for those sick enough to require hospitalization, saying wider testing is exhausting supplies of protective equipment.  In an advisory issued Friday, the health department said outpatient testing should stop unless results would impact a patient’s treatment.  It said demand for unnecessary testing is contributing to a national shortage of masks, gowns, collection swabs and other supplies, all of which need to be discarded by health care workers after each test.  As of Friday morning, more than 32,000 people had been tested in the state, almost a third of them in the last day, Gov. Andrew Cuomo said.  More than 7,000 New Yorkers have tested positive. More than 1,200 have been hospitalized.source

Contrast that to South Korea’s capacity to test 15000 people per day.  Again, why testing is important:

Testing allows infected people to know that they are infected. This can help them receive the care they need; and it can help them take measures to reduce the probability of infecting others. People who don’t know they are infected might not stay at home and thereby risk infecting others. source

Also, points to note regarding Coronavirus disease 2019 (this is from wikipedia) and why testing is so important:

  • Details about how the disease is spread are still being determined.
  • The virus can remain infectious for hours to days on surfaces.  Specifically the virus was infectious for up to three days on plastic and stainless steel, for one day on cardboard, and for up to four hours on copper.  This however varies based on the humidity and temperature.
  • One study found that small droplets with coronavirus, generated by laboratory equipment, could stay airborne for three hours.

Personal comment: Have you handled money recently?

We need to start testing!

Other reading:

How prepared is the US to respond to COVID-19 relative to other countries?

What WHO advisor David Heymann told us about COVID-19

 

What is Productivity in the Context of Software Development?

With regards to software development, productivity is an entirely subjective concept.

It’s entertaining and somewhat disheartening to read some of the definitions of productivity:

“…the state or quality of producing something, especially crops.”

okaaaay.

“…the effectiveness of productive effort, especially in industry, as measured in terms of the rate of output per unit of input.”

Unit tests anyone?

And my favorite:

“…the rate of production of new biomass by an individual, population, or community; the fertility or capacity of a given habitat or area.

Wikipedia has an interesting definition:

“Productivity describes various measures of the efficiency of production. Often, a productivity measure is expressed as the ratio of an aggregate output to a single input or an aggregate input used in a production process, i.e. output per unit of input, typically over a specific period of time.”

The thing is, the concept of “input” and “output” with regards to productivity is highly abstract when it comes to defining productivity for software development. Input can range from someone saying “this needs to be done” to a full blown spec. Output can be anything from a bug fix to an entire application.

Because of this wildly ludicrous range, we have scrum and agile methodologies which create sprints, breaking down “productivity” into more chewable (but not necessarily more digestible) units:

“A sprint is a short, time-boxed period when a scrum team works to complete a set amount of work.”

It accomplishes this by forcing an arbitrary time interval to the work from which, again somewhat ludicrously, the team’s “velocity” can be measured to create nice graphs for the venture capitalists that are keeping the sinking ship from, well, sinking.

Because only so much can be done within a fixed time period, we have “iterations” and “refactoring” and “only do the minimal amount necessary to get the task in the sprint done.” So velocity looks good on paper, but does anyone measure how many times the same piece of code (and its dependencies) get refactored over a thousand or ten thousand sprints because the task wasn’t given enough time to do it right in the first place?

Of course the solution to that is to decompose the task into, you guessed it, smaller tasks which are “sprintable.” Rinse and repeat until you get a tower of babbling developers, project managers, and C-level managers, each speaking in unrecognizable tongues to the others.

Outsourcing addresses this bottomless pit by getting rid of costly developers and hiring droves of cheap developers that have laser focused myopic vision (see post below on the 737 Max) which results, if you’re lucky, in a failed product, and if you’re less lucky, death. Of the project, of people, of the company, of the management, any and all of the above.

So how do we then measure developer productivity? Let me ask a different question. Why should we measure developer productivity?

The productivity of developers is meaningless before the product hits the market. How can you measure “input” and “output” when the damn thing isn’t even generating any money? Charts of velocity are useless, at best they might tell you when your money is going to run out or when the VC’s are going to pull the plug. I feel my argument is weak here, but I stand by the premise.

The productivity of developers after the product hits the market and is generating revenue might be measurable against certain criteria, such as sales and customer satisfaction. It is also easier to perform sprints on an existing product that is in its maintenance cycle rather than its development cycle because maintenance is mostly tooling improvements, bug fixes and specific new features and the eternal pendulum swing between fragmented (microservices, serverless, etc) and monolithic architectures.

Using sales as a criteria becomes useless when you have a monopoly, or more PC, “cornered the market.” Or you have enough money to buy your competition. Customer satisfaction? Who really cares as long as you’re making sales?

So how then do we measure productivity? Simple. How much money did I make today vs. how much did my developers cost today? If that ratio is > 1, someone (not necessarily your developers) is productive. It could even be the consumer, being productive enough in whatever they do to afford your product, be they person, collective, corporation, or government. If that ratio is < 1, then you have a productivity problem. Somewhere. Not necessarily your developers. Maybe the consumer isn’t buying enough of your product due to an economic downturn. Or simply that your product sucks.

The only time you can actually measure developer productivity is when the company is too small to have a gaggle of managers, a shoal of lawyers, a caravan of tech support people, and a murder of sales “engineers”, on a product already bringing in revenue.

In other words, a startup company that has succeeded in making some sales, usually to corporations or government which will pay for maintenance contracts (hence some revenue stream after the initial sale) and that is most likely growing too fast and too hard and can’t keep up with the customer requirements and bug fixes but hasn’t yet hired the gaggle, shoals, caravans, and murders that a well greased “where did my productivity go?” company requires.

Which brings me to my Alice in Wonderland conclusion, that developer productivity can only be measured in that awkward, painful, stressful, and insane period when a company has “hit it” but hasn’t “gotten it”, there is a minimal amount of obfuscation between the customer and the developer, the backlog of work is far beyond what the current team can accomplish without the tech to transfer brains upon death, and productivity is measured against “this has to get done by Friday or we lose the customer or sale.” In that specific circumstance, productivity is easy to measure. You either succeeded to keep the customer or make the sale, or you failed. Binary. Black and white. You succeeded in producing the output or you didn’t. You were productive or you weren’t.

One final rabbit hole. Developer productivity is also meaningless because it assumes a manufacturing style of “input” and “output” over a given time period. Software isn’t like that. It might take years to write a Google or Facebook, but once it’s done, well, it’s done. The “consumption” of the product is a web link or a 30 second download (unless you’re Microsoft.) So how the heck do you measure productivity <i>now</i> when once the product (the software) is produced, the “output” is little more than a click that clones onto your hard drive (if even that) the pattern of 0 and 1 bits that define the product? Wow, my developers are insanely productive! We’ve had a million visitors to our site this year!!!

Which gets us to the evil of productivity — is Marc more productive than Joe? Meaning, given similar tasks, does Marc get the job done faster and with similar “accuracy” than Joe, or not?

Which, going back to my Alice in Wonderland scenario, is not an issue when your developers are “expert islands” and the developer to island ratio is 1:1. You have no basis for comparison until your company get passed that birth process and the developer to island ratio is 2:1 or more.

And this ratio, by the way, defines “job security” vs. “eek, I’m replaceable”, and therefore drives developers to be perceived as productive when in the “eek, I’m replaceable” corporate structure. Fortunately, there are many islands and the key to success (both for the developer and the corporation) is to have a healthy balance in the developer:island ratio, because developers want to feel unique and valued and not a cog in the machine, but a healthy level of stress and knowledge sharing is also socially rewarding. Which, in terms of psychology, makes for a happier and more productive developer! And ironically, in a corporate environment, leads to the conclusion that only the developer can tell you whether he/she “feels” productive and to what degree, so you’re productivity measure in that scenario becomes entirely subjective. Which was the first sentence in this tome that just killed your productivity.

Progress

In the series Halt and Catch Fire, Joe MacMillan (Season 1, Episode 2) says:

“Progress depends on changing the world to fit us, not the other way around.”

I decided to google that, and it turns out it’s a condensed version from George Bernard Shaw, Man and Superman.

“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”

Feels like the story of my life (being the unreasonable man) when it comes to software development!

 

 

The Rock

I first found it in my back yard – the lawn mower ran over it and the blade took a sizable chunk out of it.  How it got there, I don’t know, as I’d mowed that section of the lawn many times.  But there it was, sort of grey with flecks of quartz scintillating in the sunlight.  And of course with a chunk missing.  Good thing rocks can’t bleed.  Still, looking at it, I felt a kind of sadness that this rock had somehow found its way into my yard only to be partially decapitated by the spinning blade of the mower.

I brought it into the house.  No point leaving it in the yard for further potential mutilation.  It wasn’t that big – maybe the size of a big melon, and seemed rather light for a rock.  I put it on the coffee table, the bloodless gash turned out to make good “foot” for it to rest on, its grey face and many faceted pinprick eyes staring up at me as if to ask some unspoken but important question.   I had no answer; even the cats sniffed it just once and promptly got bored.  Still, there it was, a stranger that had been invited into the house but that leaves an awkward silence between guest and resident, each wondering where this was going.  Each wondering, maybe this was a mistake.  But then again, the rock didn’t have much choice in the matter, did it?

As I did the various chores of the day, I’d occasionally walk by the coffee table that was somewhat reluctantly, but without complaint, holding this new burden.  I kept noticing how the living room felt different – yesterday, there was no rock on the coffee table, today there was.  It felt strange having this new addition in the house, and yet at the same time growing familiar.  Every time I walked by it I experienced surprise, a flash of “what is that?” that was replaced immediately with “oh, it’s the rock.”  This happened several times throughout the day until at one point I realized I had passed the rock without even noticing it.  Even the coffee table seemed more at ease, as if it were growing more comfortable with this new addition.

After feeding the cats their late night snack and turning off the kitchen light, I headed upstairs to bed, and on a whim stopped in front of the coffee table and gave the rock a gentle pat, whispering “Good night.”  With no light to reflect off the tiny quartz shards, it seemed asleep, eyes closed.  Yet it also seemed to feel a kind of foreboding, the way someone pretends to be asleep because they don’t know what’s going to happen next but hope whatever it is will just go away.  I decided not to disturb it further.

Blame it on the glass of Shiraz I had with dinner, or the full moon, but I awoke at 2 AM with the call of nature.  Heading downstairs, the rock was awake now, moonlight glimmering in its eyes.  Feeling a bit self-conscious in my nakedness, I stepped quickly past it.

The next morning I noticed something odd.  There were flakes of what can only be described as debris in a circle around the rock.  Sort of like dandruff or flakes of dead skin.  Maybe the cats had after all taken an interest in the rock and had brushed off some loose material when they rubbed their noses on it.  It glared up at me in the morning sunlight, its eyes dulled a bit by whatever had transpired in the night.  I decided to leave it, as I wasn’t sure if wiping it with a rag or washing it would do more harm than good.  That night I decided not to give it a goodnight pat.

Every morning, there was a new pile of this rock dandruff in a ring around it.  Every morning, I brushed it off the coffee table, being careful not to touch the rock.  The coffee table was annoyed – these flakes were sharp and left tiny, almost microscopic scratches in its surface, something that wounded its antique pride.

A week later, the rock died.  I came downstairs to find a pile of rubble instead of a rock.  It had shattered into a hundred fragments, its bones and cartilage and sinews all crumbled apart.  Perhaps the original blow by the lawn mower had created tiny internal fractures and it had finally succumbed to its wound, which must have been more grievous than had first appeared.  Its eyes were gone, only dull, empty grey ruin remained.  I grabbed the dustpan from the kitchen shelf and swept up its corpse, and was about to toss the remains into the trashcan when something in me stirred – this was an ignominious burial for my house guest.  Instead, I took the dustpan outside and scattered the remains on the lawn.

That night, in the waning full moon, the lawn gleamed like diamonds.

Microservices: Myth, Madness, or Magic

Excerpt:

If you drink the Kool-Aid, the key phrases to the microsservices bandwagon are:

  • loosely coupled services
  • fine-grained
  • lightweight
  • modular
  • resilient
  • parallelizes development
  • scaleable

The irony here is that we’ve heard pretty much the same mantra starting with object oriented programming / architecture / design, so why are microservices now suddenly (one of) the in-vogue solution to problems that have not been already solved? 

Read more on Code Project.