IT Meritocracy. Part 6: Everything Is About Technology.

The Soviet Union I grew up in, was an oppressive Orwellian state. However it had none of the modern Western safety BS like “choking hazards” and lawsuit-inspired warning labels. I played with finely-detailed hobbyist-quality East German toy trains and realistic-looking toy guns. As far, as I can remember, Lego blocks were normal (small) size too.

One can build an infinite number of cool things (see the F1 car above) out of universal Lego blocks. It wasn’t until I became a parent (here, in the US), that I discovered different chocking-safe Lego for toddlers. Its big crude blocks immediately reminded me of countless attempts to replace programming with DIY software “building”. All of them: from COBOL to present day CMSes and DMSes have failed — to deliver robust industrial quality software.

Software is the same set of programming instructions (universal small Lego blocks) to the computer, whether it is consumer blog platforms or enterprise systems. Both have UI, business logic, users/security, notifications, databases, and other universal Computer Science concepts. So why the enterprise software industry continues its attempts to dumb down programming even today, after seeing how well the consumer tech: Google, Amazon, Netflix, and Uber to name the few did without “simplified” tools, languages, “almost-turnkey” packages and other “big Lego”?

Not enough (smart) developers, right? All went to Google? Paid 2–3x higher, duh? That only partially explains the problem. Even if the (inflation-adjusted) wages stayed at the pre-outsourcing level i.e. Google’s $300–400K today, there still wouldn’t be enough real (top) programmers in IT: the ones who end up writing all the code, including cleanups and rewrites after their little code monkey “helpers”. Sure, a capable programmer aka “10x developer” can do 10x more — with conventional technology. 100x more is needed. And so, in the same pathetic industry status quo, anything (non-technical) still goes: throwing third-world bodies at the problem and mythical DIY. Let’s talk about the latter.

Can one build a mission-critical ERP out of connected scripted Excel spreadsheets or “online forms”: the modern take on the original screen-per-record Oracle Forms concept from the 80s? Theoretically… yes. How many people it’d require to maintain? Not the end users or “subject matter experts”. Programmers. Even the “simplest” scripting languages like Excel’s VB still have variables, ifs, and loops, only programmers understand. No “power user” wants to learn that stuff. The same goes for “customizable” ERP packages from IBM, SAP, and Oracle offering “almost-turnkey” functionality which requires “a few tweaks”, that typically last months/years and involve 100+ man teams of “customizers”: a poorly disguised programming.

CS-educated software engineers stayed away from those in the 90s. The awkward “business-oriented” languages and pseudo-DIY “visual tools” are not just intelligence insulting. They are 100x less efficient, than simply typing normal industrial-strength programming language (C++ or Java) keywords. We thought we made it known to our bosses, that we didn’t like that stuff. But they never gave up the dream of getting rid of “expensive” engineers by dumbing down programming. All they needed was an overpopulated poverty-stricken country with more pliable “resources”.

Whether it was reviving prehistoric “customizable” ERPs, Western programmers looked down at, or butchering a normal language like Java e.g. by building progressively smaller pieces comprehensible by code monkeys — instead of cohesive bigger systems, it doesn’t matter how exactly the business software development was dumbed down, while consumer one continued to evolve at Google and Netflix. It happened. One day we found our big kid toys taken away: replaced by chocking-safe crude toddler size Lego blocks and soft cubes. How long we, big kids, are going to stay in daycare learning alphabet?

Slave Labor Doesn’t Work In Creative Occupations.

Everyone including the cheapest CFO knows that. So why expert programmers are still paid half of what they made before 2002 — outside Google and the handful of top consumer tech employers? Our bosses must still need us, mustn’t they? That’s what I thought, seeing the project failure rate predictably jump from 70% to 90% in the mid-2000s. I thought IT CIOs would come to their senses after enough failures — to remove the “slave” from the equation. Instead they removed “creative”.

The enterprise software industry comprised of corporate IT departments and the Consulting Food Chain serving them doesn’t need experts anymore. Nor you want sh_tty bug-fixing jobs focused on coping with limitations of outdated tools and the incompetence of semi-technical (at best) bosses. Recall the last recruiter email that excited you. Speaking of which…

What if it’s predatory recruiters? The industry has been invaded by the worst telemarketer-type foreign staffing bodyshops. Should we, engineers, change the recruiting process? By paying their commissions instead of employers being the “paying customers”? The irony of the situation is that recruiter commissions and rate markups come out of our pocket anyway. Suppose we make it official, so they know who’s paying their salary and respect us instead of “shooting the fish in a barrel”. Then what? There are many “executive search” firms charging fees upfront to place somebody. Have they ever convinced a single IT manager to pay Google compensation: 2x the IT average?

For what? Fixing more complex bugs? Being the last line of defense and rewriting all of the crap that comes from the “outsourced” code monkeys before it hits production? It’s needed. It’s critical. But it’s still coping. Google doesn’t cope. It can be too “scientific” and snobby, avoiding unglamorous enterprise automation like a plague, but no one would deny, that Google employees are paid 2–3x IT wages for truly creative work. So before demanding Google wages in your cheap IT department, don’t you think, you need to bring back the creative part? No MBA CIO will do it for you. It is our, engineers’ responsibility.

Everything, primarily our wages, is about technology.

Things weren’t rosy in IT before the start of mass outsourcing of 2002. The only difference is, we, programmers were blissfully unaware of the eternal 70% project failure rate, paid six figures just to show up to work. In 2002 the reality caught up and everyone (sadly except for corporate CIOs and other top brass) felt those failures through the 50% wage reduction.

Was it unfair — to replace us with code monkeys? You tell me. What did you do in 2002? Wrote the same boilerplate code: DAOs and DTOs, I did? Drew 5y.o. coloring book worthy “architecture diagrams” of multicolor boxes and arrows? And wondered why you are paid $90–100/hr? Yeah, that’s what I thought. Code monkeys can imitate work just as well — paid a fraction of American wages.

How did it happen? Did everyone do it by choice? I am sure many lacked the talent to deliver more, than glue code. I always wanted to, but… My bosses didn’t let me? Or another excuse? No. The technology wasn’t ready. OOP only matured (merged with FP and gaining important programming patterns and tools) towards 2010s, forged in the free (of outsourcing) world: Google, Amazon, Netflix, and a handful of the companies leading consumer tech. Web development evolved a lot. Most importantly all of the above, and more, is free (open-source).

It wasn’t around in the early 2000s, let alone late 90s, when the industry desperately needed new programming paradigms to replace Y2K-affected mainframe stuff with the robust new systems. Unfortunately in 2002, when the floodgates opened, programming was still dominated by outdated software development models and processes pushed by IBM, Oracle, and Microsoft — all relying on headcount. Well, you know where American IT found it.

The Technology Landscape.

Even if you are more mature, than CS grads dreaming of reinventing Facebook and Snapchat — unaware of B2B niches, that, unfortunately can only be discovered via job-hopping from one sh_tty employer to another… knowing those niches at the expert level does not enable you to approach investors and businessmen with a finished product they can make money off, rewarding you for your hard work and ingenuity.

It doesn’t matter, how trained and eager you are to “pedal in a higher gear”. If you don’t have the right (modern) bike with that gearing, any attempt to develop a product/service bigger than a blog or messenger will turn into the same unproductive IT commotion you witness daily at your dysfunctional employer. You’ll end up with the same dozen of narrow specialties, your teammates are pigeonholed in. No matter how smart and hard-working you are, you will not be able to do everything by yourself. Anything beyond an unimpressive feature-poor prototype will require a, no doubt smaller, but still sizable “team” tasked with, as much as you hate it, the same daily code monkey activities.

Love it (as a staffing middleman) or hate it (as an underpaid programmer), the current software development process is suited to accommodate the cheapest most junior workforce the overpopulated third-world allegedly has limitless supply of. It’s not just the CFO greed or stuffing middleman push for more man-hours. The technology wasn’t very robust before “offshoring” — to keep up with the rapidly increasing government regulation and overall business process complexity. Not surprisingly, the “offshore outsourcing” came to the rescue.

Comparing software development with cycling, most people on Earth are not competitive cyclists, though they do know how to ride a bike, Many even use it as their daily transportation. Will they win any races? No. Not to mention those commuter (beach cruiser, etc.) bikes are different from pro ones. They can be compared to DIY website builders: from Wix to Wordpress.

Then come hobbyists aka “weekend warriors” that ride amateur race bikes, but will never enter, let alone win a professional race. Those are freelancers, who use tools ranging from the aforementioned Wordpress (with manually written PHP scripts) to more “programmatic” CMSes like Drupal and Sharepoint. A CMS (Content Management System) is a Web/Cloud reincarnation of the good old 1990s Access, which in turn was the PC version of 1980s Oracle Forms, tracing its roots to pre-relational 1960s COBOL.

All today’s allegedly “pro” bigger tools for bigger projects are takes on either COBOL directly (old-school SAP, IBM, and Oracle ERPs) or Oracle Forms (Salesforce). Hasn’t it always been about the DIY Holy Grail of software development: eliminating the poorly understood and feared programming itself by “simplified” “business-oriented” languages and “visual” (drag and drop) tools? It doesn’t work this way.

Finally there are pro tools: industrial programming languages like Java. Are they used to their full potential? Not in the business aka enterprise software industry, recreating Salesforce (improved age-old Oracle Forms) with Java. Conceptually everything in that space is one or another: “customizable” or “custom” (more on this below) version of Oracle Forms. Nothing new has been invented. Never needed after the Y2K “aversion”?

Only a handful of top consumer tech companies like Amazon and Google leverage full C++ and Java potential, albeit for a limited range of applications. Consumer services (blogs, messengers, ecommerce, etc.) is a relatively narrow niche best compared to track racing in the world of cycling. Amazon’s and Google’s Java (C++, Python, etc.) tools are beautiful and efficient (geared high) machines. The problem is, they are single-speed ones. No derailleurs or brakes — to save weight. None needed at the flat track.

Imagine, that derailleurs and sprocket cassettes never been invented. That’s the single-speed software world today: heavy 1940s cruisers with coaster brakes and super-light fixies for the track. The most refined cruiser: 1999 Salesforce is almost 20 years old. And minimalistic fixies cannot be improved further mechanically, so the consumer tech industry turned its attention to the racer health/strength i.e. they switched from engineering (building bigger and more complex products) to science (improving the analytics of the existing single-purpose sites and apps.

Sharing the Road… In Your Dream.

Does the world need bigger (enterprise) products? More specifically w/o the “enterprise” price tag? That’s exactly what I am doing now: all-in-one systems for doctors, insurance agents, auto dealers, and countless others. The current technology leaves a big void between little single-purpose freelancer openings of form-filling and ecommerce kind and heavy ERPs.

Most freelancers are Web (mobile) designers who learned rudimentary programming and switched from completely static online brochure websites to 10-page data entry ones. Anything beyond that will require a team of narrow specialists, not only reluctant to be generalists and “pedal in a higher gear”, but lacking the equipment to do so in the first place. As a result there cannot be enough generic “widgets” (e.g. chat or shopping cart buttons) one can cram into an otherwise static page to turn a (WordPress) “website” into a coherent enterprise application — to automate a specific and custom business process e.g. doctor’s office document flow and appointment/visit management.

How everyone did it before the Internet? Sadly that’s half of the business software today. If you are a C++ or Java developer, you don’t want to know. Let alone work at a typical super-cheap SMB (auto dealer, insurance agent, caterer, etc.) software shop relying on grayish Win95 era Access.

That only leaves you a choice between working at an old-school ERP vendor like Oracle or Salesforce, a consulting company selling (tweaking, integrating, etc.) Oracle and Salesforce products, or a custom solution provider mindlessly using Java basics to replicate Salesforce. All three are heavily “outsourced”, because they cannot find (and/or afford) enough pros trained to pedal in a higher gear.

It is not the pro’s fault or the race team owner’s greed. It is the lack of the proper equipment. One can only do so much by the book in Java, let alone prehistoric ERP packages with their awkward COBOLish languages. Say, the project requires 50 experts — nearly impossible to find, let alone coordinate due to their complex output and zero margin for error. That’s how experts work. Otherwise they wouldn’t be such. The project owner has no choice, but to hire amateurs to augment experts.

Theoretically both can coexist i.e. the experts take most complex tasks, while the lesser skilled team members take care of the routine “coding”. In reality, over my 25 years in IT, things turned exactly the same way, as busy freeways. There are fast and slow lanes, so in theory fast drivers shouldn’t be held up by slow ones. You know how it works in real life. A freeway flows at the speed of its slowest driver. Inattentive slowpokes have uncanny ability to find each other and hog all four or even six freeway lanes. It’s human nature aka the lowest common denominator.

Just like good and bad drivers on the road, experts and mediocre “coders” don’t mix at all and should be physically separated. Essentially any expert (driver or programmer), no matter how fast, is forced to slow down to match the speed of those who simply shouldn’t be driving (programming). Worst of all such expert is paid according to his actual speed (output) instead of potential. Here we go again about racing with pros.

Sorry for a selfish remark (in Borat’s voice… NOOOT!), wouldn’t “spirited” drivers’ daily commute be more enjoyable, if every clumsy slowpoke who hates driving, took a f-cking bus instead of struggling with the power-assisted steering wheel and two pedals? Things are better in NYC, but LA’s infrastructure is not built for that — at the current technology level. Invent the right technology to reduce the number of drivers on the road — to those who love and excel at driving, and everything falls into place.

Unfortunately 20 years ago, when the enterprise industry desperately needed such technology: to employ five experts instead of 50, it was nowhere to be found. Y2K-affected 1970s mainframe systems were supposed to be replaced at least a decade before the 2000 deadline. The original creators didn’t expect their software to linger that long. Hence, pressed with the RAM and disk space shortage, they only allocated two spaces for the year. Well? Were the highly skilled pros able to develop suitable replacements with the 1990s technology? If the industry had a bit more of expert programmers, sure.

They had their chance, didn’t they? They tried. Eventually their bosses got fed up and found the experts cheaper helpers to keep trying on a more massive scale. Eventually the low-tech approach of zip-tying and duct-taping the aging mainframe succeeded.

Sucks for you and me — seeing the low-tech solution work and our wages halved, since no one needs high-tech in IT anymore. Both falling apart old crap and the stillborn new one can be made to (barely) work after enough bug fixing — for another decade or so, until the real Y2K-geddon happens in 2030s. However to give code monkeys some credit, they solved the problem for the time being, didn’t they?

I’d have let the code monkeys be after solving the Y2K problem with brute force. I wouldn’t have been bothered by clumsy drivers on the freeway — if they adhered to (I mean had the knowledge of) lane discipline and the right of way. And stayed the f-ck out of the way. I am forced to move at their speed. That’s the problem.

Empowering the Strongest to Do More? Sadly, It Is Mitigating the Mistakes of the Weakest.

Today’s business software technology is suited for mediocre developer output. It focuses on mitigating expected beginner mistakes via TDD. It dumps the spaghetti dependencies mess into magical server “containers” like Docker — instead of keeping things clean and simple. It breaks the project into the smallest pieces aka custom “micro-services” comprehensible by the lowest-skilled programmer.

Then it custom-“integrates” those small pieces a code monkey can be trusted with — a completely unneeded task, since integration should never be one-off. Software integration is a means of code/service reuse. It should reliably and consistently tie together fairly universal modules usable across some domain or industry.

Micro-services and Docker were invented for reasons, vastly different from the Consulting Food Chain’s push to place more “bodies” and cope with their output. Didn’t Scrum, hated by almost every programmer today, come from the development community too? Look what brain-dead PMs turned it into. Not surprisingly Docker and micro-services were butchered in exactly the same manner.

None of the integration BS is new. SAP, Oracle, IBM, and other old-school ERP vendors have a long history of fooling non-technical corporate CIOs with “almost-turnkey” modules expected to “click together”. Half of such module is always useless — covering someone else’s requirements, while another half needs to be completely rewritten to be useful.

There are three well-known, albeit only among programmers, truths about software development.

First, the only DIY tool is Excel. Not Access or its Web/SaaS reincarnations: CMSes and DMSes all the way up to Salesforce. All of them require programming, which no end users would touch with a 10ft pole.

Second, there are no almost-turnkey “packages” evident by teams of hundreds “just tweaking” SAP and Oracle ERPs for a couple of years. There is either universal single-purpose software: Word and Excel or completely custom systems automating custom business processes. No two companies are alike. And even if it is a 5% difference, it will always, every single time require rewriting half of the “almost-turnkey” package that wasn’t initially designed with a specific feature in mind.

Third, just like one cannot magically tune an almost-turnkey package to automate a custom business process, no one can assemble i.e. “integrate” a custom solution out of 10 or even a 100 of different “packages” like that. Only small machine parts are versatile enough to be connected together.

Here we go again about Lego. A small “Lego block” is a programming command like “if” and “for”. A big “Lego block” is an Accounting or Supply Chain ERP module. It requires a lot of cutting, drilling and other fabrication to produce a fine shape — effortlessly built with universal i.e. small size Lego.

B2C or B2B, anything moderately complex is always custom and generally shouldn’t be broken (externally) into simpler pieces unless it is natural — for the customer. Certainly not because engineers cannot handle the complexity. Everyone talks about “loose coupling” (of software modules), but how many remember “cohesion”, described on the same page in the same (Grady Booch’s) book?

In any case, if one is able to write 100x less code, he/she can develop those modules as a whole. No proponent of so-called “micro-services” I’ve met could argue with that logic.

Imagine a car that for some reason (no right parts suppliers or no right materials) couldn’t have a hydraulic line or another hose longer than 10”, requiring hundreds of couplings — each presenting a potential point of failure. Imagine an electronic device e.g. a mobile phone, that couldn’t have a single main chip, because it was too complex to develop by a single manufacturer. Those were the 80s “brick” phones, weren’t they? Full of soldered resistors and capacitors inside. Thankfully no one applied the “integration” (soldering) idea to electronic gadgets. Otherwise mobile phones would have been in the stone age too, like enterprise software since the successful “aversion” of the Y2K — concerned only about connecting one prehistoric (saved from Y2K) crap to another.

Integration of somebody else’s (third-party) package is even more expensive. Not only you have to build a coupling to connect it, but that coupling has to comply to the third-party specifications you need to learn and maintain, distracting you from the task at hand: implementing your customer’s business logic, that has nothing to do with all of the integration plumbing and unneeded baggage of the component you are integrating with, 90% of which was developed for others and is completely irrelevant for your application.

Breaking the Vicious Cycle.

There weren’t enough experts to solve the pressing automation problems e.g. the Y2K issue, so the industry hired amateurs to help, which required dumbing down the engineering process to accommodate them. Experts stopped to matter (and being paid well), but worst of all, unlike them, the amateurs never solved the problem completely and required more and more amateurs to keep up: e.g. automate 80% of the remaining 20% they couldn’t comprehend, fix 80% of the 20% of remaining bugs (producing more bugs in the process), etc. Use any other, than the common 80/20 rule. It doesn’t change those dynamics. As a result the world is running out of code monkeys, even in the well-known overpopulated countries.

I am not going to sugarcoat things. Yes, Google search is synonymous with the Internet itself. Yes, Amazon is THE shopping. Yes, Netflix is the TV. No one can imagine life w/o social networks whether it is Facebook or one of its clones. But with all due respect the less glamorous massive B2B software e.g. bank and hospital automation is still more important, isn’t it?

How long we need to cope with essentially the reversal of technical progress in enterprise software?

Ever since the dawn of the Industrial revolution the technical progress has been about reducing the labor through automation — due to the shortage of said labor. Introduce the excess of labor, and the progress itself reverses as well.

Can modern skyscrapers be built by millions of slaves equipped with shovels and hammers — in the same manner, as Egyptian pyramids? Why software is different?

No robust technology > no need for expert programmers > no more robust technology invented by those programmers. Is it an unsolvable chicken and egg problem? Or one of vicious cycles, we, engineers are responsible to break — to move the technology forward?

Reduce the waste of unneeded “integration” along with many other things: managerial “communication problems”, rework, 3am server crash heroics, etc. All that sh-t simply disappears with lower headcount, which in turn shrinks exponentially, opposite of its growth pace at a typical dysfunctional outsourced team, where a single code monkey introduces five bugs by fixing one, and thus requires five new “coders”.

The only spark that can ignite such massive implosion of headcount is technology: to write less code. I am solving this problem for our customers and don’t want to keep philosophizing. Imagine everyone armed with the technology I am talking about, The labor (lines of code to write) shrinks to the level, finally matching the finite number of true programmers on Earth — yes, pedaling in a higher gear, as every pro should.

You can be part of this revolution: way overdue in the stagnating enterprise software industry. Or you can compete with the tiger mom bread scholars for Google jobs, which pay 2x the outsourced IT average, but still won’t make you financially independent.

The best part is that the revolutionary assault on the rotten system doesn’t need to start with the heavily guarded (by the IT Consulting Food Chain) big cities: multi-million corporate ERP projects. There is a vast completely defenseless SMB automation space not even the leanest and cheapest e.g. $20/hr “offshore” solution provider can enter without the right technology — yes, the one empowering you to pedal in a higher gear.