IT Meritocracy. Part 4: Pedaling in a Higher Gear. Is Writing Less Code Better?

The focus of this series is less programmers doing more — by writing less code. As you can expect the new kind of code is more “concentrated”. Is writing less code to cover the same amount of product functionality always better? The answer is not as straightforward, as one may think.

Let’s get something out of the way first. I am not a “technologist”, “practitioner”, or CTO with a “technical background”. I am an engineer delivering stuff that works. I don’t engage in linguistic (i.e. infamous Haskell/Go/whatever-trendy-“new”-language-comes-next vs. “old” Java) discussions or puristic (OOP, FP, etc.) arguments. They are theoretical.

The only measure of the code quality is the end product’s quality itself i.e. how fast it was built vs. covered functionality, ease of use, and ease of change: future extensibility. Goes without saying, the product is expected to work — with minimum bugs and maximum uptime. Nothing else matters.

Sports Perspective

Doping (cycling) scandals aside, many athletes possess the genetic advantage over the general population e.g. more red blood cells and thus higher oxygen saturation — increasing the strength/stamina and slowing the heart rate. Others train since the early childhood to become pros.

Whatever your case is (maybe even coke or another stimulant, j/k), you just know you were meant to be a programmer. You love it. You outperform your colleagues. However you arrived at your current pro level, you posses the superhuman strength to pedal uphill in a higher gear, passing everyone. So you watch your favorite tournament: say, Tour de France (pictured above) for the sake of this discussion, admire the winner on the podium, and can’t help thinking “Shouldn’t I get a big prize for finishing first in my sport?”

Sports are the ultimate meritocracy. It doesn’t get any purer than that. Directly translating one’s excellence into the prize. I am sure lots of HR dissertations have been written on the subject of fully utilizing the employee’s strength. How many of them touch the topic of reward… like you know, money? Not “perks” or “work-life balance”.

Why money discussions are always so hush-hush — carefully avoided in HR and other corporate circles? Is it just my — eternally overqualified employee — perception? Everyone, starting from the government with its tax scale and down to my immediate bosses, who reminded me every day about being “the second highest-paid developer” or simply “we are paying you a lot” were trying to make me ashamed of making and wanting more. Why? Even and especially when the “second highest” was slightly below the industry average, and 1/3 of comparable Google’s compensation.

The latter partly explains things. If we were to compare programming to cycling, the handful of the top tech employers (Google, Facebook, Amazon, etc.) represent pro tournaments like the Tour. Everything outside that circle are local charity races at best, if not just Sunday pastimes for “weekend warriors”. One needs to compete professionally to win the prize.

Having acknowledged that, what if I happen to specialize in another kind of “racing” (than Google’s and Facebook’s consumer tech)? Imagine if only minimalistic brakeless single-speed track racing (single-purpose consumer products like blogs, messengers, and eCommerce) was declared pro, while month-long multi-stage endurance races over a tough mountain terrain (robust automation of complex business processes and convoluted government regulations) were deemed amateur.

See the issue here? Rare and demanding quality programming itself is no longer a pro sport. “Science” of AI and other flashy kinds is. Just like it was vague dignified “enterprise architecture” before it. All three: programming, data/AI science, and software architecture being completely foreign to rich and influential, yet technically illiterate MBA decision makers, declaring something worthy.

Has the world fixed all programming bugs and resource leaks to prevent hourly server crashes of mission-critical software installed at banks and insurance companies? Has it finally got rid of the prehistoric 1970s mainframe aka “legacy” systems that weren’t expected to outlive Y2K? No, it hasn’t. Long after annexing 1B population India to the Western IT labor pool the Earth is running out of programmers to keep the old crap alive. Along with tons of spaghetti mindlessly typed by code monkeys in “newer” (circa-2002) enterprise Java.

If you ask me, the world will survive w/o sharper consumer gadgets and cleverer (still quite annoying) Facebook content push and likes analysis via ML and other data science algorithms. However it cannot afford the real Y2K crisis, which, make no mistake, is coming sooner than everyone expects, when everything produced at/for major banks, hospitals, and other Fortune 1000 IT departments since 2000 succumbs to exponentially multiplying bugs and memory leaks. While smaller information-centric businesses like doctor’s offices and solo insurance agencies suffocate w/o true enterprise software, which was declared multi-million Oracle and Salesforce domain. AI is not going to write those systems for SMB customers any time soon, is it?

Meritocracy is not so simple outside sports. There is a lot of logistical obstacles between one’s strength and the prize, the biggest one — applying the professional strength to build something that sells to pay the well-deserved reward. The real problem here is not the cheapness or unfairness of one’s bosses, but, like I hinted above, the lack of the “tournament” itself. You claim, that you always finish first. Finish what? A Sunday ride? Even the longest and toughest ones still do not qualify as races. A pro race has advertisers and other sponsors (meaning customers) backing the big prize financially. Why you are competing with your amateur buddies instead of pros? How one gets invited to exclusive races? Everything falls into place once you join the race and apply your strength to win it. But how one finds the right race?

Now, we are not pro cyclists waiting for agents to notice us. That (recruiting) system barely worked before the so-called “outsourcing” which dumped a billion of people claiming to be top athletes on the market.

We are engineers. Our “higher gear” is exceptional analytical mind. We design and build things for living. We guarantee the reliable function of our products. Shouldn’t we apply our logical thinking to come up with a consistent: predictable and reliable process to find the necessary foothold — to jump up, translating our (technical) strengths into money?

Pedaling in a Higher Gear

Pedaling in a higher gear is sure fast… once you get used to it. Whether one does everything in life for fun/joy or finds satisfaction pushing past the comfort level, making an extra effort is hard and very unpleasant. This post is biased and personal. I chose “work hard, play hard”. Something tells me, that if you are concerned with your compensation fairness, you rarely buy the “work-life balance” HR BS. The ultimate meritocracy is about rewarding the painful extra effort. The Tour is not a leisurely beach cruise, you know.

Things seemed clear when I embarked on the first version of Lionstack development platform codenamed Px100 framework. It stands for Productivity x100 if you are wondering. Like a good engineer, I carefully planned everything. I ran that plan several times in my head. I tweaked it during the development. Everything checked out. So I thought — until the “higher gear” surprise.

Px100 is not rocket science — for Google or Amazon to “poach” me via a special hiring route bypassing the long line of tiger mom bred perfect GPA scholars competing in algorithmic interviews. There is no mind-blowing ML or another AI science backing Px100. It is assembled of conventional mature pieces like Spring, Mongo, and distributed memory grids, albeit connected by very unconventional Java code. It incorporates well-known programming patterns, as well, as many lessons learned over my 25+ years in IT: primarily of what not to do.

I just thought: let’s use object-oriented programming for its intended purpose: modeling the complex outside world. OOP is crucial to model complex drug molecules and decipher human genome. Why 30+ years after C++ was invented everyone in business process automation still writes dumb “structs with functions” so to speak? I know, who cares about old fundamentals, right? It is the trendiest abbreviations, that allegedly get you a little extra — at least until the hordes of outsourced code monkeys catch up in a few month by cramming the same tutorials and putting the same acronyms in their resumes.

I thought: let’s use 100% of core Spring to find the right balance between the configuration techniques essential for code reuse (connecting various existing pieces) and functional programming to inject little pieces of precise logic directly where they need to be. How many use Spring this way? Every Java developer just has to “use” Spring, right? It’s always been there? I happen to remember the world before Spring. It was bad. Spring wasn’t invented to be put on resumes and get people hired. It solves the fundamental software development problem: configuration of the code and services.

Cumulatively everything from big decisions e.g to use RAM grids as a primary database to little optimizations clicked together like I planned, and I got what I wanted: a precise tool to write 100x less code compared to the old-school (mainframe era) ERPs and dysfunctional enterprise Java teams, staffed by outsourced code monkeys, and 10x less compared to the current best of breed enterprise software development platform: 1999 Salesforce and “good” (boutique) Java solution providers. Salesforce’s Apex looks a lot like Java, so the productivity difference is minimal, if you are wondering.

One would naturally assume that less (code) means less effort. That’s where my “higher gear” metaphor came from. See, it’s not about code. Unless you are developing some blog, messenger, game, or another single-purpose consumer product, a complete automation of an average moderately complex business process e.g. auto dealer’s or insurance agent’s daily routine involves thousands of little pieces of logic: ifs, loops, exceptions, scheduled/timed operations, and tons of other stuff way outside of an average single-purpose consumer tech (Google, Facebook, etc.) developer attention span.

Google and Facebook do amazing (scientific) things. But they don’t want to touch enterprise software with a 10ft pole. Using the cycling metaphor above, consumer tech is indeed akin flat track racing, whether business software development is a thousand-mile multi-stage endurance race. Here is the cycling version of the outsourcing. Come on, you knew I couldn’t resist.

You are asked to deliver some package from point A to point B by taking the shortest route: a tough mountain road. Your competitors won’t even last the first 10 miles. So they pick a five times longer, though flatter detour (writing more code and integrating barely useful modules you can simply write from scratch to the exact spec, etc.). Their team has more riders, each riding for 20 miles and then falling breathless, replaced by the next teammate. They’d beat you, right? Strength in numbers? Depends. The package is fragile. It can only sustain a certain amount of handling — passing it from one rider to another. The software breaks, as more people, especially the cheapest “coders” work on it, introducing five bugs by fixing one.

Now, the real question is, if there is enough pro racers like you to handle delivery of all fragile packages. Or the world has to cope with broken and lost packages, attempted to be delivered by cheap amateurs?

Here’s a better one: does every pro has the right bike to pedal fast (in the higher gear) and deliver the package in time? I am not a scientist, employable at Facebook or Google, which see themselves as privately funded giant MIT and CalTech labs nowadays. The only science, that applies to Px100 is the First Law of Thermodynamics (known as the Golden Rule of Mechanics in the gearbox context). No energy can be created or destroyed. Behold the classic concept of a higher gear vs. going slowly and wasting more energy e.g. on friction. But hey, the energy (money) doesn’t disappear. It just goes into different (than yours) pockets, particularly middlemen’s.

What Is That Damn “Business Logic”?

The creative part of enterprise programming left after automating most of the technical plumbing, is called “business logic”. It’s always been a somewhat controversial concept in IT. I am not talking about the so-called “middle tier” of the classic three-tier mobile or Web application. There is no logic there, just dumb passthrough to the database; made obsolete by NoSQL databases anyway — by saving captured user input in its natural: hierarchical (“object-oriented” if you will) format.

The real business logic typically lives in the UI, evident by the popularity of UI-centric robust JavaScript frameworks like React. It is responsible for user input validation, all neat animations to hide/show and enable/disable fields and sections, screen navigation, security (who sees what), and the general flow of data.

It sounds somewhat like this: “And when the user enters a value greater than 15 into this field, the back end should call the following API to verify that the current user’s account has permissions to do that, and if he/she does, the following three fields are disabled, while the other five appear in the right column, the last one highlighted in red, along with the updated main menu badge across the Account Limits item. However if the user have used all his/her credits and no longer has the permission to bid on more than 15 counts, the system should display a popup with the following four fields, the last two enabled only under the following conditions…”

I am making this up, but you are starting to get the idea, aren’t you? Add contradicting billing requirements for shits and giggles. “I want to charge my subscribers per seat (user), but I want to offer both monthly and discount annual plans.” Well, how the f-ck the computer is supposed to know how many users one adds during that year if he/she paid upfront for the whole year — which was the whole purpose to “save” by prepaying and never worrying about additional charges.

It gets better. You want to give your subscribers a month-long trial period. What if one signs up, uses some expensive service e.g. mass-texting, says he/she is not interesting, closes the account, and signs up again from a different IP? Gotta track the content (record names, emails, locations, etc.), right? If that wasn’t bad enough, absolutely everything should be correctable and reversible e.g. most charges should be refundable, nothing really gets deleted, since the customer can say “oops” and ask you to “undelete”, etc. etc.

All those little things add up, completely invisible to the end user. The magic behind the scenes — the only kind of “business logic”. Remember what I said about the attention span? It becomes an exercise in remembering little things and running all those what-if scenarios in your head. IT has come up with many ways to capture business logic e.g. “use cases” among other types of documents and diagrams. I find indented bullet lists the most concise and straightforward, as illustrated in the post that started this series.

My bullet lists grow details-wise as I discover and recall more logic pieces. E.g. a phrase “update the menu badge” looks simple enough at first, but when I actually start implementing it, I’ll be asking “Wait a second… How do I calculate that badge? Does it count the account limits already paid for or adds the customer service issued credits as well?” OK, I don’t want your head to explode, so I’ll stop.

One interesting observation I made in my very first Px100-based project, was the absence of another, more technical, yet straightforward programming logic (vs. domain-centric business one described above): the so-called “plumbing” aka “glue code”. We are taught to eliminate (hide) as much technical plumbing, as possible, freeing us to work on the precious business logic — the only one that matters to the customer. A typical example of programming logic is translating data from one format to another, because some database or third-party API expects a flat list instead of an object-oriented hierarchy and vice versa. Another example is making library or API calls in a specific sequence to gradually process the data or calculate something.

Those logistics have nothing to do with the task at hand: implementing customer requirements. But, strangely enough, periodic “distractions” like that let programmer’s mind relax by dealing with familiar single-purpose, well thought out, and generally more straightforward concepts invented by fellow geeks instead of the confused non-technical customer, let alone the government with its convoluted regulations. Not to mention that most of the technical stuff is simple copy-paste from a manual, tutorial, or your own code, whether every business rule is unique.

Can one handle just business logic all day long? Can a human breathe pure oxygen? You know what’s limiting the current fighter jet performance i.e. combat turn radius? Not the powerful vectored thrust engines. Not the airframe strength. The pilot’s ability to sustain high-g maneuvers. So while I don’t think Px100 “gearing” is too high, it’s pretty damn uncomfortable. I just have no other way to succeed, than suck it up, pedal in a higher gear and deliver something, a team of a 100 cannot — to claim my prize. While others leisurely cruise “reinventing” Facebook and Snapchat, or competing with free Zoho’s form-filling apps.

The software industry has seen numerous attempts to eliminate “non-business” logic — typically as a naive desire to eliminate programming itself, turning software development into a DIY activity for “power users”. Turned out, all “simplified” business-oriented languages starting with their granddaddy COBOL and all the way to polished Java-like Salesforce Apex still had variables, if statements, and loops incomprehensible by the smartest computer-literate “users”, and thus still required programmers. The “simplification” just made programming more awkward, removing the complex tools pros rely on.

Imagine conventional hydro-coupled automatic transmissions in race cars — in an attempt to popularize Formula 1 making it a soccer mom sport. Because those damn racers make too much. I am not being a geek snob. Until the true AI replaces programmers the world still needs them. Let the pros pick their industrial-grade tools. That’s what I did with Px100 — solved the same problem (eliminating the technical plumbing) the programmer’s way. The result was somewhat of a surprise, like I said.

After enough Spring, OOP, and FP alchemy I found the right recipe to completely hide the technical plumbing inside the configuration. Now I only had to worry about the business logic i.e. write a thousand lines of code devoted solely to say 100 business rules (10 lines per every piece of business logic) on average. Vs. the same 1000 lines of conventional code daily, that only covered 10 business rules at best along with all of the manually coded boilerplate technical plumbing. We are wired to subconsciously judge our performance by coding effort. I feel bad if I only write a 100 lines of code covering the same 10 business rules. Now, if I still worked for the man, I probably would limit myself to those 10, paid zero for doing extra. But I am my own worst boss now, and I want to do more, paid directly by my customers.

It is not typing the code, that wears you down. It is the “creative” part: analyzing the business rules and making sure everything fits. “Damn, I forgot about this little condition here.” “I cannot do this because it has dependency on the output of these two validations.” “The user wants everything on this screen instantly. How do I pretend to calculate those five charts in real time w/o overloading the server?” The average programmer’s comfort level is two-five of the aforementioned thoughts daily. You work on two-five “tickets”: bugs or little enhancements describing one business rule each, log your hours, and go home. Experts do 10 when they feel like it — the limit of the conventional technology. A 100, made possible by my “code condenser” tool? That’s pedaling in a higher gear.

Excruciatingly painful to start, it becomes tolerable once you get up to speed. If that wasn’t bad enough, as you probably expect, multi-tasking i.e. brain CPU switching is extremely unpleasant at this level of concentration. Next come hiring woes. Finding developers that can sustain the pace, is hard, though I only need a few. A team of five Tour-level pros can take on a project of any complexity.

Again, this is my personal effort/reward ratio. By all means, tell me about your plans to get rich by taking things easy. The laidback cubicle life of the 90s, as well, as I am sure several decades prior, has been uhm… outsourced. I don’t think today’s top “slash-architect” stateside developers even deserve their $150–180K — half of inflation-adjusted pre-outsourcing wages. It puzzled me big time, why I was paid $90/hr ($180K) upon coming to the US in 1996 — to write boilerplate DAO and DTO crap and report hours. Mystery no more. $5/hr ($20–50/hr billing rate) code monkeys do that now with $90/hr “architects” providing the oversight and cleaning up all inevitable messes. I only “signed up for that shit” (quoting Michelle Rodrigues in Avatar) for the first couple of years in the US. I want to do (and make) more.

Now, whether you can “do more” with conventional technology catering to bloated teams of mediocre developers, busy typing boilerplate glue code, is a different question. Beware, if you go my route and develop a more advanced tool, you don’t retire by selling it to Oracle or anyone else in the headcount-centric Great IT Consulting Food Chain. Let alone Google, that doesn’t give a sh-t about business logic heavy enterprise software. They wouldn’t know what to do with your higher-geared bike. You’ll have no choice, but to take your purpose-built equipment and start pedaling in a higher gear.