IT Meritocracy. Part 1: Can You Found Your Current or Other Known to You Employer?
My Journey from Glue Code to the Systems I Build Today.
I discovered programming in mid-80s at the age of 13. There were no personal computers, let alone smartphones. Our (middle and high) school rented 1/3 of two floors to some mysterious data center, which had AFAIR two CM-4 minicomputers: Soviet ripoffs of PDP-11. A couple of other, luckier schools in my two-million city (Minsk) had “proper” kids’ computers: ripoffs of Commodore and Apple II. Though I do not regret, that my first OS was UNIX and first languages: industrial FORTRAN and C instead of the “proper” beginner’s BASIC.
First Jobs.
Fast-forward to my college years, I got my first internship at the AI Lab of the National Academy of Science during my freshman year: 1989. It happened by accident. My family had no industry connections whatsoever and I needed access to a computer. Any computer. Turned out state-funded scientists had them. If you are counting years, it was almost three decades before the Silicon Valley and Forbes “tech trend” analysts discovered AI or ML.
The following year I got a part- and then full-time job at one of the three software companies in the same two-million city. The economy completely collapsed following the fall of the Soviet empire. Not that it had too many (non-military) software development opportunities during the Soviet era. My family had relatives in the US, so those (defense clearance) doors were permanently closed for me. Luckily the communist regime crashed just in time.
The company I worked for specialized in banking software, since the young post-Soviet banks were the only clients who could pay in that economy. I was trusted with the architecture and development of a specialized bond trading system, which I completely blew if you are wondering. I made all kinds of beginner mistakes, the biggest of which was modeling the UI after my favorite game. Windows 3.1 just came out. A mouse was still an exotic device. Drag and drop blew my mind. It was one of the first GUI (yes, the graphical one vs. the text DOS UI) business systems in the whole ex-USSR. I wanted to make it slick and pretty. The users needed something else. I learned that lesson well.
By the time INS finally processed our application and my family permanently moved to the US in 1996, I job-hopped to another (second out of three if you remember) Belarusian banking software company and developed a similar trading system there.
Finding a programmer job in the US was easy. They paid well in the 90s. I don’t want to lie, that it bothered me.
I wish I had Elon Musk’s balls to never work for the man instead of accepting the golden cage, IT was at the time.
It was just a bit weird: the change of pace, compared to the first five (Belarusian) years of my career. I worked my ass off at those two banking software companies. I did things no one dared to before me. And I got rewarded for it. Meritocracy worked.
In the US though… it “worked” too well. I was paid $90/hr just because I knew C++ along with a few “hot” frameworks. All I had to do was copy-paste DAOs, DTOs, and other boilerplate code. And log my hours of course. I thought “Alright, if the country, meaning its Fortune 100 corporations, is so rich and generous, I’ll gladly take what they pay me.”
That era abruptly ended in 2002, which marked the beginning of mass “outsourcing”.
Millions of IT jobs moved to India practically overnight. The pay rates were slashed in half and remain such today — adjusted for inflation.
At first, senior developers like me were “promoted” to slash-architects, tasked to oversee the “discount resources” and clean up all the messes. It wasn’t the end of the world — just a realization, that I had reached my compensation cap. Honestly it happened several years before and had nothing to do with the “outsourcing”. The latter simply lowered my cap, making it obvious. From that point on, no matter what I did, how I sold myself (as a dignified “architect”), and what employer’s games I played (bonuses, stock options, etc.) I couldn’t increase my pay.
Looking back, yes, it sucked. No one likes pay cuts. But aggressive job-hopping and other desperate attempts to find a “different” employer allowed me to see it all: B2B and B2C, Fortune 100 IT departments and small solution providers, “funded” startups and dying eternal ones, founded with a vanilla website during the dot-com era.
I am ashamed to admit, that it took me more than a decade to realize, that no one was going to be paid pre-outsourcing wages for writing DAOs and DTOs, and more importantly, no one in corporate IT departments and the Great Consulting Food Chain serving them needed me to do anything else. It was my, not recruiter’s responsibility to match my skills against great projects.
Has it ever been possible in American IT? Of course not. But by 2010s the new generation of employers, led by Google emerged outside of the rotten corporate IT (meaning IT departments of non-technical companies like banks and insurance carriers, plus the Great American IT Consulting Food Chain serving them). Not only “Googles” respect engineers. They pay adjusted for inflation pre-outsourcing wages: $300–400K in 2018.
Unfortunately the handful of top tech employers (Google, Amazon, Facebook, Netflix, and a few others) are 100% consumer aka B2C. My enterprise experience means nothing to them, not to mention their well-known desire to hire fresh grads and groom them into ideal scientifically-minded “Googlers”. I’ve always had a problem with grooming, so I decided, it was finally time to quit the rat race and see how I fair on my own.
I am still learning my startup lessons.
I’ve changed two co-founders, finally finding my soulmate: Jason Barber — a street hustler, rather than all-talk VP of Sales, or worse one of the “people with ideas” in need of a cheap developer. Ironically, Jason has a recruiting background, albeit not in IT. He did something, no IT recruiter could do for me in 20+ years here, in the US. Jason gave me the foothold to jump up, applying my skills and hard work towards something orders of magnitude greater than DAOs, DTOs, and other mindless typing to look busy and make middlemen commissions.
What is it? Google, Facebook, and others brought us mind-blowing consumer tech: mobile gadgets, social networks, ML-backed marketing… the list goes on and on. Meanwhile corporate IT has been celebrating the Y2K “aversion” (prolonging the 1970s systems end of life beyond 2000) for the past 18 years. Google doesn’t want to touch targeted i.e. custom business software (workflows, reports, databases, etc.) with a 10ft pole. Mobile games and social networks are great, but companies, especially smaller ones, are suffocating w/o mission-critical business software.
Sales is sorcery to me, just like programing is for Jason. He is amazing. It just seems too easy. Selling, that is. The development is a bit different. I call it “pedaling in a higher gear”, which I’ll explain in this series. Here’s an example.
A Real-Life Business Problem and Solution.
I just finished an MVP for an all-in-one insurance agent SaaS. There is nothing like that on the market and not going to be for a while (developed by the book), which I’ll get into in a bit. Obviously I can’t possibly describe the entire application in depth in one post. The very last feature, critical for the MVP, was mass-texting and emailing.
The exact customer requirements:
“I need to run text and email campaigns for my leads and clients”.
There was no business analyst to explain it, and no solution architect to pick the right APIs (already used for previous projects actually: Twilio for texting and Amazon SES for emailing). I am going to skip other IT roles: UI designer, front-end developer, back-end developer, DBA, DevOps, etc. As you can probably guess, the MVP was delivered by a one-man “team: of yours truly — to hand it over to the newly hired developer for further work.
The new developer is a generalist, like me: doing everything from talking to the customer (representing her SaaS clients) to integrating with third party providers: insurance carriers and their aggregators/hubs. The project is well-planned for the next six months with standard two-week releases. There weren’t any surprises during the three-month MVP development, and won’t be any during the v.2.
I’ll explain the generalist approach in a bit. The schedule is rather relaxed, because we let (our proprietary) technology do the heavy lifting. Without it, done by the book, comparable projects I was part of during my IT career employed about 20 developers and coordinators of several kinds in various narrow specializations. You’ll get a taste of the overall scope once you look at the detailed task below. Remember the requirement? It resulted into the following list (copied verbatim from my internal design document).
Just scroll through it. I pasted it here to give you a rough idea about the so called “business logic” — what it takes to implement the simplest of client’s requirements, often invisible to the user (until something goes wrong) and almost always taken for granted. Still wondering “why damn programmers are paid that much?”
- Campaign entity (used for both text and email)
- user (campaigns are by user)
- can only be edited by the same user, though can be copied
- type: text or email (passed to the constructor)
- status: New, Finished, Canceled
- campaign name
- recipients:
- customerId
- cached description (name, etc.) for the list
- noContact (phone or email)
- sending result: null — not sent, true/false for success or error
- lastModifiedAt
- extends EmailTemplate since we need to attach docs to it
- only the body is relevant for text campaigns
- default value for text campaigns: “Type your message here…” and “The ${tenantName} Team.”
- similar default value for email campaigns
- additional replyEmail dropdown for email campaigns (see below)
- make some robust list of fields based on customer (name, dob, etc.)
- no drilling down to cases though
- one test (fake customer, current user)
- execution (non-persistent):
- schedule an async task via TenantProfile.startTask(): returns a task ID
- utilize message group events similar to chat: campaign<campaignId>
- updates: every 5%
- tracked inside the Campaign entity
- the page subscribes to them and refreshes the progress HTML
- shown only in the executed state
- execution:
- update the campaign with the executionTime and taskId, commit the transaction and create another one
- iterate over the non-error recipients:
- check the abort flag via TenantProfile.isTaskCanceled()
- text: send SMS (Twillio can do one at a time)
- verify it is logged in the service log
- the long body does not prevent the campaign from executing unless it is double the max size
- email:
- add the following context params:
- replyEmail: based on the campaign’s choice
- send email synchronously via ((DefaultTenantProfile)getContext.getTenant().getProfile()).getCampaignMailer().executeSynchronously(customer, this)
- log as communication: standard subject, text in the description
- upon normal finishing update status to Finished and clear the taskId (then save in DS via the ctx.tx — it is still valid)
- canceling does nothing
- recipient logic:
- remove (list of IDs)
- clear
- add (list of IDs)
- add (baseCriteria, list filter values, AND semantic)
- reuse ListView.criteria()
- refresh logic
- Campaigns menu:
- list view of campaigns
- filter: type, date range, name, status: executed/new
- additional criteria: current user only for non-admins
- or such filter: Mine/All for admins
- columns: name, type, status, user (for admins only), Recipients, Errors (counts)
- buttons to create text and email campaigns
- edit view
- base:
- similar to EmailTemplate showing how to use fields w/ “Type your message here…” and “The ${tenantName} Team.”
- an email campaign will show exactly the same, as template
- also has a dropdown of reply email
- team (tenant’s — listed on the right)
- own (user’s — listed on the right)
- custom: an edit box (enable/disable logic)
- text campaign will only show the body. No changing the email type, no pictures, etc.
- the body is a different field (similar approach as email template)
- counter of characters, like other texting screens
- navs: Template, Recipients, Errors, History
- Template:
- messageGroups((item, ctx) -> new String[]{“campaignExecution” + item.getId()}).
- update the progress (lambda script — empty if that field is not shown because it is not yet executed)
- always shown when executed
- Delete button (only non-executed ones)
- R/O when executed
- ability to cancel the execution if the campaign has taskId
- call TenantProfile.requestToStopTask()
- clear taskId in the campaign
- set the status to Canceled (still considered executed)
- Copy as Text, Copy as Email buttons: similar to EmailTemplate’s Copy — goes back and opens the new one
- converts to text type when changing the campaign type
- the body can be longer than needed
- if the campaign is executed its status is changed to New
- check recipients for no phone and no email errors
- clear all execution error flags
- Preview and Run button
- test:
- testTemplate()
- put a full-height iframe similar to the doc viewer popup on the page
- Run button
- executes the campaign and goes back (the prev. page should refresh and start showing in R/O w/ the progress)
- Recipients and Errors (R/O if executed):
- a list of all customers w/ ability to open the customer
- page size 1000
- name filter
- click script instead of workspace: does nothing if the user doesn’t have the permission
- to protect against the customers being reassigned
- a fairly detailed traditional list view (no cards). Show the phone (phoneto link)
- show the cached description
- additional info: errors or Pending Email
- run a check in workspace bean getter for non-executed campaigns:
- if the customer’s lastModified is fresher
- update the description
- correct the no phone or no email error
- can add, remove, and clear the list — unless executed
- Add button:
- multi-select list similar to the recipients, only w/ a different title
- page size 200
- load the campaign, and then its getCustomerIds() into the ctx beans in the workspace bean getter
- notHighlightableRows
- robust filters: take from all customers and extend w/ a couple (status, etc. — see deployed Helios)
- additional list criteria: active customers only
- non-admins are also limited to their customers
- two filter buttons:
- Add Selected on This Page,
- normal operation w/ going back
- Add the Entire List
- operation w/ ButtonConfig.convertEditDataToParams (goes back)
- when adding:
- duplicates are eliminated automatically
- checks no phone and no email errors according to the campaign type
- Errors (make it a red span):
- the list of recipients w/ errors
- editing the customer until the campaign is executed can move him out of the errors and into the normal recipients
- run a check of lastModifiedAt and if it was modified check for errors again
- can remove, but cannot add
- turn it into a step by step wizard, so the semi-technical user knows what to do
- test:
- adding recipients:
- test all selection filters
- add selected
- add all
- clicking on already added customer w/o a checkbox
- clicking on a non-added customer
- clicking in the Recipients
- clicking in the Errors
- no errors:
- errors:
- no customer phone (text)
- no customer email (email)
- deleting the error record
- fixing the error record
- add duplicate recipients
- deleting:
- cannot delete an executed campaign
- test all campaign list filters
- executing email campaigns (two of my emails)
- HTML w/ pictures
- copy, modify and run again
- error: bad email
- R/O view
- how the progress is being refreshed
- introduce artificial delays into the loop
- test aborting
- executing text campaigns (myself and Jason):
- short
- fairly long body
- errors: bad phone
- copying:
- a new campaign
- an executed campaign
- text to email
- execute
- email to text
- execute
- admin vs. agent campaigns
- the admin can view every campaign, but the agent cannot
- selecting recipients
- reassign an existing recipient of an agent’s campaign to another agent, login as the old agent and try to open it
It took me one day to plan/design, a day and a half to write (code) and several hours to debug: three days total — while taking care of the essential CTO duties like researching other projects and hiring.
As I mentioned, the infrastructure was already in place. Those text and email campaigns reuse the core email template sub-framework utilized throughout the entire application. It allows customer-configurable plain text, WYSIWYG “rich” text w/ links and pictures and completely custom HTML content. The latter is un-sanitized, yet fully protected against XSS attacks. I know, magic.
Don’t forget the (automatic) audit: tracking who did what, more CRMish automatic communication log: of sent and received calls, texts, and emails, and the special service log to (again, automatically) bill SaaS clients (aka “tenants”) for every text or email, since we pay for them. Anyone who spent a couple of years in the industry knows, that those things require database entities, admin UI screens, notifications, and generally the whole three-tier or more complex shebang. Which, done by the book, typically takes man-months to develop.
See, it’s not rocket science, but if a single customer’s sentence blows up into three pages of bullet points, plus 10–100x more of those for the plumbing itself, that absolutely everyone develops from scratch for a new project… Oh, and that’s before one actually starts coding. All those little things add up, and before the small-time “person with an idea” or big-time “project stakeholder” knows it, he/she will be on a “hiring binge”, which (higher headcount) makes things exponentially more difficult and painfully slow.
It already feels awkward dumping several pages of bullet points here. Obviously I cannot describe the rest of our internal software development platform that made those things possible. If you are wondering, the bullet list style document for the last, fourth version of our Px100 framework, had 3000+ lines. Count the ones above.
Nothing of that nature has ever existed or will be developed (by the book). Anyone attempting to cover the same (insurance SaaS) business requirements would spend 20 x 18 = 360 man-months (a very optimistic estimate) vs. our nine. According to my preliminary (indirect) competition research that is indeed a typical team size and timeframe for a comparable project in the same (insurance) niche.
Reminds you of something?
I am sure there are other (40x in this case) developers like me out there, who founded their SaaS startups in a similar manner. Speaking of email campaigns, MailChimp immediately comes to mind. It wasn’t a thousand-people company when it started. Our main difference with MailChimp, HelloSign, RadiusBob, and other single-purpose SaaS providers is that their services are just modules inside our all-in-one systems. Everyone can develop a generic CRM or mass-mailing tool. However tying everything up in a very custom workflow is extremely time consuming and still unscalable — done by the book.
Thus the majority of today’s SaaS offerings remains single-purpose, driving the end users mad. I’ve personally come across exactly the same issue in at least three industries: auto sales (independent dealers), insurance, and catering. Not only the poor customer needs to spend several thousands monthly on 4–5 systems, but also learn different UI/workflows/configurations, and copy/paste data between those apps.
Any out of the box DIY “integration” of the little single-purpose pieces mentioned above is a pipe dream, let alone orchestrating custom workflows and covering custom data e.g. for our specific insurance niche, divided into complex sub-niches of life, auto, home, and other insurance lines with their own business rules and regulations. If you thought the breakdown above was long and complex, it only covers three UI screens and rather generic texting and mailing logic. Not surprisingly traditional (single-purpose) SaaS providers only offer generic services. And like I said, even the mighty Google doesn’t want to touch inherently custom business software with a 10ft pole.
Every business process is custom. There is no way around custom programming, whether in (best of breed) Salesforce or pure Java. Salesforce’s language: Apex looks very similar to Java, and not surprisingly rough implementation effort estimates are the same per my experience: e.g. the 360 man-months for our insurance system. The language largely doesn’t matter. They are all descendants of ALGOL. And all programming recipes are taken from the same age-old Oracle cookbook.
That’s a lot of cooking considering the number of business rules (bullet items above) a seemingly straightforward e.g. mass-texting and mass-mailing requirement blows up into — way beyond the average “Googler” techie attention span. Though, incomprehensible by consumer tech engineers, it is nothing, but code to write — in the same industrial programming languages e.g. Java.
There is no magic, just more effort to make: either write more regular (book) code, or “pedal in a higher gear” writing more complex concentrated code to cover the same e.g. 10 business rules with a 100 lines of original code instead of 10000 lines of copy-pasted one. All that for a significantly narrower specialized user base e.g. insurance agents. Compared to all businesses using text and email marketing campaigns in the single-purpose MailChimp’s case.
It’s not very lucrative for mainstream tech investors, if you are wondering why they haven’t lined up at my door. Thankfully, no investors are needed. Find a way to write 40x less of “concentrated” code, and you can churn out (meaning hire a couple of generalist developers to do it) those rather “narrow” i.e. precise all-in-one systems for one vertical after another.
Can You Do It?
Let’s revisit MailChimp and HelloSign (or comparable PandaDoc for my Belarusian colleagues). How many developers founded those companies? Scrap that. How many developers wrote the first version of Facebook? VK? Viber? Не боги горшки обжигают (“Gods didn’t make them” loosely translated). What’s stopping you from doing the same? The competition, right? Plenty of mass-mailing and other marketing SaaS. Plenty of e-signature solutions? Do you know how much HelloSign charges per document? $.50 to $1. Do you know how long it took me to develop my own internal e-signature module used in all our products? One day.
Salesforce started with three developers in 1999. I am not going to speculate about their initial technology: language, etc. Armed with today’s robust tools and databases, a single generalist developer can do a barebones CRM in two hours. Not by the book of course. Once I get some breathing room, I am going to host a hackathon to prove it.
Every startup grows. Ours will hire more developers in the next few months. That is not my point. Ever wondered how your own employer was founded? It is only hard to make the first million, right? I can care less about motivational speeches of being your own boss. Just merely describing my way — to get past the compensation cap. It’s within reach. Mere mortals like you can do it. No investor buddies or other “connections” required.
Engineers and CTOs
I know what some of you are thinking. If all code was written by software company CTOs, right? Or at least “architects”… If every cheap auto repair shop was staffed by Formula 1 mechanics… That’s not my point. How about every car designed by Formula 1 engineers? Not many needed compared to mechanics. The car is designed once and then manufactured for years. Provided it is designed well and then built by robots, which don’t make mistakes, not many mechanics will be needed to service it. Sure, those fewer skilled mechanics can come from Formula 1 as well.
Some may bring up the “bus test”: what happens after the single key developer gets hit by a bus? The leanest (self-funded) startup is still a team — after the initial MVP. A 10–100x smaller, or better said tighter team of likeminded and highly motivated experts. Fewer people doing and more importantly earning more.
You are right about one thing though. It starts with a CTO. Generals carry a sidearm during wartime, don’t they? They know how to shoot. They were infantry (sailors, pilots, etc.) once. It comes as a big surprise to my Belarusian colleagues, that most of the American IT leadership — from so-called middle managers (BTW why they are called “middle”?) all the way to CIOs are semi-technical at best i.e. barely able to comprehend Wired articles on technology trends.
I am an engineer first, and always have been. We hire generalists aka “jacks of all trades”, doing the business analysis by talking directly with the customer, figuring out the right architecture, and sketching the UI design, since it is not rocket science in “admin-themed” enterprise systems. Software development involves “management” too — part of a true generalist skillset. Planning, coordination, mitigation, hiring, and other managerial activities require logic and patience i.e. long attention span and superior analytical abilities — natural for engineers. So called soft skills? The matter of training/habit.
Google doesn’t have formal aka “functional” managers, only PM assistants to lead developers. Remove the thin, yet noticeable overhead and other BS of that big public corporation owned by bleak “shareholders” and find out what a tight team of programming commandos can do.
