What tech worker shortage? This one!

This article is circulating, suggesting that the tech industry's claims of a worker shortage are really just complaints that it "can't hire them as cheaply as they would like", instead bringing in cheaper foreign labor on work visas. In general it's arguing that the software industry is trying to suppress wage growth. While there are cases like the non-poaching agreement between some major companies, overall the industry is thriving and there's more and more money to be made.

I find this a silly argument for a number of reasons.

1. Microsoft (and other large companies) hire lots of foreigners into their full-time staff. These people all make comparable salaries, bonuses, benefits to American full-time employees. Additionally, the company must sponsor their work visa, which requires significant HR, legal and financial resources. Essentially, the company is forced to prove needing this person to do the job because there are insufficient local alternatives. So, Microsoft (and other large companies) are saying they would rather spend more money (extra cost all front-loaded too!) to staff the same job. The only incentive to do this is that Microsoft (and other large companies) are unable to find quality employees from the USA (or it's costs more money to find a few diamonds in all that rough).

2. The argument about real-wage growth being flat since 1999 is cherry-picked. 1999 was the peak of the tech bubble. The industry was nascent and there were very few qualified people for a suddenly exploding area. Companies' stock prices were skyrocketing and they were desperate to grow. I expect significant compensation was given in stock and that stock was doing very well. We should see how real-wage growth did from 1999 to 2002, BusinessWeek/Josh Eidelson. I'm just a single data point, but since starting at Microsoft in 2007, my real-wage growth has been very significant (without being too specific, over 5 percent per year, compounded). While I may (or may not) be promoting faster or performing better than others, I know for a fact that a median performer's comp growth easily outpaces inflation. For other major companies to be competitive and not lose workers, they need to be keeping similar pace as well.

3. Higher initial salaries are unlikely to drive much more interest. Tech already pays very well, especially considering the high-paying jobs are available directly out of college, canonically at age 22. Many of my peers were single-family detached house owners by age 23, and could also comfortably afford a fairly nice car and had money to spend on travel/dining/entertainment. Unless wage suddenly jumped to day trader levels or something, I don't know that the industry's appeal would elevate. A smaller increase may serve to re-route engineering-minded people from other areas (like mechanical, civil, chemical, electrical, ... ), but then we'd have a shortage there. Or, it's all enough money that people will follow their passions (which is better for the industries anyways).

4. Major employers like Amazon, Microsoft, Google, etc are perpetually looking to hire. In other words, they're chronically unable to fill positions for all the work they'd like to get done. So, it's not like they are just driving some excess of employees to lower-paid/contract jobs.

 

Of course tech companies (like any companies) want to pay their workers the least they possibly can while still getting good results out of them. Their competition with each other has lead us to the state we're in. And the biggest constrained resource in growth is qualified people. The end.

PS: I participate in interviews regularly, and can personally attest to the lack of amazing candidates.

 

Overeager beaver

I'm currently in Houston. As usual, I flew United. This is the norm because Houston is a United hub and there are many direct options a day.

The flight was actually somewhat icky. We took off 20 minutes late, which is minorly annoying. We hit a lot of turbulence along the way, which happens. I'd never actually had the flight staff not come through the cabin for final cleaning before, so that was a little different. So was them over-explaining that they were being asked to sit by the captain. That was also a touch awkward. But whatever, I arrived in one piece, half an hour late, no big deal.

A day later they sent email, asking me to fill out a survey about my experiences. I don't like surveys because people have a hard time being honest when they are put on the spot. They either don't want to say anything too negative, or they go into bitch-rant mode, or they say nothing meaningful.

Here's the survey:
1. On a scale of 1-10, how satisfied were you with your flight.
I dunno? I mean, I got there. I was a bit late. The staff made me slightly unnerved, but through no real fault of their own. I was sitting next to a 4-year old who was wriggling around the whole time and kept kicking me. They only weigh like ... 30 pounds? So whatever, I barely felt it. It's not a big deal. The overhead light wasn't quite lining up with my book, so that was slightly annoying. Putting all that into the Gergely heuristic engine returns, in this case, a 7. Don't ask me to elaborate, cuz fucked if I know why I picked 7.

2. On a scale of 1-10, how likely are you to recommend United to others?
... and, carry the 1, multiply by waxing-moon offset and polish: lucky 7 again! Ugh. The indicator bar, which clearly has 10 stages, is currently showing that this is the 2nd. I have to answer 8 more questions??

3. What is the top thing we can do to improve your satisfaction rating?
Hey, that's a solid question. It's actionable. It forces me to prioritize. It doesn't matter if I've given them a 1 or a 10. I said "be on time". I later regretted that and wished I could go back and say "don't gouge me for flying around Thanksgiving".

4. What is the primary reason for your satisfaction rating?
Aww crap, they've asked me to elaborate. And the best honest answer I can give is:

5. Please share with us any other feedback you have.
I decided to just roll with the above and give them a brain-dump, the essence of which is that sending me a survey is just annoying, and how come they can't learn what they need to know from telemetry they already have?

Thankfully the progress indicator was buggy and at this point I was done.
However, I really do wonder how much useful information they get back? or are they just trying to make me feel like I have input? or are they just unable to figure out where/how I fly and then cross-analyze when I use United (to/through Houston) and if I'm opting for other airlines for the same routes at times?

Or does this mean we can't tell if someone is happy unless we ask them? And then we may or may not trust what they tell us?

Solar energy almost won

I was watching Cosmos last night. In episode 12 it addresses atmospheric carbon dioxide. Several interesting and daunting facts are presented:
1. There's no question warming is due to the level of CO2 in the atmosphere.
2. The CO2 levels are unprecedented in all meaningful historical contexts.
3. We produce orders of magnitude more CO2 than all the volcanic eruptions on the Earth, combined.
4. There's an easy-to-describe positive feedback loop contributing to the warming:
   a. Higher CO2 leads to higher heat.
   b. Higher heat leads to polar ice melt.
   c. Less polar ice means less highly reflective surface, leading to more retained solar energy.
   d. More retained solar energy leads to even higher heat, resulting in more [b].
5. The arctic tundra/ice has as much frozen organic material locked up as the total carbon currently in the atmosphere. If that starts to melt and the material essentially rots and is released, we get more [4.a]. This scenario is likely "catastrophic and irreversible".

So how did we get here? The culprit is really the invention of the engine. Before the engine, our primary carbon emissions were firewood and cow farts. The engine lead to widespread automation of mechanical tasks. A machine could be built once, then supplied with an energy source to produce way more work than any human is capable of. Humans started doing meta-work: they built the machines, which could then be leveraged to do energy-consuming tasks and to build other machines. In this way, we can essentially scale to arbitrary energy consumption capability. The only limit is how fast we can collect energy sources.

The particulars of the engine design were such that motion is driven by expanding gas. The first design (steam engine) used gas pressure to drive fan blades attached to a wheel. The gas is steam, and is created by heating water. The heat comes from burning coal. The design, in some sense hasn't been updated since. The internal combustion engine is the same concept, just substituting a self-contained piston for the wheel blade, and generating gas pressure through the direct explosion of an air-and-carbon fuel mixture. Even our mighty nuclear power stations use this approach; they just use nuclear reactions instead of burning carbon to produce the heat (hence why they are, in at least this dimension, much cleaner). However, this leads to a natural question: why, if the heat doesn't have to be carbon-powered, did we end up using carbon (and still are using it) as the primary source?

The easiest reason is that carbon is plentiful, distributed around the world, energy-dense, and highly portable. No special engineering is needed to store it or distribute it. Refilling a train with coal or a car with gasoline is a snap. Each can carry enough fuel to cover meaningful distances. We simply didn't have (and still are in the early stages of developing) other energy sources that fit the above criteria. Electric cars, for example, are becoming more common, but they are still limited in their role and convenience. And, if a solution is adequate and incrementally improvable (energy efficiency, mining efficiency, ...) and there seem to be no fundamental concerns, why change?

It turns out, we could and should have seen all this coming.
A 1932 paper by REO Hulbert described the theory of climate change due to atmospheric carbon dioxide change. And a 1938 paper GS Callendar actually called out observed warming (though he actually argued that they were a good thing because crops could grow over a wider range of latitudes). You read a thorough history here. And, we knew as far back as 1896 that doubling our atmospheric carbon dioxide would melt the polar ice.

And think, if not for chance discovery of lots of North African oil, solar power may have taken off a hundred years ago.

When in doubt, make a shitty connection

Oh no, a popular whiskey was recalled in 3 countries due to a toxic antifreeze ingredient!!
Good thing other countries still let people chug Fireball to their liver's content.

The issue at heart here is the attention grab of comparing the Fireball to antifreeze. Why? Because they have an ingredient in common. I suppose I'd better stop eating kale because it shares the ingredient "water" with extra-strength bleach ... and no one without a death wish would ever ever ever drink bleach!

As a smart reader, you already know this is all a logical fallacy. A and B contain X. Given that B is toxic, that does not mean X is the toxic component of B. Furthermore, even if X is toxic, we'd need to know a lot more: how much X does it take over what timeframe to harm a person? How much X is in A? How much A does a person ingest in a canonical and an extreme scenario?

The latter is a staple anti-vax argument about things like thimerosal and formaldehyde. Sure, both these ingredients are acutely harmful in large enough quantities. In both cases, there are key points not included in the argument: thimerosal is a much less absorbable mercury variant and poses far smaller hazards. Also, the amount per injection is well below the safe level limits. Same for formaldehyde. It's used in the production of vaccines (I believe to kill the infectious agents) and the removed. Trace amounts are left behind. So yes, vaccines contain formaldehyde. Turns out the body produces formaldehyde as a metabolic byproduct. And the natural concentration leads to something like 1 million times more formaldehyde quantity in you than what you'd get from a vaccine.

If something is dangerous, point to the exact reason why it's dangerous. I'm not arguing that Fireball should have propylene glycol in it, or that it couldn't maybe do with less or none, but if it's truly bad, let's talk about that. Not antifreeze.

Note: even some news outlets (Huffington Post, Gawker) who have mysteriously risen to non-trivial credibility are pointing out the antifreeze "link".

Car, take me there!

Google and others have been playing around (quite successfully) with self-driving cars. Wouldn't it be great if we all had access to one of those? And while sitting in traffic, we could go ahead and do email, play on Facebook (or Instagram, or whatever is popular then), watch movies, etc. Or partake in any of these other activities?

However, I don't think moving to a primarily autonomous vehicle world would be as incremental as people think.

I've come to a similar conclusion as the linked article that individual car ownership would become obsolete. Even a heavy commuter only uses their car 2-3 hours a day, why would they pay for the other 21 hours if they didn't need to? If you no longer own your car, you wouldn't spend a bunch of money configuring it to your liking either. We'd probably end up with a smaller selection of vehicles in the world, primarily differentiated by size. Granted this is all on the pretty far end of the spectrum of possibilities, we may instead just see Uber-esque companies pop up that maintain a fleet that we can all share. Come to think of it, most usage would be in commuter hours, which would leave a lot of cars sitting around ... I suppose we'd see higher rates at those times.

All of the above would grow organically along some path. We can speculate all we want, but most likely it would just happen as it makes sense. What is a lot harder to figure out for me is the licensing model and responsibility.

If I buy a self-driving car, who is liable if it's found guilty of an infraction? I see two major possible models:

1. The company who created the software (or possibly shared with the automaker who put the software on the car). While some estimates state that accidents would reduce by an order of magnitude or more and fatalities would drop from 30,000 to around 1,000 per year, no company in the world would sign up for potentially being liable for all, some, or even any of those fatalities. So would the company make the consumer sign some blanket no-fault agreement? Would you, as a consumer, accept all liability when, inevitably, a bug causes a crash? This is essentially a central responsibility model and probably doesn't scale well.

2. The company who created the software sells a license to the user. The user then installs it and takes the robot to get a license. The robot is associated with the car and the person from an insurance and liability standpoint. The responsibility is shared somehow (though it's not quite clear to me how this would happen well).

Just musing out loud here, but I think the license/responsibility model is the biggest barrier to autonomous cars. That, and the lack of fun-to-drive-ness. Can software robots drive stick?

Musical cars

So, you've been driving stick, eh?

 

Next level: do you do double-clutching and heel-toe-ing?

I find both to be useful, especially with the close-ratio gears for higher speed situations when I need extra torque. Heel-toe-ing covers the added scenario of brakes being involved, usually that means going through a turn.

Double-clutch

Scenario: Cruising 70mph in 6th and suddenly you want power. Downshifting to 5th isn't quite gonna cut it, so you probably want 4th. However, the ratio diff between 6th and 4th is too big for the synchros to handle cleanly.

Sequence: Clutch in, stick to neutral, clutch out, rev to match (or really slightly exceed) speed in 4th, clutch in, stick to 4th, clutch out.

Why it works: the clutch disconnects the engine from the tranny, whereas neutral disconnects the tranny from the wheels (but leaves the engine attached). Thus, you can use the gas in neutral to get the revs from the engine transferred to the tranny. The wheels are disconnected, re-engaging 4th with the revs now matched connects everything. Clearly you have to do it fast enough that your revs don't fall during the post-neutral steps.

Heel-toe

Scenario: you need to slow down but want max power right after, such as in a turn.

Sequence: operate brake with toes of right foot. Execute double-clutch as above, but using your right heel/side of foot (depending on pedal layout) to operate the gas pedal.

Why it works: it parallelizes slowing down and shifting to the lower gear.

This one is a mental cluster-fuck. Ironically the first time I ever tried it, I just nailed it. Then I started thinking about it and it all went to hell.

Like so:

https://www.youtube.com/watch?v=8By2AEsGAhU

Both of the above require matching revs, which you can do mathematically based on gear ratios, however, you won't ever do math while driving. I rely on a "feel" of how deep and long I need to hit the gas depending on how many gears I skip, but this is naturally intermittently successful. It is possible to map it to music though.

Example: My car can hit 55 in 2nd, 80 in 3rd, 105 in 4th, 130 in 5th, 160 in 6th.

In the above example of 6th -> 4th, the ratio is almost exactly 2:3, which is almost exactly the ratio of a root to 5th in a scale. So, make it sound like a power chord :) Extend.

Salary disparity

Yesterday's comments by Satya Nadella at Grace Hopper have gotten a lot of people in tech talking about the inequality of pay between men and women. There's also the commonly quoted gap in percentage of each gender in tech, but I think that's a largely (or entirely) separate issue.

So why do women make less money for the same work? Multiple possibilities exist, and likely all of these (and more) are factors:
1. They typically don't demand higher pay
   a. they don't promote themselves as effectively
2. Hiring managers undervalue them
3. Hiring managers know they can exploit [1]

As with all things free(ish) market, there's the buyer's side and seller's side. These two must overlap for there to be a mutually beneficial outcome.

The seller (business)'s side:
The business's interest is in staffing a position with a qualified person for as little as possible. Considerations for as little as possible can range from literally the lowest dollar amount someone will take to long-term retention and fairness considerations. A business can:
B1. Treat women equally. All is well.
B2. Be fundamentally biased and value comparable women less than men
B3. Be aware of gender-based trends and try to lowball women because they just might get away with it

The buyer (potential employee)'s side:
The employee's interest (assuming all else equal) is to get the most money for the job they are doing.
This amount is the maximum the business would be willing to spend to staff the position.
E1. Employee knows their market value and will ask for it.
If an employee fails to get to this number it is because:
E2. The employee does not know their own value
E3. The employee knows their own value, but for whatever reason doesn't make a push to receive appropriate pay (this can be fear of losing the job offer, non-confrontational personality, ... )

We can now look at each possible combination. Suppose the position in question's salary is fairly market valued at $100k.

B1.E1. Employee is offered 100k. Hooray.
B1.E2. Employee is offered 100k. Hooray again.
B1.E3. Employee is offered 100k. The system works!
B2.E1. Female employee is offered 75k. She asks for 100k. Company doesn't think she's worth it, deal is off.
B2.E2. Female employee is offered 75k. She takes it.
B2.E3. Female employee is offered 75k. She dislikes it, but takes it.
B3.E1. Female employee is offered 75k. She counters at 100k. Company accepts. Hooray-ish.
B3.E2. Female employee is offered 75k. She accepts.
B3.E3. Female employee is offered 75k. She dislikes it, but takes it.

I've simplified a lot of the details, but I think this matrix is representative and instructive (there are various sub-divisions we can make, but I think the principles emerge. For example, whether or not every man would get 100k from B2 or B3 isn't really relevant so long as they get more on average). Of the 9 outcomes, 3 are ideal and driven by socially responsible companies. 1 other achieves the same pay, but is driven by the employee. 4 others result in lower pay. 1 results in no job.

I think there's not much more to say about the 3 ideal cases. B3.E1 is interesting though. While the female employee gets max pay, she's now also burdened with the stress of maintaining her pay growth (presumably B3 will not grow her salary at market rate without her negotiating for it). This at a minimum makes her work environment less pleasant and/or feel discriminatory, at worst it causes her to have diminished capacity to perform her best, causing a real drop in value to the company. The remaining B3 cases would turn out at least somewhat better if the employee knew to negotiate for fair market value. Nothing the employee can do will get her market value at B2.

So how can any of this change?
Ideally, companies would all do the socially responsible thing and fix their practices. However, there will still be a very real desire for them to pay employees as little as possible, and I don't think that will ever go away. With transparency of pay, the B2 group should become non-competitive. However, pay is unlikely to become transparent ... but prospective employees can address this by applying for more jobs or to companies with good reputations. The real headway needs to be made in the B3 group. If employees can gain an understanding of their market value, they can push the B3 group closer to the B1 group. While there's still the stress of maintaining market value over time, this is still a much better position to be in.

Some industries address this with unions. Unions have not been a part of the tech landscape, and I don't think they would stick for reasons I won't enumerate right now.

Athletes address this with agents. Would we be willing to have someone else negotiate our contracts for us for a fee? Agents would have more visibility into actual salaries because they would represent lots of people. Thus, they would know the actual market. On the flipside, we'd probably need to move to fixed-length contracts (like in sports) so that the agents have motivation to keep negotiating for us. Is the prospect of higher pay worth going to this model for? The risk of not getting re-employed after a contract is over? The possibility of a pay decrease if the market softens? I don't know, but it's an interesting idea.