The Agony and Ecstacy of the U.S. Government's Relationship with Apple's iPhone, Part 4

2013 was an amazing confluence of events that can best be described by a quote from Sebastian Junger's book A Perfect Storm:

"Meteorologists see perfection in strange things, and the meshing of three completely independent weather systems to form a hundred-year event is one of them. My God, thought Case, this is the perfect storm."

All thoughts, opinions, and other words here are exclusively the personal opinions of the author and do not express any official position, statement, or other communication from or on the behalf of the U.S. government. This is a work of fiction. Any resemblance to actual persons, living or dead or actual events is purely coincidental.

In their book, Dangerous Enthusiasms: E-government, computer failure and information system development, "Bring Your Own Device" (BYOD) is what researchers Robin Gauld and Shaun Goldfinch define as "managerial faddism."

They identify it as one of the primary causes for government IT project failure. Before coming across this book, I thought of the same phenomenon as "fashion trends."

Sadly, common sense, basic fundamentals, and reliable, proven technology is not something that sells to the government execs who control what gets to be spent on what, after Congress.

Companies, and the entire zeitgeist of the circuit, government press, and technology demands changing fashions, fads, something new and exciting to sell.

I would observe a new fad/fashion come and go, in and out of vogue, about every 3 years.

Each one was "the thing" that would finally "fix government IT."

Some highlights that stand out over the years:

• Bring Your Own Device (BYOD) (2010-2012)
• CLOUDTOPIA (2013-2016, also still not over somehow?)
• Blockchain (2017-2020, sort of my fault and that's a whole other story)
• NFTs (about 6 months)
• A.I. (2022, after the big crypto crash when most reasonable people realized this was just a money laundering scheme, to now)

They never did "fix it" because the reason "government IT is broken" is because of human conflict. Not technology.

At the time, the few Apple devices that did pop up in the enterprise were via BYOD. And it was a huge pain in the ass.

At first, the idea of outsourcing outsourcing to employees was quite promising to enterprises like Census. Many smart people thought it could even save money! It did not.

Many people thought we could conduct Decennial that way, but anyone who knew better knew there was no way in hell we could.

The operational, technical, and legal hurdles of that made people realize it wasn't really viable given government laws, systems, and networks.

One of the CIO's senior advisors at the time was the hatchet man for the rank and file. Hatchet man would frequently evoke Steve Job's notoriously abusive style of interacting with all of his co-workers which made the CIO giggle in glee. I can still hear their nasal, demeaning voice in my head now when I think of it.

As an Apple fanboy, he always made a point of very dramatically pulling out his personal MacBook at the start of every bi-weekly Decennial planning meeting. I swear he thought he could heard an angelic choir as he did, like it was pulling excalibur from the stone.

The once and future king…to then log into...LotusNotes, our email system at the time.

Grunts like me didn't get issued phones or laptops. We got desktop computers, Voice-over-Internet-Protocol corded phones (VoIP), and typewriters.

Yes, typewriters. But those suckers were heavy, so they rarely moved. Usually, I would attend meetings with a pencil and notebook, largely as a prop. I typically keep notes in my mind palace for government records purposes because I am cursed with a good biological backend database...that haunts me.

Good luck getting anything out of my head G-Man! It's chaos up there! Unfiltered Lovecraftian horror of cosmic dimensions.

You have to remember that at the time, electric typewriters were only 30 or so years old. For government technology, that's a spring chicken.

Technically, I can–and did–entire contracts via paper. Because our electronic contract writing system (CWS) was so bad it usually took longer to use computer with all of the text and forms than it did to do it on paper. If I did that, then I could just image scan and upload it into the system (government loves image scans).

Our CWS was developed by a military contractor that also made bad software in addition to shoddy weapons systems. This contractor, I would often point out, was one of the primary contractors entangled in the Abu Ghraib prison torture scandal in Iraq.

Based on the user experience of that CWS, only someone who also tortured people in contravention of international law could have come up with it.

Then the unthinkable happened. The first major meshing of events into a perfect storm.

At a CIO Council meeting, one of my CIO's peers at NOAA made a point of setting their iPhone down on the conference table...

My CIO, Patrick Bateman (not their real name): Nice iPhone...

NOAA CIO: Thanks it's the latest model.

Patrick Bateman: BYOD?

NOAA CIO: (silently flips it over to show the government asset tag on the back). No it's a NOAA phone.

My CAO got an angry phone call almost immediately after the meeting.

I know because he came over to my cubicle with the expression on his face that meant someone had demanded something of him and he had caved. I came to know that expression well. It meant that I had to perform a magic trick and pull a rabbit out of my ass.

My CAO: Patrick Bateman called (not the CIO's name).

Me: What do they want now?

My CAO: The NOAA CIO had an iPhone at the council meeting. He said we have to get them now.

Me: How?

My CAO: I don't know but I can't stall him anymore figure it out. I don't like being alone with him he keeps asking me about CD's...

Fortunately, I knew a lot of folks at NOAA especially at the Office of the CIO (OCIO) from my Gold Medal project. So I called them.

Me: Hey Fanny! How are you! I need your help!!! (not their name)

Fanny: (always enthusiastic) Hey there what can I do for you?

Me: How did you buy the iPhones?

Fanny: What do you mean?

Me: Explains BAA and TAA...again.

Fanny: Oh we didn't even think of that, we just traded our old ones in but we haven't gotten that many of them across the enterprise because it's a huge lift to get them integrated into our networks.

Me: You traded them in?

Fanny: Yea call Alexander Graham Bell at Big Red Phone. (not the sales rep or company name).

A practice that you won't find anywhere in the FAR is the idea of contract "credits." This is when someone on the industry side screws something up that they weren't supposed to screw up and it shows up in an invoice.

Except neither the government nor the company want to deal with the hassle of formal steps to deal with it (also a possible scandal). So instead, the government will get a "credit" that then gets applied to some future spending "like a discount" that evens out the earlier screw up.

Technically, the BlackBerries were government property and there's a whole bunch of rules around the proper disposition of government property. But, I had to pick my battles. If Big Red Phone was letting people trade in their government property to lower the unit price of an iPhone, I had to find out.

So I called the sales rep and went through the high level parameters of the "trade in discount" and how much we'd get in "credits" to trade in our BlackBerries for iPhones and reduce the total cost per unit.

Plebes like me didn't get BlackBerries. Only those whose work was deemed as absolutely necessary for having one, such as field people and managers, got BlackBerries.

It wasn't as easy as just transferring the phone numbers to the iPhones and then they'd work. The IT department needed time.

Now here's the "little p" politics issue for Patrick Bateman. Besides his total lack of experience with technology, business, or any type of government operation at the size, scope, and complexity of Decennial in his career, Patrick also had an issue with his bannermen.

Census was always highly decentralized. In fact, during my time there was a massive reorganization to destroy the various regional field offices because they had become kingdoms unto themselves.

And there can be only one king...

Patrick Bateman's problem was that, historically, his workforce in the IT Directorate historically had absolutely jack shit to do with Decennial.

Decennial was always done by the dedicated Decennial directorate and the Field directorate. Each one had their own tech people, many of whom had been through more than 1 Decennial already. And their contractors.

Neither of them relied on the IT Directorate for the actual mission.

Historically, the Census IT Directorate used to be responsible for "basic back office" IT like desktops and VoIP phones.

Now, they were suddenly tasked with accumulating the knowledge, skills, and experience that their peers had accumulated over decades of operations. And they were supposed to do that immediately because the CIO had declared himself the King of all IT.

I'll give you an example of the IT Directorate workforce at their finest. We had implemented a virtual desktop interface (VDI) because it was supposedly better than allowing us to have the freedom of direct computer on our computer.

Plus we had spent all of this money on a data center in the middle of nowhere that was never at full capacity (this is another story). But the VDI was terrible. It would often get stalled and had to be rebooted, which you could only do at the data center end.

I also got into trouble once for posting a short clip on the Census Yammer of this happening. We weren't supposed to be that kind of social on the forced fun corporate social media tool we just had to buy for some reason. After they told me to delete the clip of the VDI not working, I posted their order and this clip:

After this, I got an "exemption from using Yammer anymore" (after having been required to make a profile on it to justify the purchase). Remember, loopholes and exceptions.

Anyway, my VDI broke again so I called the IT help desk. I told them "my VDI got hung up" and that it needed to get rebooted at the data center. The IT help desk said they were on it. Locked in. They've got this.

They sent someone down to my cubicle, which confused me. That person looked at my computer and decided to unplug my mouse and take it with them as they left.

I called the IT help desk again and told them someone just came and took my mouse. A second person came with a new mouse...but that person took my keyboard.

At this point, I thought I was being hazed. I called the IT Help Desk again and said I kind of need a keyboard and still need my VDI instance rebooted.

They sent down a third person with a keyboard.

And, I swear to God this happened and my direct supervisor saw it too and started laughing from his office, this new fella started blowing on the back of my desktop tower like it was an old Nintendo cartridge.

Now you might be saying to yourself as you read this, "Ok well those are just the operations people, I bet there's some really good IT professionals who manage services, oversee contractors, and generally mind the store."

You would be right, and the Top Gun of those people was this guy.

This is Frank. Frank was one of our super Contracting Officer's Representatives (CORs). CORs are usually the "technical assistants" to the 1102's like me. We would give them limited powers, in a designation letter, to manage technical elements of a contract. We did this, in theory, because they were supposed to be more qualified than we were–as people intended to just push paper and buttons–to manage highly complex technologies and shit.

Frank was the COR for 1500 contracts, literally. This means he didn't do that for any of them, and we kept giving even more to him.

Unfortunately for all of us, he tried to murder his lover's husband several times until finally getting caught in a very hairbrained scheme that made international news. Frank was foiled when the fake beard of his murder disguise fell off, his intended victim recognized him, and then made a hasty escape because Frank was distracted by losing the beard as someone else walked into all of this.

Suffice to say, the CIO hadn't been pleased with his workforce. And he'd be damned if they stopped him now. So he gave them a deadline.

I was caught off guard by all of this because of the second meshing event that created a perfect storm. It came from Google.

Ignoring the question of whether or not the whole trade in scheme was legal, there was something completely different that could make this illegal.

I started interacting with Google right around 2011 when I started, because of this thing computer does called internet, a well known series of tubes.

We had several contracts with Google for services like Analytics, and inevitably they got curious as a new entrant to government contracting about this whole Census thing. And Google was very good at something I kept hearing about called "the cloud." That is a whole different story. Multiple ones in fact.

In the early 2010's, a lot of government contractors did not like companies like Google and Amazon. These new fangled, cloud-born FANGs coming onto their turf. The government's paycheck was a divine right for the legacy stalwarts like Microsoft, Unisys, seriously, Fujitsu, Xerox, somehow, IBM, etc.

The legacy folks took umbrage and, for example, this led to a very publicized incident at the Department of Interior, which led to a lawsuit and an apology from Microsoft to Google.

Or when IBM rigged their bid pricing to force a lawsuit, but the court noticed during their battle with AWS over the $600 million "secret" CIA cloud C2S (a dedicated physical AWS data center that lacked parity with commercial offerings).

But in Onix Networking v. Department of Interior, the Court of Federal Claims (COFC), which is an actual court a level of power above the GAO administrative court, stated:

As discussed herein, the July 15, 2010 Standardization “Determination and Findings,” on which the August 30, 2010 Limited Source Justification was based, violate the Competition in Contracting Act and FAR.

TL;DR the Department of Interior CIO, who had declared himself IT Pope over at Interior, issued a papal order. This edict stated that Interior would standardize on Microsoft for reasons without considering or accepting anyone else's potential bids. Thus sayeth the lord, his will be done.

Except, Google minded. And the COFC ruled that the CIO, in fact, was not IT Pope. Doing this was illegal. Because of Congress, again.

The Competititon in Contracting Act of 1984 (CICA) is like the source code of the FAR. It mandates that any government contract spending taxpayer money should be competed "to the maximum extent practicable."

So despite Patrick Bateman's best efforts at playing king, in a way that I would argue is un-American at the core of our soul and certainly not what Congress intended, I had case law on my side. Case law proving that being the CIO, by itself, didn't let him just pick a company and give them a contract.

The CIO telling us to trade in the BlackBerries for iPhones possibly violated CICA, but we let that one go I guess. My CAO also wanted to pick his battles and wanted to protect Decennial, he thought it was less risky to let this one go.

This CICA issue still comes up today with cloud service provider's (CSPs) "stores" where customer agencies will spend tons of money on brand name software products to add into their virtual computer builds without proper CICA competition.

But the real surprise was when, out of nowhere, Google acquired Motorola.

Motorola was the birthmother to the first "must have" cool, cell phone, the Moto Razr (sorry zoomers and boomers you'll never understand).

Even more surprising, Google was releasing a new flagship Android device called the Moto X.

Even more surprising than that, the Moto X would be made in Texas.

It doesn't get more American than Texas. Unless you ask someone in that state, and then they think nothing is more Texan than America? It's weird.

Governor Rick Perry of Texas, who once declared that he wanted to eliminate my Department if he was to become the President, was over the moon at the return of America-In-Its-Prime manufacturing to the lone star state.

Technically, they weren't manufacturing the phones in Texas. That was still being done in China.

The components were being "assembled" in America, which passed the substantial transformation test that made usually non-TAA compliant components totally legal.

But this was it. I had been struggling for over 2 years at this point.

The Texan Moto X plant would operate 24 hours a day and deliver 100,000 units a week. We could field Decennial's needs in a month and a half, two months top. Congress' laws were sort of working.

America was back baby!

"This is a bet we're taking on America ... on Texas [and] on this incredible workforce that's assembled here. We think this is a very, very safe bet." Gov. Rick Perry at the opening of the Texas Plant.

Also at this event, Gov. Perry, surprising no one who has observed him over the years, decided to spike his personal iPhone on the factory floor like it was a football. Apparently, someone caught it on camera but the online link to the footage had died.

But he could spike all the iPhones he wanted for all I cared.

I was relieved beyond belief. I remember thinking to myself, "Ok well we're maybe not breaking any laws with this trade-in...maybe. And unless we have a really big fucking bake sale or start selling our office furniture to raise funds*, I don't have anything to "trade in" for 500,000+ iPhones and iPads."

*Agencies have actually tried this, it’s called “supplementing appropriations.” Congress doesn’t like that so it’s also hella illegal per the Anti-Deficiency Act.

But the dumbest civil war was over. We'd have to buy the Moto X, not the iPhone made entirely in China!

There's no way that after returning quasi-manufacturing to America for smart phones that the U.S. government wouldn't throw the full might of its purchasing power behind this.

But, like many times in my life, I was wrong.

The third meshing event arrived and the once-in-a-century disaster for the was here.

Stay tuned for Part 5.