Why I’m excited about Dark

The Dark language and coding platform launched this week, and I couldn’t be more excited. (I had half a mind to fly to St. Louis and see them launch in person at StrangeLoop.) For a while, I’ve been following what Paul Biggar (founder of CircleCI) and Ellen Chisa (a product leader whom I respect) have been building in Dark: a product to democratize software development and make it easier for everyone, particularly for folks who haven’t previously coded before. The frustration Paul expresses in his launch video –- about infrastructure and today’s delivery process getting in the way of delivering code – resonated deeply, particularly when I think about how we used to “deploy” in the early days of web software engineering (FTP-ing PHP files to production). Obviously, that was risky.

Today, however, in the name of risk reduction, we have complicated CI/CD pipelines, Kubernetes and rolling deploys, thousands of lines of AWS CloudFormation and Chaos Monkey to kill it all, and a million other extraneous guardrails that developers today need to learn just to get something into production. An end-to-end, low-friction developer experience such as what Dark is providing is certainly appealing, not just for new developers, but for old folks like me who would like to program again occasionally but find today’s frontend and backend stacks daunting.

In this post, I’d like to give my thoughts in just three areas: what did Ellen and Paul and their team build? In what way could they reasonably go to market? And also – to address some of the early criticism about Dark, e.g. that it’s a closed ecosystem and not open source – does it matter? Are there are other valid criticisms and how could they overcome them?

What did they build?

Dark (a totally un-Googleable name, by the way, not that I know anything about that) is a complete programming environment for backends that comprises a new language, an in-browser integrated development environment (IDE) with which to write that language, code hosting (including a built-in database), and deployment mechanism to get from the IDE to hosting. The deployment semantics are the most interesting because deployments are implicit, not explicit: every syntactically correct line of code you write is automatically and immediately deployed in under 50ms. I’ll leave out all the details about how this works and why it’s safe – if you’re interested in the implementation, go watch Paul’s video.

This is exciting because it restores programming and deployment to the experience that we all used to have before the Internet exploded in popularity. Like I said, I used to “deploy” by FTP-ing interpreted language files to production, and it was a great experience, because you could get immediate feedback as to whether something was working. This is terribly unsafe, because if you make mistakes in your code, they show up immediately in production — not to mention that this workflow makes it impossible to collaborate with other people. But for the lone developer, the experience is glorious.

Dark focuses on providing a similar experience but puts all the safety and collaboration guardrails under the hood. Like Rust, the Dark language makes syntax errors nearly impossible (although obviously, logical errors are still possible). And it enables forwards and backwards code compatibility by versioning everything automatically, including method signatures and invocation semantics, such that callers invoking older versions of a method will continue to do so until they can be upgraded to work with the new one. That’s the promise, at least. Obviously, one of the challenges they have is “will this work over time as teams of people actually use it?” But that is the promised developer experience, and it, like the bad old days of PHP and FTP, is amazing.

How could they go to market?

That’s what Dark has built. The next question is “who could it be ostensibly for, and who would be willing to pay for it?” Dark is not going to be for everyone; many developers want to have more control over deployment processes and are accustomed to writing code in mainstream languages. Frankly, there is already a whole industry dedicated to creating and maintaining deployment complexity (Paul, as the co-founder of CircleCI, is as culpable as anyone ;-)), and many an engineer has built a career on this. (I’m also looking at you, Jenkins.)

Dark is also currently only for backends. The React frontend Ellen uses in her demo had to come from somewhere. I see other people like Glitch trying to solve the frontend complexity problem — just try to understand what it takes to be a frontend developer in 2019 — so perhaps there’s a natural partnership between the two firms.

Where I really see Dark shining is serving cohorts of people who don’t currently code today, don’t work in an engineering function, but have an increasing need to write code occasionally as part of their jobs – say, if you’re a financial analyst or even a product manager/product marketing manager. (Hello.) This is the target market today of categories like Robotic Process Automation (RPA) and low-code/no-code platforms like Pega or Salesforce’s force.com. Many of these products, however, are still unfriendly to the population I described; too complicated, too many steps, or in some cases, are 20-30 year old legacy products rebadged as “low-code” purely for marketing reasons. I think these are incumbents ripe for disruption.

What are some of Dark’s challenges? Which ones matter?

Obviously, there are a lot of folks complaining about Dark not being open-source, that it’s a closed ecosystem with high vendor lock-in, and so on. These facts are 100% correct and yet none of it matters because I don’t see Dark as being for the existing cohort of developers today. It’s for the larger unaddressed market of developers that don’t exist yet – the person who has yet to write their first line of code. If Dark can make the development experience so easy that they capture even 10% of enterprise users who are blocked by not having traditional development resources and are willing to self-serve, it adds up to quite a bit of revenue.

One gap in Dark’s public business model so far is how they will price and to a lesser degree, package their offering. I expect they will price based on seats and with an all-inclusive price that includes code hosting. After all, if their target user is someone who doesn’t identify with the label “developer”, simple packaging and pricing will be key. If Dark is smart, they’re using some kind of on-demand code execution engine (like AWS Lambda) that allows them to realize profit based on the difference between a premium monthly per-seat price and the low costs of relatively infrequent code execution by “business” users. (They might eventually need to consider a resource execution limit to deliberately steer heavy, traditional developers away from their solution.)

What’s next?

As I said in the introduction, I was excited enough about the launch of Dark that I wrote this whole blog post about it. So far, the team has executed brilliantly, focusing on development ergonomics for folks who I expect aren’t traditional developers. The amount of engineering work needed to pull off what they’ve pulled off is incredible. Now, I think they need to define their target market and pursue that with zeal. That’s probably going to involve heavy investments in increasing safety (e.g. unit testing, collaboration, interfaces to unpack the implicit versioning magic under the hood, etc.) as well as developing a community from scratch in a population that doesn’t code today.

There’s also the question of how some of the sector incumbents I mentioned previously will react. How much will existing low-code/no-code vendors try to market aggressively against it? The key differentiation thus far is in the developer experience, and if Dark continues to nail that and out-innovate others (they have a 12-16 month advantage right now), then the future is bright for them.

On leaving Chef

Yesterday was my last day at Chef Software. It is the second-longest I have been at any company — nearly six years. I have been very fortunate to have held four very distinct positions at Chef over my time there: professional services consultant, engineering manager, product manager, and finally, product marketing manager. When I contrast my experience working for a startup with the time I spent in the enterprise, I would, without question, state that this has been a better experience for me both personally and professionally. I could not go back to work for a huge company where the sense of urgency and determination to succeed are just not there, and where the needless bureaucracy snuffs out the flames of innovation before they have a chance to catch. I’m sure there are exceptions out there, but when I think about the poor customer experiences we all have interacting with our bank, college, grocery store, or health care providers, those are fundamentally a direct result of company-wide complacency driven by sheer size and corresponding market dominance.

It is also the first time for me leaving a company because I outgrew it — or maybe it outgrew me. In tech, we love to dump on the 99.99% of companies that aren’t Hacker News Hot Shit ™ while neglecting that many of them are doing quite well financially, thank you very much, and solving real customer problems. While the HN crowd is busy chasing shiny objects, the rest of us are off building businesses, and extremely successful ones at that. For me, though, there wasn’t much more I could do or wanted to do at Chef. The company is at a phase of its growth that is about immensely scaling the products it already has and deriving revenue from them, rather than launching entirely new product lines. As a product guy and engineer at heart, it’s therefore time for me to move along.

I wish my colleagues at Chef a great deal of success in the next phase of the company’s growth. It has been a privilege and an honor to have worked alongside so many of you and to have learned so much.

Some Thoughts on the 2018 XOXO Conference and Festival

This past weekend I had the opportunity to attend the XOXO Festival right here in Portland. For those who don’t know about XOXO (as I didn’t before I moved here last year) it’s a festival and conference for and by makers of things on the Internet: game designers/developers, programmers, filmmakers, comedians, artists, podcasters, and the like. A single-track daytime conference accompanies the evening festivities, packed with thought-provoking keynotes delivered by very accomplished people such as Jonny Sun, Ijeoma Oluo and Demi Adejuyigbe. It’s primarily this part of the event that I want to say a few words about. Continue reading

On Open-Source Business Models

Over the last couple of weeks, many open-source luminaries have written about open-source business models in response to Redis Labs’ modification of their license terms for some Redis Modules. Most of these blog posts dissect one thing: how does one create a business around open source? I actually think that this question, framed as it is, is kind of absurd. It sounds to me a lot like “how do we make money off the horses which have already fled the barn through the doors we left open?” Starting a company off on the wrong foot by giving away all of your valuable IP – and then figuring out how to build commercial products on top of whatever scraps of value remain – is silly. Many such conversations – led by technologists — start first with open-source zealotry and reverse into a business model which is the wrong way to do things. Instead, if you’re considering building a company based around an open-source foundation, first design the business model, and then contemplate how open sourcing certain pieces can serve as an accelerant to that business. If you can’t find an accelerant, maybe your business shouldn’t be based on your IP being open source. Continue reading

What’s In Amazon Linux & Why Might You Use It?

Just before Christmas, Amazon announced the release of Amazon Linux 2. In addition to the usual userland modernization (switching to systemd, providing a newer glibc ABI) and the Linux 4.9 kernel, you can now run it on premise; images are provided for VMware, Hyper-V and KVM. Sometime in 2018, Amazon will also start providing long-term support for Amazon Linux, similar to RedHat Enterprise Linux (RHEL), by promising to support ABI compatibility for at least five years.

I’ve always been somewhat puzzled by Amazon offering their own Linux distribution to customers. No other cloud provider does this, although Google runs their own Debian derivative on non-customer-accessible systems. (For more on that, the slides from a talk they gave in 2013 makes for fascinating reading, in part because live-upgrading a 10-year-old base OS to a newer one while changing from .rpm to .deb is… terrifying yet awesome.) I decided to take a pretty deep dive into Amazon Linux 2’s feature set to see if there’s a compelling reason I could see for customers to adopt it. Continue reading

From Product Management to Product Marketing

I start 2018 in a new role at Chef, as director of product marketing. Five years is a long time to work at one software company, never mind a startup. I find I have to constantly reinvent myself in order to not get bored or complacent.

Prior to coming into product marketing, I’ve worked at Chef in professional services, engineering and product management. Now is the time for me to not only help with sales enablement, analyst and press relations, customer-facing materials, and so on, but to also learn how to build a high-quality sales pipeline. As someone who doesn’t hold an MBA but someday might want to start a company, these are critical skills for me to develop. Continue reading

An Elegy for Etsy

One of the biggest news events in 2017 for those of us working in devops was hearing about abrupt changes at Etsy. In April, institutional shareholders, disappointed by Etsy’s low share price, ousted CEO Chad Dickerson, and under new CEO Josh Silverman, proceeded to slash the workforce and reduce costs. Many talented engineers that I happen to know personally as well as CTO John Allspaw himself were either fired or saw the writing on the wall and made for the exits.

These events were discouraging on a number of levels. First, Etsy’s world-class engineering and web operations team scattered to the winds. Allspaw and many others contributed a great deal to the discipline of web operations by applying lessons learned from improving reliability in other fields, such as transportation engineering and nuclear safety. More importantly, though, is that Etsy’s very existence shone a light on whether it’s possible to build a socially-responsible company within the constructs of the current venture capital, and by extension, capitalist system. What is the place of the social democratic entrepreneur within these confines? Continue reading

Blogging Again, Carefully

I was recently re-reading Steve Yegge’s You Should Write Blogs post from 2005, and started thinking about why I don’t blog anymore. I used to be a pretty prolific blogger from before the days when these things were known as “blogs”. The archives here, in fact, go back to 2003, and although there is probably some pretty cringe-worthy content that far back, I’ve never deleted a single post.

So why did I stop? It comes down to a simple reason. As I’ve moved in my career from engineering roles into leadership roles, the topics I would consider blogging about have moved from systems and software to ones of organizational dynamics, individual behavior, and management’s role in facilitating the same. To properly write about these topics would require me to write about the actions of real people with whom I work. Although I would never name them, those folks would surely see themselves in my posts, and I’m afraid of being perceived as being mean when discussing everyone’s foibles. As my friend and colleague Adam Leff said, “No matter how much we collectively say we value honest opinion and transparency, we’re all still humans with brains capable of remembering what others say of us.” This makes it very difficult to talk about challenging topics openly.

Writing is especially tough in these partisan times. Fueled by hair-trigger social media, we are all too ready to jump to conclusions about the motivations of writers and ascribe specific intent to the choice of a particular word or placement thereof. We accuse people of “sub-tweeting” when they express their observations on Twitter, or label them as “toxic” on the basis of a small number of interactions. The halcyon days of 2005 Steve Yegge, where a blogger could just publish lightly-edited thoughts without suffering severe repercussions, seem long past.

I have, however, decided to return to blogging regularly. I will sometimes be writing about difficult issues, with the objective of sharing my experiences so that we can all learn and improve together. Yet I intend to treat everyone fairly and to be kind rather than “nice”. If you see yourself in these pages please try to bear this in mind. And since I have always spent almost as much time editing my posts as writing them in the first place, you can be confident that I’ve weighed all perspectives and I’m not just shooting my mouth off after having some drinks.

With that, onward. Next up this month will be a piece I’m working on in reaction to the New York Times’s recent article on what went wrong at Etsy. I hope you’ll read it.

Some thoughts on DACA

Both my wife and I are (prospective) immigrants to the USA, so I have a few thoughts on the matter of President Trump’s ending of DACA, the Deferred Action for Childhood Arrivals program. At the risk of the implications of wading into a debate that, like most modern political debates in this country, seems to lack nuance, I would like to share them with you.

We are economic migrants, here because we find better opportunities than in our country of origin, Canada, so we have this in common with Dreamers. At the same time, we are both obviously immigrants of a privileged class, and are not fleeing oppression, war, famine, or any of the other legitimate reasons that might spur individuals and their families to sneak across the border into America. Nothing I’m about to say implies that I lack compassion for these folks or their circumstances, either before or after they arrive in this country.

There is no easy answer to solving the problem of illegal immigration to America, short of Congress actually passing legislation to rectify the situation. (I’m not a big fan of the euphemism “undocumented migration”, by the way, but evidently even the terminology we use is politically fraught.) If you believe in more open borders and compassionate immigration policy, you might be a DACA supporter simply because of the downstream effects, which, no doubt, are good for both Dreamers and the American economy.

But the way in which DACA was enacted is bad for the republic. Allowing it to stand would signal that it’s acceptable for the executive branch to enact policies instructing law enforcement to ignore any statute which is on the books. This creates a terrible precedent. Suppose President Trump would like to exempt certain elements of his white supremacist base from being prosecuted for violent crimes. Should he be allowed to create a “DAWS” program — Deferred Action for White Supremacists — that suspends prosecution of certain felonies in defiance of the laws that are on the books?

While I completely agree that the motivations of Trump in ending DACA are odious, and pander to an intolerant electoral base, I believe that DACA as an executive order cannot stand. So it had to go, sooner rather than later.

Smart cities need smart government

Several weeks ago I had the opportunity to attend the Smart Cities NYC conference. Urban planning, sustainable design, cities, transportation systems: these obviously have nothing to do with Chef, but have been interest areas of mine for a very long time. I came away with some insights, particularly about the relative roles of government and private sector, and one thing stuck with me. If we don’t reform the civil service for the wired cities of the future, we risk turning over the fabric of our society to private corporations — to the extent that isn’t already happening. Continue reading