Jekyll2026-04-04T17:53:34+00:00https://kevinsmith.io/feed.xmlKevin SmithEngineering leader on a mission to help product teams love their job and consistently deliver real customer value.Kevin SmithOur Goal Was Never To Write Code2026-04-04T17:48:25+00:002026-04-04T17:48:25+00:00https://kevinsmith.io/our-goal-was-never-to-write-codeA memo I sent to the AxisCare product development team earlier this week:
It’s now abundantly clear that we’re living through a step change in software development thanks to AI. We’ve been embracing and experimenting with the latest tech for the past several years to varying degrees of success, and you know I’ve been deliberate in my approach. Something massive changed with these models in the last few months, and it warrants a frank discussion.
As the staff engineers showed us a few weeks ago, it’s now finally possible to drive the AI agent and see your intent reliably carried out — even when fine-tuning a UI — where even a few months back the agents would clobber your work.
It’s not perfect, but it doesn’t need to be. Remember that high-quality software is both valuable to the business now and easy to change later. With the latest models, we can confidently change the software incredibly quickly. And that makes all the difference in the world.
For as long as I can remember, writing code was by far the most critical constraint in making software. It was slow, expensive, and error-prone, and every role in product development worked to minimize that time and expense. Exhaustive requirements gathering, detailed plans covering every edge case, high-fidelity mockups, story points, backlog refinement, the list goes on. The goal was to resolve as much ambiguity as possible upstream, so that when someone finally sat down to code, they were transcribing a decision that had already been made. Everything was structured around the scarcity of coding time.
But our goal was never to write code. That was always a means to an end.
So what happens when writing code takes an order of magnitude less time? Which parts of our process no longer make sense? Which pipe dreams are now cheap to try? How quickly could we get to working software, even an unshippable prototype, and iterate from there? What do we need to have in place to make all that possible?
We’re not building the entire car by hand anymore, as much as we loved doing that. We’re building the factory that makes great cars, with the human in the loop where it requires the human touch. It’s time to reimagine your role in building that factory. (And I’m not just talking to the devs here.)
This isn’t a prediction about what’s coming soon. This is here. AI is already fundamentally changing how we make software, even if the models never improve from where they are today.
If all this is unsettling — if you’ve long taken pride in meticulous PRDs, beautifully designed UI mockups, hand-crafted code — I get it. As Kent Beck said in a podcast published a few days ago, we didn’t choose this. We developed skills that were in high demand. Now changes from the outside are disrupting all that, and it’s understandable to feel some grief over it. But even though we had no say in the changing environment, we can choose how we respond.
We can’t see around corners, so we don’t know what the future will hold. The best way to respond to this moment is by stepping up to create that future rather than waiting for it to arrive.
Let’s use the full toolkit. Let’s push the AI far enough to see where it breaks down, and then try that same thing again in 3 months. Let’s throw out the old playbook and reimagine what it takes to ship things that matter, start to finish. Let’s build the factory.
In software product development, we’ve historically specialized in ways that may not make sense going forward. But there will always be a need for people who deeply understand our customers’ most important problems, solve those problems with high-quality software, and run that software reliably in production. This is the work that endures.
I’m challenging each and every one of you to step into this moment, regularly question your assumptions about how software is made, and share your wins.
This is a rare opportunity. Let’s embrace it and create the future together.
]]>Kevin SmithThinking Outside the Black Box2026-02-11T02:17:39+00:002026-02-11T02:17:39+00:00https://kevinsmith.io/thinking-outside-the-black-boxA worthwhile thought experiment:
If AI can one day quickly and reliably write new code and edit existing code to produce the intended new behavior of the software system, such that the system itself effectively becomes a black box… there’s a whole slew of things that don’t matter anymore.
Human readability, for instance. And if we’re no longer optimizing code for the human who needs to read, understand, and change it later, what else goes with it? Code review. Design patterns. Modularity. Frameworks. Fights over syntax, or even languages. Who cares what the inside of that black box looks like?
There’s a lot we always assumed was essential complexity of software engineering that was really just the goings on inside that box. And sure, it was the really expensive part of building and running a software product, but that doesn’t mean it was the most valuable. The most valuable parts have always been what happens outside the black box. Time to double down on those skills.
Regardless what the future holds, this will always be true: high-quality software is both valuable to the business now and easy to change later.
]]>Kevin SmithThere’s No Reason to Worry About AI2023-10-19T03:11:39+00:002023-10-19T03:11:39+00:00https://kevinsmith.io/theres-no-reason-to-worry-about-ai…if you bring value to a business by thinking critically, uncovering key customer struggles, and creating things to give it major leverage in the marketplace. There will always be a need for innovators, regardless the tools they use to do their work.
If, on the other hand, you’re merely putting puzzle pieces together at the direction of someone else who’s doing the thinking, you’ve got a lot to worry about. AI’s getting pretty good at clicking those pieces together.
]]>Kevin SmithHow to Produce a Rich Domain Model with Active Record2023-10-03T12:25:39+00:002023-10-03T12:25:39+00:00https://kevinsmith.io/how-to-produce-a-rich-domain-model-with-active-recordYou can’t. It’s not possible.
I know that sounds like an outrageous claim given the popularity of the Active Record pattern, but it’s true.
Active Record provides complete access to a database row’s fields through an object’s public properties (or “attributes” of an “Active Record model”, as they’re called in both Ruby on Rails’ and Laravel’s ORMs). Any part of the codebase can access those attributes to read or change any of the data in the database row, making it easy to start working with a real database in your code.
The trouble starts when the complexity of the business inevitably reveals itself and you need to restrict the inherent permissiveness of Active Record to ensure that information is only retrieved or modified according to the rules of the business. In other words, you need a rich domain model: a layer of the codebase where the expressiveness of the business is encoded into a community of collaborating objects, each properly bounded and equipped with the information and behavior they need to model the relevant interactions and responsibilities of the business.
Need to make a property private so that only the object has access to it, a fundamental principle of software design called information hiding? It’s not possible.
Need to require multiple pieces of information for an update and validate them before accepting the changes, like making sure you have both longitude and latitude before updating geo-coordinates? Sorry, you can’t enforce that.
The best you can do is apply business rules in domain services that wrap the Active Record model, but you can’t actually enforce the rules anywhere.1This is the critical, inescapable problem with using Active Record in the domain. Nothing is stopping the rest of the codebase (or worse, third-party packages) from bypassing your constraints to manipulate the data directly. And make no mistake, that will happen. The inevitable outcome is data corruption, inscrutable bugs, and feature development slowing to a crawl as you try to regain stability in your system.
By its very nature, Active Record’s impact on a domain cannot be contained and thus will largely determine the resulting system architecture, despite whatever plans its developers may have at the outset. Given enough time under real-world dynamics, it will lead to a big ball of mud that becomes increasingly expensive to maintain.
Now you could try to contain Active Record to the edge of your application, only using it as a conduit between your database and domain objects that properly enforce the necessary constraints. This would keep it out of your domain and go a long way toward avoiding architectural distortion.
But in practice, this tends to be a fool’s errand. The most popular Active Record implementations are clearly not meant to be confined to such a petty task. The documentation for Rails and Laravel both assume you’ll be using it in the heart of your system and passing these Active Record models around to use everywhere. All the third-party packages assume the same. So do the creators of these frameworks, as do their developer communities. You’d be swimming against a mighty current, and nothing would prevent the “batteries included” features of these ecosystems from bypassing your domain entirely.
To put it plainly: Active Record fundamentally cannot allow the enforcement of constraints on its objects, leaving you with no option but to apply all your business rules in services and therefore “completely miss the point of what object-oriented design is all about” and “incur all of the costs of a [rich] domain model, without yielding any of the benefits”.
]]>Kevin Smith“What’s the deal with Bitcoin?”2022-11-27T05:07:21+00:002022-11-27T05:07:21+00:00https://kevinsmith.io/whats-the-deal-with-bitcoinYou’re probably here because you’ve asked a Bitcoiner that very question. What follows attempts to answer for the average reader what it is, why it matters, and how you can start using it securely, without getting mixed up in scams or losing your bitcoin by mistake. I’ll keep this guide free of any deep, technical detail, but I’ll provide links to go down the rabbit hole if you want.
First, let me just say this: we’re still in the early days of this new technology. Anyone who was just learning about movable type a decade after Gutenberg invented the printing press hadn’t missed out on the printing revolution. In fact, it took a while to catch on. But it supercharged the Renaissance, enabled the Protestant Reformation, and ushered in the Scientific Revolution and the Enlightenment. Every aspect of our world is changed because of the printing press.
You haven’t missed out on Bitcoin. It’s still early.
What is it?
Imagine you could use the cash in your pocket to instantly pay for things on the Internet and didn’t need a bank to keep your money safe. That’s Bitcoin.
Bitcoin is peer-to-peer electronic cash.Peer-to-peer meaning it’s transmitted from person to person without the need for a trusted intermediary like banks or Venmo to process the transaction. Electronic meaning it naturally exists in digital form, so it’s global and can easily be used for online payments. Cash meaning it’s an asset that’s readily available to pay for goods and services without needing anyone else to follow through on their responsibilities—like your bank’s promise to process your debit card transactions.
When someone talks about Bitcoin, they’re referring to one of two things:
Bitcoin (with an uppercase B) is a global network for transferring value between participants that isn’t owned or controlled by anyone, and everyone can be a part of it by running the open source Bitcoin client software.
bitcoin (with a lowercase b) is a digital asset often represented as BTC, and it’s the way to represent value on that network. This is a concept called the unit of account in economics. The total number of bitcoin available now and in the future is known by everyone and unchangeable by anyone. At most, there will only ever be 21 million bitcoin.
Most of the news gets caught up in the price of BTC, but it’s the network that’s revolutionary. BTC is just a necessary part of it, and it wouldn’t be valuable at all without the incredibly well-designed dynamics of the network.
Why does it matter? What problem does it solve?
It’s critical to human flourishing that money remain free from the influence of those who would manipulate it or control populations with it, even for what they consider noble reasons. History is rife with wretched examples, often to the benefit of the elite and powerful and the ruin of everyone else. This sounds overly pessimistic to those of us in the West, but zoom out a bit and you’ll see that we’re enjoying a rare period of stability.
Ancient Rome debased their currency to fund their insatiable spending, leading to runaway inflation, soaring taxes, and ultimately the collapse of the Roman Empire.
European settlers discovered that Africans used glass beads as money, something the Europeans could easily manufacture in mass quantities. They were “able to strip the African continent of its wealth by trading easy-to-produce glass beads for hard-to-produce human slaves” and precious natural resources.
Governments engulfed in a financial crisis—like those in Venezuela, Brazil, Indonesia, Greece, Turkey, China, Taiwan, Argentina, and Russia—often enforce capital controls or block bank withdrawals in a desperate attempt to regain control, forcing destitution on anyone who would leave.
In many parts of the world, protests often result in bank account suspensions (as we’ve seen in Russia, Nigeria, and Hong Kong) and bans from payment channels (as they’ve done in China). Belarus and Canada both recently seized funds from people who merely donated to support protestors. But “no government can turn off Bitcoin if it’s threatened by your ideas.”
Aside from military might backing up the dollar, the current financial system is a patchwork of inefficient, costly, and politically-vulnerable systems that move money around and settle financial obligations for individuals, corporations, and nation-states. Think Visa transactions, Western Union, ACH, Fedwire, CHIPS, etc. Bitcoin and its Lightning Network layer replace all of this with a faster, more reliable system for a fraction of the cost. One company is now using the Lightning Network to offer instant payments at Shopify shops and retail stores using NCR point-of-sale systemsin the shop’s local currency for pennies compared to Visa’s transaction fees. This is the future.
Bitcoin is open 24/7, censorship-resistant, unstoppable, efficient, borderless, pseudonymous, and egalitarian. No one controls it, and no one gets special treatment.
Bitcoin is revolutionary. It may not happen immediately, but eventually you won’t be able to ignore it. There will be a gravity to it, an incentive to be a part of it. Those who don’t use it will put themselves at a disadvantage. Over time, the world’s financial systems will all reorient themselves to the new reality ushered in by Bitcoin.
Each computer running the Bitcoin client software (called a “node”) in the network checks every new group of transactions (called a “block”) to make sure all participants are following the rules. Each node has the same rules baked into its software. Rules like you can’t spend the same bitcoin multiple times and the person trying to spend this bitcoin has the authority to do so.
No one can change the rules in their favor because none of the other nodes will go along with it. Trying to spend the same bitcoin twice? Everyone else will call shenanigans. All of this happens automatically in software without requiring human involvement.
Note that nodes aren’t the same as miners. You’ve probably heard a lot about miners and never really understood what they were doing. Simply put, miners are specialized, expensive computers that are paid bitcoin to process transactions. Want to go deep on mining? Here’s a fantastic video on how it actually works, complete with illustrations to make it easy to understand. I highly recommend it.
Nodes, on the other hand, make transaction requests, ensure miners are following the rules, and keep a copy of the entire history of Bitcoin transactions. Anyone can run a node from their home with very inexpensive hardware, like an old computer. Because they’re the ones enforcing the rules, nodes keep the network secure. River does a great job explaining nodes if you want to learn more.
But bitcoin’s been so volatile lately!
Most Bitcoiners will tell you that we expect bitcoin to go up in value over time, but no one can predict the price in the short term. A few points to keep in mind:
At 13 years old, bitcoin is a relatively new asset on the world stage and one that most people still don’t understand, so it’s helpful to consider bitcoin’s current value in the context of the global economy. At the time of writing, bitcoin’s market cap (the current price x total number of available bitcoin) is about $317 billion. Relatively speaking, that’s still small. Gold’s market cap is over 36x larger at approximately $11.5 trillion. Even the market cap of a single company, Apple, is more than 7x larger than bitcoin at nearly $2.25 trillion. When you jump into a pool, you stir up that body of water far more than when you jump into the ocean. It’s the same thing with buying and selling assets. The more an asset grows in total value, the less any one person’s purchase or sale will move the price. Every asset stabilizes as it grows in value. Bitcoin is no exception, and there’s still a lot of room to grow.
Bitcoin experiences many of the same forces that cause stock market fluctuations because for most people right now, bitcoin is just one of their many investments. They invest heavily during bull markets, and pull back with negative economic news and rising Fed rates. So when you see bitcoin’s price move, take note of what else is happening in the economy.
Don’t mess around with those things. Bitcoin is the genuine article.
How do I get started? How can I keep my bitcoin safe?
You’ve no doubt heard horror stories of people losing their bitcoin through hacks and forgotten passwords.
In the first few years, the most secure way to save bitcoin was using an offline wallet that was never connected to the Internet and was therefore extremely difficult to hack. Unfortunately, if you lost your keys or forgot your password, there was no way to recover it.
For that reason, many people have felt more comfortable leaving their bitcoin on the crypto exchange where they bought it. Unfortunately, that leaves you vulnerable to hacks, phishing, and the poor business practices of the people running the service. As I’m sure you’re well aware, there’s been lots and lots of news recently about companies that haven’t been trustworthy.
The solution to both problems is self-custody in a multisig wallet, a technology that maximizes security and minimizes failure by requiring multiple keys to unlock the wallet. A typical 2-of-3 multisig wallet requires any 2 of its 3 keys to spend the bitcoin. Like launching missiles from a submarine, there’s no longer a single, precious key that unlocks the wallet. These wallets are much harder to hack and much easier for the rightful owner to recover if they were to accidentally lose one of their keys. You can keep bitcoin in your custody instead of trusting an online service to hold it for you and still be confident that you’re not going to lose it.
How do you set up a self-custody multisig wallet? Thankfully there are services like Unchained Capital and Casa that can help you set up your own multisig wallet, put an inheritance plan in place, and recover funds if something goes wrong. And importantly, they never have access to your bitcoin. Take a look at both services, and see which one works best for you. Unchained Capital has concierge onboarding to personally walk you through the process, and Casa has a handy guide to help you get set up.
To buy bitcoin, use a reputable service like Swan Bitcoin, River, Strike, Unchained Capital’s trading desk, or Casa and move it to your multisig wallet as soon as possible. I do not recommend using a crypto exchange that deals with altcoins, yield farming, or the like, since all too often it means the exchange is at risk of liquidity issues and could halt withdrawals at any time.
I also recommend setting up recurring buys to take advantage of dollar cost averaging. I’ve got my automated weekly purchase set up at River since they offer free recurring orders, and I transfer to my multisig wallet once a month.
These recommendations will get you started, and as you learn and get more familiar with the world of Bitcoin, you’ll find many other ways to get involved.
Inventing Bitcoin is an easy, non-technical book that explains Bitcoin’s fundamentals by walking through the problems it solves and explaining how Bitcoin solves them.
River’s Bitcoin Learning Center is a fantastic set of clear definitions and accessible guides to learn about all aspects of Bitcoin and its surrounding ecosystem.
A most peaceful revolution is a widely lauded essay discussing Bitcoin’s implications for the unquenchable, inexorable state.
The Quest for Digital Cash is a history of cryptography, prototypes of digital cash, and how Satoshi Nakamoto pulled together all the best ideas so far to ultimately solve the problems preventing full decentralization.
Robert Breedlove on the Lex Fridman Podcast is a discussion of Bitcoin through a philosophical and economic lens, coming at it from first principles. The ultimate “why” without focusing on the engineering. A great listen.
Above all, remain skeptical, even of everything I’ve told you here. Check it out for yourself. Be especially wary of claims of untold riches. If an opportunity sounds like easy money without work, it’s probably too good to be true.
Updated on Dec 2, 2022: Rewrote the section on multisig wallet services and where to buy bitcoin to provide multiple reputable options for each.
]]>Kevin SmithHigh Cohesion, Loose Coupling2021-09-22T03:03:45+00:002021-09-22T03:03:45+00:00https://kevinsmith.io/high-cohesion-loose-couplingHigh-quality software is both valuable to the business now and easy to change later. While it’s simple to be dogmatic at either end of this spectrum, both goals are important to the business in the long run. Aiming for highly cohesive and loosely coupled code is key to finding the balance, but this concept can be tough to understand at first glance. So what does it actually mean?
High Cohesion, Loose Coupling characterizes software designed with a keen focus on Separation of Concerns, the foundation of effective, resilient, malleable, performant, secure, maintainable software.
High Cohesion describes the internals of a well-bounded module that consists of coupled elements working together to accomplish something, which improves the comprehensibility of the codebase at large. Such a module contains all the decision-making power necessary to do its job.
Loose Coupling means that with respect to a given change, one module can be changed without requiring a change in other modules.
Coupling inside a module leads to cohesion, while coupling outside a module leads to multiple modules that must be kept in sync to execute a given change, increasing the cost of such changes. It’s all about where you draw the boundaries.
It’s worth noting that a module is merely a distinct part of the system that performs some kind of important job and hides its inner workings from the rest of the system, according to David L. Parnas in his
seminal paper on software modularization.
There are small-scale jobs and large-scale jobs, so this characterization of high-quality code applies equally to the system at every scale, from the smallest function to the entire application and beyond.
Updated on Oct 25, 2023: Inspired by Kent Beck’s excellent conference talk on software design, I adjusted some of the language for clarity and added a paragraph to highlight the importance of where the boundaries are drawn.
]]>Kevin SmithSeparation of Concerns is Fundamental to Building High-Quality Software2020-08-18T16:31:00+00:002020-08-18T16:31:00+00:00https://kevinsmith.io/separation-of-concerns-is-fundamental-to-building-high-quality-softwareThere’s something that’s been on my mind a lot lately: the significance of meaningful boundaries. It’s a theme that keeps coming up wherever I look. Not just in software engineering either. The absence of meaningful boundaries means trouble in just about every area of life.
Relationships without healthy boundaries encourage nosy friends and meddling family members, often bringing an endless supply of anxiety along with them.
Poor boundaries with news sources can easily give you a skewed sense of the world.
A business without meaningful boundaries is likely an aimless yet frantic organization where no one knows exactly what’s going on or who’s responsible for it. Everybody is responsible and nobody is responsible, all at the same time. Need to know what’s going on with a particular initiative at the company? You’ll have to ask somebody in every corner of the organization and piece it together yourself.
It’s no different in software engineering. When a piece of software looks like that, we call it a Big Ball of Mud.
A Big Ball of Mud is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated.
If you’ve been building software for long, you’ve likely had a tour of duty in one of these monstrosities. Every change is another opportunity for failure to ripple throughout the system. You start to seriously question why you ever got into this field.
A piece of software with boundaries in the wrong place is a whole other kind of pain. The wave of interest in Microservices in the 2010’s brought about a whole generation of what we’ve come to call the Distributed Monolith, a system with “distinct” services that are actually deeply tangled up with each other. You often can’t even make a simple change to the system without updating multiple services and coordinating a simultaneous code release. Every inch of progress is exhausting.
Separation of Concerns isn’t merely some overwrought principle peddled by purists too busy fussing over theoretical rightness to ever actually ship something. (Though without a doubt the purists do enjoy fussing over it.) It is fundamental to building effective, resilient, malleable, performant, secure, maintainable software. In other words, high-quality software.
In software engineering, there’s a pair of factors we must confront to prevent the system’s natural trend toward disorder.
Interdependency is the degree to which a part of the system relies on or is relied upon by other parts of the system or other systems, especially in non-obvious ways.
Comprehensibility is the degree to which a part of the system can be easily understood for the sake of making accurate changes with predictable results.
These two key factors have an inverse relationship. As interdependency increases, the size of the system that must be accounted for grows and comprehensibility decreases dramatically. And as a result, risk of failure rises exponentially.
Striking the right balance is all about meaningful boundaries. How do you reduce interdependency? Separate the system into distinct parts and make their interactions explicit. That’s the core idea behind Separation of Concerns.
(We’ll dig into how to identify and organize those distinct parts of the system in an upcoming post.)
You can often make tremendous progress in a greenfield project by throwing caution to the wind, but the cost of mixing everything together will inevitably catch up with you. We spend most of our time working within existing code, and once a software system becomes difficult to reason about, every aspect of quality takes a major hit when someone makes a change. To ignore Separation of Concerns is to invite the sclerotic disasters discussed earlier.
At the end of the day, what does Separation of Concerns mean for the business and the people involved?
Faster feature development because changes don’t require a complete, expansive understanding of the entire system.
A far better chance of hitting estimate windows because you won’t discover halfway through that your changes are tangled up with and complicated by other parts of the system.
A much tidier codebase when things need to be removed or refactored, which contributes to future comprehensibility.
Protection against costly failures since problems in one spot are far less likely to ripple throughout the system. Meaningful boundaries help contain the blast radius when something goes wrong.
A quicker recovery when things do go wrong because the issues are likely already isolated, and bug fix testing doesn’t need to be comprehensive.
Less turnover thanks to happier developers who spend their mental fuel on the fun work of shipping new features instead of constantly getting pulled away to troubleshoot and untangle the knots in the system.
Quicker onboarding, which means new developers are meaningfully contributing in days not months.
Sure sounds to me like it’s worth the cost of a being a little more diligent about system design.
Many thanks to Ryan Singer for the enlightening Synthetic A Priori podcast episode Design as a Multi-Scale Network, Risk as Network Structure which was instrumental in clarifying my thinking here. If you’re interested in getting into the weeds on the theory behind all of this, I’d highly recommend giving it a listen.
Say goodbye to cargo cult solutions. Finally get a grasp on character set encoding, learn how it works for PHP apps and MySQL, and become confident in fixing encoding issues once and for all.
]]>Kevin SmithA Better Way to Handle Validation Errors2019-08-13T21:28:00+00:002019-08-13T21:28:00+00:00https://kevinsmith.io/a-better-way-to-handle-validation-errorsWhile working on a project recently, I finally got so frustrated with the awkward way validation errors are typically handled that I was determined to figure out a solution. The usual methods tend to be:
True to their name, exceptions should be reserved almost exclusively for exceptional circumstances that result in program failure. This really isn’t supposed to happen, and now this part of the program can’t continue running. This doesn’t work well for validation errors for a couple of reasons.
First, exceptions give the application a fail-on-first-problem behavior, which means that even though there’s a possibility of multiple validation errors in a method, we can only report back one validation error at a time. That can be pretty frustrating for the user.
Second, it requires the caller to know about the internals of the called class, and that’s not the caller’s responsibility. It’s reasonable to catch a whole category of exceptions (or even all exceptions) when we want to gracefully handle actual program failure, but validation errors each tend to require specific handling which means the caller needs to know exactly which exception to be on the lookout for.
That amounts to a hidden API. Pretty tough to work with, if you ask me.
Another common technique is to bundle up all the validation errors and return them in some sort of payload—an array, a custom Errors object, a Domain Payload object, or the Notification pattern.
This improves on the technique of throwing exceptions by allowing a method to report all the validation errors that it might encounter, but it’s clumsy for a couple of other reasons.
It requires the caller to inspect the response and understand what to do with it. That’s not the caller’s responsibility.
The best case scenario, from the caller’s perspective, is that the response is some kind of known collection of error message strings so that the caller doesn’t have to inspect anything. It can just allow the error messages to bubble back up to the surface of the application. But that brings us to the next problem.
To provide fully-formed error messages, the called object has to take on presentational responsibilities. Now rather than merely noting that a particular validation error has occurred, this technique enlists the called object to provide a user-friendly error message as well.
To be clear, the problem here is that the called object isn’t just reporting what happened, it’s deciding how the caller should handle it. Here, just show this message to the user, it says. But what if the caller wants to handle a particular validation error in some other way? It would have to inspect the bundle of errors and perform string matching to find the one it’s looking for. Gross.
Object-oriented design is fundamentally about keeping responsibilities in their proper place. Tell, Don’t Ask is a well-known guideline for designing objects that stick to their responsibilities.
The opposite approach—the procedural approach—is to ask for data and then make decisions based on the result. As we saw above, that approach leaves us with a brittle system with objects that are inextricably bound together. Change even the phrasing of an error message in the called object, and now you’ve broken the behavior of the caller(s).
So how can we leverage Tell, Don’t Ask for validation errors?
The basic idea is this: when a method could have many possible outcomes, it should tell some other object about those significant events.
Create an interface (CreateUserHandler) with a method for each event and require it in the method signature. Think of it as a very localized event handler.
publicfunctioncreateUser(CreateUserHandler$handler,string$emailAddress,string$firstName,string$lastName):void{if(!$this->isNameValid($firstName)||!$this->isNameValid($lastName)){$handler->nameWasInvalid();}if(!$this->isEmailAddressValid($emailAddress)){$handler->emailAddressWasInvalid();}// If we don't have what we need to create the User, return// Otherwise, create User and transform to data transfer object $userDto$handler->userWasCreated($userDto);}
When calling the method, pass in any object that implements the interface, leaving it up to that object to handle the called method’s events in a way that makes sense for that part of the system.
For example, if you’re using the Action-Domain-Responder architectural pattern, the responder itself might be the most appropriate event handler.
The action passes in the responder when calling createUser.
classCreateUserAction{/** @var UserService */private$service;/** @var CreateUserResponder */private$responder;// ...publicfunction__invoke(Request$request):Response{// Get request input data as $attributes$this->service->createUser($this->responder,$attributes['email'],$attributes['firstName'],$attributes['lastName']);return$this->responder->response();}}
The responder uses the events signaled by createUser to generate a response.
classCreateUserResponderimplementsCreateUserHandler{private$response;publicfunctionemailAddressWasInvalid():void{// Add validation error message to $this->response}publicfunctionnameWasInvalid():void{// Add validation error message to $this->response}publicfunctionuserWasCreated(UserDto$user):void{// Create API payload with $user and add it to $this->response}publicfunctionresponse():Response{// Returns HTTP response with application/json content-typereturn$this->response;}}
All the validation errors? Only the first one? Could other method events have an impact on the response?
Sure, maybe. That’s entirely up to the responder to determine how to handle it. That is the responder’s responsibility after all.
This example is for an HTTP request. Hopefully you can imagine how to set up a CLI command for creating a user as well. The CLI command handler class would just pass in its own responder object that implements CreateUserHandler to generate the command-line response, and the createUser method wouldn’t have to change at all.
For what it’s worth, TJ Draper and I developed this technique together as an improvement upon the aforementioned techniques, but I’ll admit that I find it impossible to believe we actually invented this pattern. Seems to be a variant of the delegation pattern, but it doesn’t include that pattern’s requirement of shared context.
If you know the name of the pattern, please let me know.
]]>Kevin SmithSanitize Your Inputs?2019-01-03T13:58:00+00:002019-01-03T13:58:00+00:00https://kevinsmith.io/sanitize-your-inputsI’m often accused of being particularly fussy with regards to language and word choice, especially in technical discussions. It’s true, but I’ll wear that badge with pride. In software engineering, there are many instances where clear communication is so critical that the success or downfall of an entire organization may rest upon it.
There’s one particularly slippery term that wreaks havoc in the pursuit of application security.
Sanitize.
I say it’s slippery because there is simply no industry-wide agreement on its meaning, and therefore when used, the speaker and his or her audience cannot be entirely sure they understand each other. Its appearance in any discussion should immediately prompt the question, “What do you mean by that?”
Does it mean removing undesirable data while letting the good stuff through? Or converting potentially harmful data into a harmless form? Or flat-out rejecting a request when any invalid data is detected? Or perhaps it even means using prepared statements to protect the database from malicious input. I’ve seen “sanitize” used to mean any (and even all) of these things.
That’s worrisome because these techniques are not interchangeable, especially when it comes to preventing SQL injection. In that case, using prepared statements is the only way to reliably protect your database from SQL injection attacks without the risk of mangling incoming data.
Perhaps the author of the famous Bobby Tables comic actually intended the mom’s snarky response to mean “use prepared statements” instead of filtering the input, but that would be entirely lost on the beginner developer who reads the comic and Googles “sanitize database inputs” to find scores of highly-ranked guides that confidently recommend modifying the input string. (Thank goodness the one guide that tends to top the search results makes it clear that “sanitizing” your inputs is prone to error and promotes prepared statements instead.)
Sanitize your inputs? I think not.
Not just because it’s wrong. Because it’s meaningless.
Let’s look at a few fundamental principles for web application security, and through them find the best language for clear communication.
At every stage of input, ensure that the incoming data is valid according to the requirements of that part of the application. There are many layers in any application, and they all have a job to do. Each one expects to be given certain information that it needs to do its work, and it pays dividends to be as explicit as possible.
Does this PHP class method need a date and time to do its job? Type hint that you’re expecting DateTimeImmutable in the method signature, and if the code calling that method doesn’t provide the right information, PHP will throw a TypeError. This is validation using the built-in capabilities of the language right at the point where the method is being invoked.
<?phpdeclare(strict_types=1);classFoo{publicfunctionbar(DateTimeImmutable$dateTime){// Do something with $dateTime}}
Need a positive integer? Declare the parameter type to be int (with strict typing enabled), and consider using an assertion library like Webmozart Assert to require the incoming data to be greater than 0 before any other work is done in the method. This combines built-in validation features and a broadly-used, well-tested third-party solution to ensure you’re working with meaningful data.
<?phpdeclare(strict_types=1);useWebmozart\Assert\Assert;classFoo{publicfunctionbar(int$eventId){Assert::greaterThan($eventId,0,'The event ID must be a positive integer. Got: %s');// Do something with $eventId}}
Expecting an argument to be a string with the value of either “month” or “year”? If the value of the incoming data doesn’t match one of the two (and your business dictates that it’s not possible to set a reasonable default), throw an InvalidArgumentException (or use Webmozart Assert’s oneOf assertion). This is a technique called whitelisting.
<?phpdeclare(strict_types=1);classFoo{publicfunctionbar(string$timeFrame){$timeFrame=mb_strtolower($timeFrame);if(!in_array($timeFrame,['month','year'])){thrownew\InvalidArgumentException("TimeFrame must be either 'month' or 'year'. Got: {$timeFrame}");}// Do something with $timeFrame}}
Even better, consider using enums when the value should always be one of a declared (i.e. enumerated) list of values.
Input validation is stricter than what most developers imagine when they think of sanitizing inputs. Rather than merely “cleaning” the incoming data, we’re ensuring it adheres to a very specifically-defined format or rejecting it entirely.
By declaring and enforcing these expectations, the application is a lot less likely to exhibit unexpected or undesirable behavior, the playground of nearly all security vulnerabilities. This approach is not sufficient to protect against any threat—no single technique is—but ensuring the integrity of the data moving around the application goes a long way in reducing an application’s attack surface.
Beyond the improvement in security, your engineering team will enjoy working in a far more intelligible codebase, and the business will benefit from more reliable features delivered more quickly.
#
Send Query and Parameters Separately to the Database
SQL injection happens when an attacker sneaks additional database instructions into your existing query. As noted above, the most famous example is smuggling a “drop table” statement in with an existing query, designed to maliciously destroy an entire database table.
The technique to prevent this type of attack is fairly straightforward. Isolate data from the instructions designed to operate on it, then (and this is important) literally send them as separate messages to the database server.
This allows your application to query the database server like so: “Give me all the columns from the Students table for rows where the first_name is ___; I’ll send a separate message with something to fill in the blank.” Then a very short time later, your application sends another message, “Fill in that blank with Robert.”
If the database server receives Robert'); DROP TABLE Students;-- instead, it won’t execute the DROP TABLE statement. The database server knows that’s a value, so it won’t let it alter the original instructions it received. It will treat that value literally, search for a student named Robert’); DROP TABLE Students;–, and return nothing.
It’s straightforward, fool-proof, and unlike “sanitizing” an input string, carries no risk of accidentally mangling the incoming data.
Escaping has the specific goal of preventing injection attacks — typically HTML injection, of which the most famous form is XSS. If our application is passing along user-supplied data, it’s our application’s responsibility to ensure that data is never regarded as code that should be executed or interpreted.
The eagle-eyed reader will notice a parallel with SQL injection prevention: we’re giving the data special treatment to prevent its execution as code. But unlike with prepared statements, there’s no way to send application output code and data separately to ensure such a clear distinction. We must use some other method to ensure the data cannot be executed. That’s where escaping comes in handy.
Escaping is the conversion of user-supplied data to a form that the receiving system will not mistake for code. As a very simple example (that absolutely is not advice to be followed blindly), running a blog comment through PHP’s htmlspecialchars() will ensure any HTML included in the comment doesn’t actually get rendered by the browser.
Unfortunately, the terms “escaping” and “encoding” havea long historyof being used interchangeably for this purpose. Either term should be understood to refer to the same concept outlined here. I have opted for “escaping” to align with the language most commonly used in the PHP community and to avoid confusion with the unrelated concept of setting the character encoding of output content. But as you’ll note later on, even some of the quotes and sources I reference use “encoding”. As much as I would love there to be One True Word to use here, there isn’t. Try not to get too hung up on it.
I wholeheartedly suggest using a well-tested library to handle escaping. There are simply too many gotchas that are easy to miss if you try to do it on your own. Zend Escaper is a fine choice, or if you’re already using a template engine like Twig, look for escaping utilities built right in. Both of these tools offer options for escaping in the appropriate context.
Escape user-supplied data as late as possible. This ensures that nothing is erroneously assumed to have already been escaped, allowing it to slip through the cracks, and that the appropriate escape method is performed for the given context. For a web application serving up a web page, this would probably be in the HTML template itself since the context is obvious by that point.
So how do you handle user-generated content that actually should be rendered on the front-end? Consider a blog post editor where the author writes in a WYSIWYG field. That rich text content is going to include a lot of HTML. How can you get the browser to safely render it?
Filtering is a security technique that involves examining user-supplied data and removing anything that shouldn’t be there. This is the high-wire act of web application security, so it’s a tactic that should be reserved for a very narrow set of circumstances where escaping content doesn’t make sense. That’s almost exclusively when content originates from a rich text editor.
There’s a lot that could go wrong here — it’s easy to be too aggressive in stripping HTML elements or not aggressive enough and fail to remove malicious input — so save yourself the trouble and use a well-tested library like HTML Purifier. And as with escaping, you should apply this as late as possible.
It’s not uncommon to see applications attempt to filter and escape data coming into the application — this is what’s often called “sanitizing” — but that should really be avoided.
There are security concerns at stake:
If you store sanitized data in a database, and then a SQL injection vulnerability is found elsewhere, the attacker can totally bypass your XSS protection by polluting the trusted-to-be-sanitized record with malware.
And escaping and filtering on input raises maintainability issues as well:
Be warned: you might be tempted to take the raw user input, and do the encoding before storing it. This pattern will generally bite you later on. If you were to encode the text as HTML prior to storage, you can run into problems if you need to render the data in another format: it can force you to unencode the HTML, and re-encode into the new output format. This adds a great deal of complexity and encourages developers to write code in their application code to unescape the content, making all the tricky upstream output encoding effectively useless. You are much better off storing the data in its most raw form, then handling encoding at rendering time.
So escape and filter where it makes the most sense: on output.
Don’t rely on these techniques to protect your database or ensure the validity of data flowing around your application. They might sometimes provide those security benefits accidentally, but that’s not what they were designed to do and they often come with a hidden cost.
We don’t build this stuff alone. Software engineering is a human endeavor, and building great software means working well with other people. I hope this overview helps encourage much clearer communication amongst your team.
Update: A few people suggested that many of the concepts discussed here are covered by the maxim “Filter Input, Escape Output”. I have no desire to reinvent the wheel, so if there’s an industry standard, I certainly want to stick to it.
Chris Shiflett coined this phrase almost 14 years ago, and I was fortunate to have a good conversation with him shortly after publishing this post. That conversation led me to dive into quite a bit of additional research, resulting in a significant rewrite of the section dealing with output.
On the surface, there remains a small disagreement over whether to filter on input or output, but that fades quickly with a look at how Chris defines input filtering, which lines up with exactly what I’ve recommended here for input validation. He even uses the word validation in his definition. And importantly, in the intervening years since the maxim was coined, the web application security community seems to have settled on “input validation” to more clearly describe this technique.
Further, “filtering” is commonly understood to mean blocking or containing the bad parts of a stream while letting the rest through. An air filter doesn’t test the air and completely block it if any parts are considered bad; dirty air goes into the filter, clean air comes out.
With that in mind, I maintain that filtering is context-sensitive modification of the data and thus most appropriate during output.
]]>Kevin SmithWhy Your Team is Burnt Out and Getting Nothing Done2018-09-28T02:50:20+00:002018-09-28T02:50:20+00:00https://kevinsmith.io/why-your-team-is-burnt-out-and-getting-nothing-done.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }
]]>Kevin SmithWhere the Real Work is Done2018-08-15T19:38:11+00:002018-08-15T19:38:11+00:00https://kevinsmith.io/where-the-real-work-is-doneIf you want to make a difference in software, get better at modernizing legacy applications. The tech world is desperately in need of such skills, and it’s only gonna get more dire from here.]]>Kevin Smith“If you think of coding as the manipulation of data, you’re going to have a hard time writing object-oriented code.”2018-07-11T20:32:00+00:002018-07-11T20:32:00+00:00https://kevinsmith.io/if-you-think-of-coding-as-the-manipulation-of-data-youre-going-to-have-a-hard-time-writing-object-oriented-codeThat’s an off-the-cuff comment I made during a presentation at the Nashville PHP meetup last night, and one of the attendees messaged me later asking if I could expand on that thought.
First, I think a definition is important. Data is facts, statistics, quantities, values, characters, etc. In the context of programming, it’s anything that can be assigned to a variable.
Procedural code sends data through a series of instructional steps to accomplish the goals of the business. Functions may be used to make it more modular, but if the emphasis is on manipulating data as it flows through a series of steps, it’s procedural code.
Object-oriented programming is all about accomplishing the goals of the business through a community of collaborating objects, the design for which must start with a deep understanding of the business itself. Data isn’t irrelevant in OOP, but an object isn’t just a dumb bag of data either. It’s an agent of purpose for the business. Data is important only insofar as the object needs to know certain things to accomplish its role. (Notably, in many cases the specific data an object holds in order to accomplish its purpose could change without the purpose itself changing.)
Data is in service to the purpose. It isn’t the purpose itself. Ultimately, all data in software is surrounded by a boundary that defines purpose and gives meaning to the data contained within. With procedural code, it’s often the user interface layer doing that work for the entire system. In OOP, it’s each individual object.
So it’s a big difference in mindset. In my opinion, if you start by thinking about the data and how it’ll be stored, structured, and manipulated, you’re already concerned with tactics before you even have a strategy. You’re starting from the wrong end of things, and it will dramatically affect how you design the entire system.
]]>Kevin SmithWhat’s So Great About OOP?2018-07-09T16:52:00+00:002018-07-09T16:52:00+00:00https://kevinsmith.io/whats-so-great-about-oopOne of my core responsibilities as a senior software engineer is mentoring the junior engineers and interns at work. It’s one I don’t take lightly, and as a follow-up to discussions where new concepts are introduced, I usually try to find a good written reference for them to bookmark for later.
Recently a question came up with our new interns while talking through the design of a feature:
What is object-oriented programming? How is it different from what we’re doing now, and why should we write code that way?
I wanted to find a fundamental explanation to send them for later, but nearly everything I came across immediately jumped into SOLID or design patterns or advanced concepts of some kind.
The code the interns and I were looking at during this discussion was written in a procedural style. Procedural code accomplishes the goals of the system through a series of instructional steps, often breaking those steps up into procedures—called functions, routines, subroutines, etc.—as a way of making the code more modular and easier to understand.
It’s the way most of us learned to code. Once you start thinking of programming as the manipulation of data, this style comes pretty naturally. Do this, then do this. Assign this value to that variable. If this condition is true, do this. Then pass these variables into this function, etc.
Here’s a real world example based on a project I actually worked on, and it should help illustrate a fairly common pattern in procedural code.
Imagine a front-end form that allows users to create a new “schedule” (recurring set of events). Once a form is submitted, the form data is passed to a function that figures out if anything is missing and does a few transformations to get the data into the right format. Then it passes it to another function that saves it to the database.
You’ve been asked to build an importer to load data from an uploaded CSV, so you’ll need to reference this existing code a lot and figure out where your importer will need to intersect with it, if at all.
Here’s a simplified version of what that looks like.
functioncreateSchedule($schedule_data){if(createScheduleErrors($schedule_data)){returnfalse;}// Apply default values if optional fields are empty...// Create new array $fields from part of $schedule_data, ignoring irrelevant data// Manipulate $fields in a variety of ways to make it acceptable to saveSchedule()returnsaveSchedule($fields);}functioncreateScheduleErrors($schedule_data){// Check for required keys in array}functionsaveSchedule($fields){// Save to the database}
Hopefully you can see how… imprecise that can be. It’s not at all obvious what the data means or which parts are required to actually create a new schedule. That makes reusability hard.
name is obviously the customer’s name, right? Nope, it’s the schedule name.
customer_id and client_id have the same value. Is that a coincidence? Or are they required to be the same? One of them is actually unnecessary, left over from old code before a refactoring.
frequency of what? Does this schedule end after 1 iteration? It does not.
Ah, so no_end_date must be required for this schedule to repeat forever. Nope, just leave end_date, which we don’t even see here, blank. no_end_date is an artifact from a checkbox on the form that clears and disables the End Date field.
What’s a valid value for mileage, if we need to provide it?
Where does plan_id come from? Is that generated in the form? A pre-existing ID? It’s generated after the submission and added to $_POST.
Are there more optional fields we could provide that just weren’t part of this form?
You actually have to read through quite a lot of code to try to get answers to these questions. And that’s just code that’s obviously related. What about some other part of the codebase you don’t even realize you should be concerned about? You know, the part that alters a global that affects how this code works?
At the time this code was written, the author certainly understood the requirements and constraints that were central to the concept of a schedule. Unfortunately, those business rules were only partially included in the logic of the program and not in an easily reusable way, so they’re a bit of a mystery for the next developer to work in this part of the codebase.
Object-oriented programming is a way of building software that encapsulates the responsibilities of the system in a community of collaborating objects. An object consists of information (properties of the object) and behavior (methods of the object that operate on the object’s information) that work together to fulfill the object’s role in the system.
OOP is more than just different syntax. It’s a paradigm shift away from thinking of programming as the manipulation of data. This is an entirely different way of modeling solutions to real problems.
You may have heard that OOP is all about modeling the real world. This is true in a general sense, though this axiom is often misunderstood to mean that objects themselves should literally model real world objects. That goes too far with the analogy.
Instead, think of it from a higher-level view. A community of objects should model the interactions and responsibilities we see in agents of purpose in the real world. Put another way, objects should be designed to solve problems like we solve them in every day life.
When you go to a restaurant, they show you a menu of choices. You tell the waiter what you’d like, and they bring it to you. You don’t run back to the kitchen to check the inventory, get out the necessary ingredients, and tell the chef how to prepare your dinner. You leave that responsibility to the restaurant. They advertise a delicious dinner, and you trust that they can handle it.
Returning to our “schedule” example, let’s see what a class that defines schedule objects might look like.
classSchedule{private$client;private$events=[];private$id;private$name;private$serviceRate;private$startDate;private$worker;publicfunction__construct(int$id,string$name,Client$client,Worker$worker,ServiceRate$serviceRate,Event$event,DateTimeImmutable$startDate=null){$this->id=$id;$this->name=$name;$this->client=$client;$this->worker=$worker;$this->serviceRate=$serviceRate;$this->startDate=$startDate??newDateTimeImmutable('now',newDateTimeZone('UTC'));$this->addEvent($event);}publicfunctionaddEvent(Event$event):void{$this->checkEventConflicts($event);$this->events[]=$event;}privatefunctioncheckEventConflicts(Event$event):void{// Throw exception if this event conflicts with any already in $this->events array.}}
Note that the constructor requires everything that constitutes a valid schedule according to the business:
Name
Client, for whom this schedule exists
Worker who will attend to the client
Service rate, so we can determine billing and payment
At least one event
This is already a major improvement over the procedural example. What’s the bare-minimum required for a schedule to be valid? Now we know.
We can also declare an optional start date upon instantiation or allow it to default to now. According to the business, schedule start dates can’t be changed, so this is only available in the constructor. A method for changing the start date on an existing Schedule would violate our business rules.
Let’s set up a new object with the data from the procedural example.
$schedule=newSchedule(42,'Doe, John',$client,// Object representing client with ID 11$worker,// Object representing worker with ID 27$serviceRate,// Object representing hourly service rate with ID 7$monday,// An event object representing Monday from 8:30am to 5:00pmnewDateTimeImmutable('06/24/2018',newDateTimeZone('UTC')));$schedule->addEvent($wednesday);// An event object representing Wednesday from 8:30am to 5:00pm$schedule->addEvent($thursday);// An event object representing Thursday from 8:30am to 5:00pm// etc...
For the sake of clear examples, I’m omitting some things, like the definition and creation of the objects we’re passing in here. But that’s kind of the point, isn’t it? A Schedule object needs to know specific information about the client, worker, service rate, and events, so it type hints against Client, Worker, ServiceRate, and Event objects to ensure it has exactly what it needs. Other parts of the system are responsible for the creation and validation of those objects, so by the time they get passed in during the construction of Schedule, we know we’ve got valid information to work with.
Remember, OOP is about more than just a single object here or there. The benefits of OOP emerge in a community of objects, when objects begin collaborating with each other.
Again, this is a major improvement over the procedural code, where the requirements of the createSchedule function were entirely hidden.
As we’ve already seen, we can’t change the start date on an existing schedule. But what can we do? Some additional methods are called for here, and whatever is allowed should be clearly advertised by the method names.
classSchedule{// ...private$endDate;private$repeatEveryNumberOfWeeks=1;private$roundTripMileage;// ...publicfunctionchangeRoundTripMileage(float$mileage):void{if($mileage<=0){thrownewInvalidArgumentException('Mileage must be a positive number.');}$this->roundTripMileage=$mileage;}publicfunctionendOn(DateTimeImmutable$endDate):void{$now=newDateTimeImmutable('now',newDateTimeZone('UTC'));if($endDate<$now){thrownewInvalidArgumentException('Cannot make a schedule end in the past.');}$this->endDate=$endDate;}publicfunctionreassignTo(Worker$worker):void{$this->worker=$worker;}publicfunctionrepeatEveryNumberOfWeeks(int$frequency):void{if($frequency<=0){thrownewInvalidArgumentException('Frequency for repeating every x number of weeks must be a positive integer.');}$this->repeatEveryNumberOfWeeks=$frequency;}}
Note that each method is responsible for ensuring the information being added or updated is valid according to the business rules.
This just barely scratches the surface. There’s a lot more that Schedule might be responsible for as part of its role in the system. The point here is just to introduce you to the object-oriented way of thinking.
That objects are responsible for the important work of the system is not only the distinguishing characteristic of OOP, it’s fundamental to the very practice of writing object-oriented software.
This is a point worth repeating. As programmers learn OOP, it’s easy to get lost in theory, principles, and design patterns and lose sight of the objects themselves. If you’re new to OOP, first commit yourself to thinking in terms of objects, each with its own role. It’s crucial. Everything else in OOP is built on it.
At some point you’ll start running into mental obstacles, unsure of how to proceed. And instead of seeing those principles and design patterns as oppressive, dogmatic rules to follow, you’ll find them to be a helpful toolkit for writing better code.
I’ve alluded to it already, but let’s be clear about something. There’s a common misconception that using classes means the code is object-oriented. That’s not even a little bit true.
Think of a class as a blueprint; a code organization tool, if you will. That blueprint can be for a set of related functions and scoped variables. That’s still procedural code. Or it can be the blueprint for objects that are created when that class is instantiated, each of which with knowledge and behavior that helps it accomplish its role. That’s object-oriented.
#
Seems like a lot more code to do the same thing. What's the real benefit?
In a word: sanity.
Object-oriented programming is orders of magnitude better at handling complexity than procedural programming. Divvying up responsibilities frees you up to focus on the problem at hand, which means a dramatically lower cognitive load. It makes it so much easier to reason about the code you’re working in. While there are quite a few big benefits to object-oriented programming, that’s the one that wins the day for me.
With procedural code, business rules are scattered all throughout the system and locked up in the head of the most experienced programmer on the team. To effectively write new code, a coder must build up an understanding of how the entire system works and what state it might be in by the time it reaches the code they’re working on.
Is “service code” a required data point for invoices in this system? What is a valid service code? Only digits? Letters and numbers? Are any special characters allowed? Once an invoice has been sent, can it be deleted? Or changed in any way?
In a procedural codebase, the answer to these fairly important questions could be anywhere from an input form, to a code comment that may or may not be out of date, to the many places where they’re saved to the database.
With OOP, those business rules are written into the responsible objects, hiding the complexity of their implementation. Code is read by humans many times more often than it’s written, and OOP helps optimize for the humans who read it.
Stop running into the restaurant’s kitchen with your arms full of groceries and a list of instructions for the chef. Future developers will thank you. Even if it’s just you in six months.
]]>Kevin SmithProtect Your PHP Application from SQL Injection2018-04-10T02:54:00+00:002018-04-10T02:54:00+00:00https://kevinsmith.io/protect-your-php-application-from-sql-injection
SQL injection is when a malicious SQL query is injected into a legitimate query run by the application, usually by a nefarious user through an input field in the user interface. It happens when the application isn’t protecting the database from raw user input—common in so many PHP applications—and a cleverly formed input tricks the database into running the malicious query.
Imagine this query in an application with user profiles, where $_GET['userId'] comes from the query string to identify the user for the profile currently being viewed.
$query='SELECT * FROM Users WHERE id='+$_GET['userId'];
This query would return the user data we’re looking for.
But what if the viewer changes the query string to profile.php?userId=1%20OR%201=1?
Given the code above, the query would look like this going into the database.
SELECT*FROMUsersWHEREid=1OR1=1
That’s a valid query, and if your application just runs it as-is, doing nothing to protect the database from the user input, that query will evaluate true for every record in the Users table. It will return all the data for every record in the table. You’ll likely leak details about all your users.
Now yes, of course you should be validating all user input coming into your application, but that is an entirely different responsibility. That’s more about rejecting bad data than protecting the database.
While it’s possible to prevent attacks by rewriting the incoming data stream before you send it to your database driver, it’s rife with dangerous nuance and obscure edge-cases. (Both links in the previous sentence are highly recommended.)
Unless you want to take the time to research and attain complete mastery over every Unicode format your application uses or accepts, you’re better off not even trying to sanitize your inputs. Prepared statements are more effective at preventing SQL injection than escaping strings.
Furthermore, altering your incoming data stream can cause data corruption, especially if you are dealing with raw binary blobs (e.g. images or encrypted messages).
Prepared statements are easier and can guarantee SQL Injection prevention.
(You may see this same technique referred to as “parameterized queries” or “parameter binding” elsewhere. Practically speaking, the terminology is interchangeable.)
It’s actually not that hard to prevent SQL injections in a PHP application with a MySQL database.
Native prepared statements are guaranteed to prevent SQL injection attacks.
Think of prepared statements like this: you “prepare” the database for a query you’re going to run by sending a template with placeholders for variable data in the query. Then separately, you send data for those placeholders and tell the database to execute the query.
The code above rewritten as a prepared statement looks like this.
$statement=$pdo->prepare('SELECT * FROM Users WHERE id=?');$statement->execute([$_GET['userId']]);$user=$statement->fetch(PDO::FETCH_ASSOC);
With native prepared statements, that interaction does indeed take place in 2 separate requests. Because the statement and the data are sent separately—literally sent as separate packets of data to the database server—there’s no way for cleverly formed data to alter the structure of the SQL query.
The prepared statement and data are not sent separately if emulation is enabled, so you should disable it. Emulated prepare statements are constructed differently (by PHP rather than by the database server), and don’t fully protect you from SQL injection.
Important:never allow user input directly in the statement template, such as letting user input dictate which table or column is being queried. Doing so would still leave you open to attack. Consider using a whitelist in this case.
Use PDO to connect to your database, explicitly set your connection charset, and disable emulated prepare statements. For clarity on errors, I strongly recommend enabling exceptions for error handling as well.
If you’re using a good query builder or ORM that always uses native prepared statements and sets the connection charset in PDO’s DSN parameter under the hood, you’re already protected. But don’t breathe that sigh of relief until you do some research to ensure your favorite database abstraction makes use of native prepared statements.
Still, it’s important to be very familiar with these techniques. At some point, you’ll run into a situation where you need to directly query the database. Or you’ll find yourself working on a legacy application that doesn’t have a query builder or ORM that you can rely on for protection.
It is ultimately your responsibility to ensure your application is protected from one of the most continually pervasive web application threats out there.
]]>Kevin Smith“Didn’t you just build your own framework?”2018-04-04T16:17:00+00:002018-04-04T16:17:00+00:00https://kevinsmith.io/didnt-you-just-build-your-own-frameworkWow, my last post got way more attention than I expected, even making it all the way to the front page of Hacker News this past Saturday! To my surprise most of the response was positive. Even Reddit and Hacker News weren’t too brutal. There was the expected amount of snark, to be sure, but there was also some genuine misunderstanding and confusion about a few things, so let’s clear them up.
#
So... this is a framework. You just built a framework.
Well, no.
You could blindly use the resulting code as your new micro-framework, but I wouldn’t recommend it. It’s not intended to be used in production, and the only documentation for it is the blog post that shows how it was wired up.
Indeed, the point of that post was to **show how it was done.**
At some point in the future, you will find yourself starting down a legacy application. Perhaps you didn’t make any of the decisions to get the application to this point, but here you are. It doesn’t have a DI container, but it needs one.
Faced with such a need, I’ve seen countless developers assume this means they need to rebuild the application in a framework, or worse, try to slap a framework onto the existing codebase. In all but the simplest of applications, both courses of action are a fool’s errand.
Aside from a rewrite nearly always taking longer than estimated and blocking new business value while in progress, it’s impossible to know all the business knowledge contained in that existing application. You’ll almost certainly lose some of it in the rewrite.
Oh, and you’ll introduce new bugs. The rewrite won’t be perfect either.
To avoid a rewrite though, you need to know how those frameworks do what they do under the hood. If the only experience you’ve ever had building PHP applications was with a framework, you might not even know how to add Composer or how to send a response back to the client that requested it. And that’s okay. It’s actually pretty hard to find an easy-to-follow guide that connects all the dots, especially for those things that a framework usually handles for you. The blog post attempts to fill in some of those gaps.
I saw some variation of this comment several times, and I’ll bundle with it another one I saw: If you want to grow as a developer, you should learn Go, Node.js, Python, etc.
Missing the point, of course. Yes, you can learn a lot by trying your hand in another language, but the blog post is specifically speaking to PHP developers about a circumstance that they will one day face if they haven’t already: an existing PHP application in much need of some modernization.
I can’t think of more terrible advice than to tell that developer they should rewrite the whole thing in another language.
#
Interoperability is over-rated. Nobody actually swaps out components in real applications.
I remember years ago hearing that a great benefit to using a database-agnostic query builder was that you could swap out your database system later on. That was ridiculous, of course. Nobody actually did that with real applications.
So I get where this argument is coming from. But the benefit to interoperability and the PSR standards isn’t that you can swap out implementations later on. The benefit is that packages can be designed and built by different vendors without being tightly coupled to a particular implementation of a more foundational component like HTTP messages, a DI container, a logger, or a middleware handler.
Before PSR-11, any PHP library that needed to resolve dependencies would simply require a particular DI container. If you didn’t use that DI container, you couldn’t use that library. A common interface for dependency injection containers means that such a library can easily be written for compatibility with a wide variety of DI containers.
Even more commonly, third-party packages that dealt with any aspect of HTTP messages tended to be tightly coupled to a particular framework. Since one of the previously unique benefits PHP frameworks provided was an abstraction of HTTP messages, that was the only way these package developers could be sure their library would work in a given codebase.
Ultimately, interoperability standards lay the groundwork for a richer component ecosystem with packages that are available for much wider usage than before.
As an application developer, interoperability means you can build with the confidence that you won’t be pigeonholed into a particular dependency graph just because your application makes use of a particular package now. If you stick to the standards, there’s a wide variety of packages available to plug in whenever you need them.
#
But a framework lets me get started building a real application right away.
Sure, but the blog post wasn’t talking about greenfield projects. It was encouraging developers to learn how it works under the hood so they can apply modern coding techniques to existing PHP applications.
#
This is wildly over-engineered just to display Hello World.
I mean, come on.
]]>Kevin SmithModern PHP Without a Framework2018-03-20T13:29:00+00:002018-03-20T13:29:00+00:00https://kevinsmith.io/modern-php-without-a-frameworkI’ve got a challenge for you. The next time you start a new project, try not using a PHP framework.
Now, this isn’t an anti-framework screed. Neither is it a promotion of not-invented-here thinking. After all, we’re going to be using some packages written by several framework developers in this tutorial. I’ve got nothing but great respect for the innovation going on in that space.
This isn’t about them. This is about you. It’s about giving yourself the opportunity to grow as a developer.
Perhaps the biggest benefit you’ll find working without a framework is the wealth of knowledge about what’s going on under the hood. You get to see exactly what’s happening without relying on the framework’s magic to take care of things for you in a way that you can’t debug and don’t really understand.
It’s quite possible your next job will not grant the luxury of starting a greenfield project with your framework of choice. The reality is that most high-value, business-critical PHP jobs involve existing applications. And whether that application is built in a framework currently enjoying popular support like Laravel or Symfony, a framework from days gone by like CodeIgniter or FuelPHP, or even the depressingly widespread legacy PHP application employing an “include-oriented architecture”, building without a framework now will better prepare you to take on any PHP project in the future.
In the past, it was an uphill battle to build without a framework because some kind of system had to interpret and route HTTP requests, send HTTP responses, and manage dependencies. The lack of industry standards necessarily meant that, at the very least, those components of a framework were tightly coupled. If you didn’t start with a framework, you’d end up building one yourself.
But today, thanks to all the autoloading and interoperability work done by PHP-FIG, building without a framework doesn’t mean building it all by yourself. There are so many excellent, interoperable packages from a wide range of vendors. Pulling it all together is easier than you think!
Before we get into anything else, it’s important to understand how PHP applications interact with the outside world.
PHP runs server-side applications in a request/response cycle. Every interaction with your app—whether it’s from the browser, the command line, or a REST API—comes into the app as a request. When that request is received, the app is booted up, it processes the request to generate a response, the response is emitted back to the client that made the request, and the app shuts down. That happens with every interaction.
Armed with that knowledge, we’ll kick things off with the front controller. The front controller is the PHP file that handles every request for your app. It’s the first PHP file a request hits on its way into your app, and (essentially) the last PHP file a response runs through on its way out of your app.
Let’s use the classic Hello, world! example served up by PHP’s built-in web server just to make sure we’ve got everything wired up correctly. If you haven’t already done so, be sure you have PHP 7.2 or newer installed in your environment.
Create a project directory with a public directory in it, and then inside it create index.php with the following code.
When you first started with PHP, you may have used includes or requires statements throughout your app to bring in functionality or configuration from other PHP files. In general, we want to avoid that because it makes it much harder for a human to follow the code path and understand where dependencies lie. That makes debugging a real nightmare.
The solution is autoloading. Autoloading just means that when your application needs to use a class, PHP knows where to look for it and automatically loads it when it’s called for. It’s been available since PHP 5, but its usage really started picking up steam with the introduction of PSR-0 (the autoloading standard that has since been superseded by PSR-4).
We could go through the rigamarole of writing our own autoloader, but since we’re going to use Composer to manage third-party dependencies and it already includes a perfectly serviceable autoloader, let’s just use that one.
Make sure you’ve got Composer installed on your system. Then setup Composer for this project.
composer init
This will take you through an interactive guide to generate your composer.json configuration file. Once that’s done, open it in an editor and add the autoload field so it looks something like this. (This makes sure the autoloader knows where to look to find our app’s classes.)
{"name":"kevinsmith/no-framework","description":"An example of a modern PHP application bootstrapped without a framework.","type":"project","require":{},"autoload":{"psr-4":{"ExampleApp\\":"src/"}}}
Now install composer for this project, which brings in any dependencies (if we had any yet) and sets up the autoloader for us.
composer install
Update public/index.php to bring in the autoloader. Ideally this is one of the few “include” statements you’ll use in your app.
If you reload your app in the browser, you won’t see anything different. The autoloader is there, it’s just not doing any heavy lifting for us. Let’s move the Hello, world! example to an autoloaded class to see how that works.
Create a new directory from the project root called src, and inside it add HelloWorld.php with the following code.
Dependency Injection is a programming technique where each dependency is provided to a class that requires it rather than that class reaching outside itself to get the information or functionality it needs.
For example, let’s say a class method in your application needs to read from the database. For that, you’ll need a database connection. A common technique is to create a new connection with credentials retrieved from the global space.
1
2
3
4
5
6
7
8
9
10
11
12
13
classAwesomeClass{publicfunctiondoSomethingAwesome(){$dbConnection=new\PDO("{$_ENV['type']}:host={$_ENV['host']};dbname={$_ENV['name']}",$_ENV['user'],$_ENV['pass']);// Make magic happen with $dbConnection}}
But that’s messy, it puts responsibilities in this method that don’t really belong here—creating a new DB connection object, retrieving credentials, and handling any issues that might come up if the connection fails—and it leads to a lot of code duplication across the app. If you ever try to unit test this class, you’ll find that you can’t. This class is tightly coupled to both the app environment and the database.
Instead, why not be as clear as possible about what your class needs to begin with? Let’s just require that the PDO object be injected into the class in the first place.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
classAwesomeClass{private$dbConnection;publicfunction__construct(\PDO$dbConnection){$this->dbConnection=$dbConnection;}publicfunctiondoSomethingAwesome(){// Make magic happen with $this->dbConnection}}
That’s a lot cleaner, easier to understand, and less prone to bugs. Through type hinting and dependency injection, the method declares exactly what it needs to do its job and gets it without calling up an external dependency from inside itself. When it comes to unit testing, we’re in great shape to mock up a database connection and pass it in.
Now a dependency injection container is a tool that you wrap around your entire application to handle the job of creating and injecting those dependencies. The container is not required to be able to use the technique of dependency injection, but it helps considerably as your application grows and becomes more complex.
A brief aside: Autowiring is a great feature offered by many DI containers to help minimize mindless configuration, but we’re disabling it in this tutorial to maximize learning. Explicitly configuring all dependencies here will give you a much better understanding for what an autowiring DI container is doing under the hood.
As it stands right now, this just looks like a lot of extra fuss to do what we were already doing before.
Not to worry, the container comes in handy when we add a few other tools to help direct the request through our application. They’ll use the container to load the right classes when and where we need them.
If you imagine your application like an onion with requests coming in from the outside, going to the center of the onion, and going back out as responses, then middleware is each layer of the onion receiving the request, potentially doing something with that request, and either passing it on to the layer below it or creating a response and sending it back up to the layer above it. (That can happen if the middleware is checking for a certain condition that the request does not satisfy, like requesting a non-existent route.)
If the request makes it all the way through, the app will process it and turn it into a response, and each middleware in reverse order will receive the response, potentially modify it, and pass it on to the next middleware.
A sampling of use cases where middleware can shine:
Debugging issues in development
Gracefully handling exceptions in production
Rate-limiting incoming requests
Responding to incoming requests for unsupported media types
Handling CORS
Routing requests to the appropriate handler class
Is middleware the only way to implement tools for handling these sorts of things? Not at all. But middleware implementations make the request/response cycle that much clearer for you, which means debugging is that much easier and development is that much quicker.
We’ll take advantage of middleware for the last use case listed above: routing.
A router uses information from an incoming request to figure out which class should handle it. (e.g. The URI /products/purple-dress/medium should be handled by ProductDetails::class with purple-dress and medium passed in as arguments.)
To get our app to work with the FastRoute middleware—and any other middleware we install—we’ll need a middleware dispatcher.
PSR-15 is a middleware standard that defines interfaces for both middleware and dispatchers (called “request handlers” in the spec), allowing interoperability amongst a wide variety of middleware and dispatchers. We just need to choose a PSR-15 compatible dispatcher, and we can be sure it’ll work with any PSR-15 compatible middleware.
Now let’s add the FastRoute and request handler middleware. (FastRoute determines if a request is valid and can actually be handled by the application, and the request handler sends Request to the handler configured for that route in the routes definition.)
The astute reader will quickly notice that although we’re still going to the trouble of configuring and building the DI container, it’s not actually doing anything for us. The dispatcher and middleware can do their job without it.
So when does it come into play?
Well, what if—as would nearly always be the case in a real application—the HelloWorld class has a dependency?
Let’s introduce a trivial dependency and see what happens.
That’s happening because now HelloWorld requires a string to be injected at construction time in order to do its job, and it’s been left hanging. This is where the container helps out.
Let’s define that dependency in the container and pass the container to RequestHandler to resolve it.
Remember earlier when I made a point of mentioning the exit statement sitting in HelloWorld?
Well that’s a quick and dirty way to make sure we get a simple response while we’re building things out, but it’s not the best way to send output to the browser. Such a crude technique gives HelloWorld the additional job of responding—which should really be the responsibility of another class—it overcomplicates sending proper headers and status codes, and it shuts down the app without giving the middleware that comes afterHelloWorld a chance to run.
Remember, each middleware has the opportunity to modify Request on its way into our app and (in reverse order) modify the response on its way out of the app. In addition to the common interface for Request, PSR-7 also defines the structure for another HTTP message that will help us out on the back half of that cycle: Response. (If you want to really get into the nuts and bolts, read all about HTTP messages and what makes PSR-7 Request and Response standards so great.)
If you reload the page now though, you’ll get a blank screen. Our app is returning a proper Response object from the middleware dispatcher, but then… what?
Doing nothing with it, that’s what.
We need one more tool to wrap things up: an emitter. An emitter sits between your app and the web server (Apache, nginx, etc.) that will send your response to the client that initiated the request. It essentially takes the Response object and translates it into instructions that a server API can understand.
Reload your browser, and we’re back in business! And this time with a far more robust way of handling responses.
Line 15 in the code example above is where the request/response cycle ends in our app and the web server takes over.
Note that for the sake of the example, the emitter configuration we’re using here is very straightforward. Though it can be a bit more complex, a real app should be configured to automatically use a streaming emitter for large downloads. The documentation for Laminas’s HTTP Request Handler Runner, a combo PSR-15 dispatcher and PSR-7 emitter, discusses one interesting way to accomplish this.
So there you have it. With just 44 lines and the help of several widely-used, thoroughly-tested, reliably interoperable components, we have the bootstrap for a modern PHP application. It’s compliant with PSR-4, PSR-7, PSR-11, and PSR-15, which means you can use your pick of any of a broad range of vendors’ implementations of those standards for your HTTP messages, DI container, middleware, and middleware dispatcher.
We dove deep into some of the technology and reasoning behind our decisions, but I hope you can see just how simple it is to bootstrap a new application without the cruft of a framework. Perhaps more importantly, I hope you’re better prepared to work these techniques into an existing application when the need arises.
If you were to use this example code in production, you’d probably want to break the routes and container definitions out into their own files so that they’re easier to maintain as the complexity grows. I’d also recommend implementing EmitterStack for smart handling of file downloads and other large responses.
Updated on March 24, 2019: Since originally publishing this post, I’ve been persuaded to rethink my stance on autowiring. The benefits are undeniable:
Development is generally faster, and with a much lower “cost” to adding new dependencies to a class, there’s one less argument against breaking up responsibilities into separate classes.
Autowiring generally only works with type-hinted parameters, which naturally discourages primitive obsession.
Comprehensibility is improved because container configuration cruft is gone, leaving you with only meaningful configuration.
As with any “magic”, autowiring should be carefully considered before enabling, but modern DI containers handle it quite sensibly. For example, PHP-DI only autowires concrete classes, so there’s no possibility of ambiguity, and Symfony’s service container will throw an unavoidable exception in dev environments if there’s any ambiguity about which dependency should be provided.
Updated on Dec 31, 2022:Zend was renamed Laminas some time back, so the tutorial now uses the equivalent Laminas packages instead.
]]>Kevin SmithVision and Hard Work2017-09-26T03:49:00+00:002017-09-26T03:49:00+00:00https://kevinsmith.io/vision-and-hard-workI love this contrasting pair of quotes, perfectly capturing the mindset of great leaders.
Vision without execution is just hallucination.
– often attributed to Thomas Edison, Henry Ford, and Albert Einstein
]]>Kevin SmithMultitasking Isn’t Really a Thing2017-09-13T13:37:00+00:002017-09-13T13:37:00+00:00https://kevinsmith.io/multitasking-isnt-really-a-thingAccording to psychologist Mihaly Csikszentmihalyi, the human nervous system can only process about 110 bits of information per second. Understanding someone speaking to you requires about 60 bits per second. That’s why we can’t listen to more than one person at a time.
If you’re feeling overwhelmed and like you’re not moving the needle on anything, it’s probably because you’re trying to do too much at once.
Start focusing on one thing at a time, get it done and put away, and then move on to the next thing.
]]>Kevin SmithA Generation That Never Asks “Why?”2017-08-25T16:43:00+00:002017-08-25T16:43:00+00:00https://kevinsmith.io/a-generation-that-never-asks-whyFrom Solitude and Leadership:
We have a crisis of leadership in America because our overwhelming power and wealth, earned under earlier generations of leaders, made us complacent, and for too long we have been training leaders who only know how to keep the routine going. Who can answer questions, but don’t know how to ask them. Who can fulfill goals, but don’t know how to set them. Who think about how to get things done, but not whether they’re worth doing in the first place… What we don’t have are leaders.
– William Deresiewicz
]]>Kevin SmithDeadlines are Meaningless2017-08-17T11:38:00+00:002017-08-17T11:38:00+00:00https://kevinsmith.io/deadlines-are-meaninglessIf your company is like the typical digital agency or software shop, you’ve got deadlines slapped on every bit of work in the entire organization as a way to get your people to deliver work quickly.
If you’re honest with yourself, you know that your people will miss their deadlines on plenty—if not most—of the work they do. And while you might crack the whip and fuss about it, you know deep down that it’s expected. You’re upset about it, but that’s just how things are. How else could they be?
The trouble is this: your people know it’s expected too. The deadlines don’t really work as an incentive, and the side-effect is constant, corrosive stress. You never intended to run a burn-out shop, but here we are.
But hey, that’s all just part of doing creative work. How else could it be?
So when something really does have a deadline, like a t-shirt design for a trade show giveaway, don’t be shocked when your people don’t take those deadlines very seriously. You’ve been training your people not to take them seriously all along.
How do you make deadlines meaningful again? Use them sparingly. When the outcome of your work has zero value if it’s delivered after a given date, that’s work that needs a deadline.
For everything else, there is no deadline. There never was.
]]>Kevin SmithIs Your Team Feeling Scattered and Unfocused?2017-08-10T22:08:00+00:002017-08-10T22:08:00+00:00https://kevinsmith.io/is-your-team-feeling-scattered-and-unfocusedIn the world of design and development, your team’s getting a lot thrown at it. It’s true if you’re running a software shop responsible for churning out a single product, and it’s especially problematic for digital agencies with multiple clients. But when everything’s an emergency, nothing is. We need a better way of keeping on top of all those demands and knowing which ones we should focus on right now.
I’ve been through the ringer with project management systems: notes on loose-leaf paper, to-do apps, purpose-built tools like Basecamp, complicated Scrum tools, JIRA. The old adage I keep coming back to is this: Keep it Simple. The harder your system is to work with, the less likely it is to actually be helpful.
The method I’ve found that works time and again—for individuals and teams alike—is called kanban. At its core, it’s about simply visualizing each bit of work as a card up on a board. (Think of a Post-it® Note on a whiteboard.) If you’ve ever toyed around with a digital tool like Trello, you’re already familiar with the idea though you may not understand just how powerful it is.
Language doesn’t come naturally to us humans. It must be taught and learned, and it requires higher-level brain power to comprehend. Understanding visual elements, on the other hand, is wired into the deepest parts of who we are as a species. Had our ancestors not been able to scan a landscape and immediately identify a threat (i.e. what must be focused on vs. what can be ignored), they would never have survived.
Some research suggests our brains process visual information as much as 60,000 times faster than text. That makes sense, when you think about it. We can process visual information all at once, but text-based information must be consumed and processed in a linear fashion.
Getting started is simple:
Get a whiteboard (or a digital equivalent like Trello or LeanKit).
Create 3 columns: To Do, Doing, and Done.
Make a card for each current and planned item of work, and put the card in the appropriate column. (Don’t get bogged down in defining what constitutes an item of work. You can improve on that later. Just get started for now.)
Prioritize the work by moving the cards and putting the most pressing at the top.
As you’re doing the work represented on those cards, move them across the board through the appropriate columns.
What’s so special about this? And how is this better than just a checklist?
A checklist is great for the grocery store, but it falls flat for ongoing work because your brain has to read all the items and actually think, “Is this something I’m working on right now or not?” You can’t see what’s currently on your plate, only what “not done”. It’s an unending list of everything you haven’t accomplished yet.
Whew, talk about stressful!
The steps above aren’t magic. They’re simple, but they include most of the elements a human needs to get a quick sense of what’s going on and feel great about what they’ve accomplished. (There’s something else you can add to the mix as you improve. I’ll cover that in an upcoming post.)
The key takeaway is this: don’t worry about anything that’s not in the Doing column. That column is your team’s job. Everything in the To Do column is for the future, but the Doing column? That’s right now. Focus all your efforts on moving those cards into the Done column. Then kick back and enjoy the sweet satisfaction of getting something done.
We’re not robots, after all. Rewards matter a lot for job satisfaction and ongoing motivation.
This isn’t a way of working that requires you to perfectly understand it before you can implement it. It’s not about some sophisticated process. Just get started with where you are right now. Put your current and planned work on the board. Don’t worry about trying to redesign how your team works just yet. That’ll all come later.
Just put the work on the board, and encourage your team to focus on getting Doing cards to Done.
]]>Kevin SmithRescuing a Slow, Buggy, and Frustrating “Store Locator”2016-05-18T19:13:00+00:002016-05-18T19:13:00+00:00https://kevinsmith.io/rescuing-a-slow-buggy-and-frustrating-store-locatorFor any brick-and-mortar business, it’s a major problem if your customers can’t find you.
That’s true for medical professionals and their patients, too. I’m delighted that I got the opportunity to help Connect Hearing, a major network of hearing clinics in Canada and the United States. They recently asked me to fix some big issues plaguing their hearing clinic locator, namely:
The locator itself took upwards of 10 seconds to load and routinely got the visitor’s location wrong during geolocation auto-detection
Individual clinic pages would often show the wrong location on the map or were zoomed in so comically close that the map lost all context
Map pins were a crowded mess in the areas where several clinics were clustered close to each other
Custom location pins were fuzzy on Retina devices and didn’t point to the right spot on the map
Visitors navigating the page with a scroll wheel would get caught in the map’s zoom feature
The user experience was pretty bad. In short, there was too much getting in the way of what every visitor wanted to know: where are the nearest clinics, when are they open, and how can I get there?
Here’s a tour of just a few of the visual issues they were dealing with:
A potential patient searches for clinics near Chicago and is shown this crowded bunch of pins. This is not helpful.
This is from a single clinic page. With it zoomed so far in, why even have a map?
“Make it better” was an obvious desire, sure. But for any project to be counted a success, it must have specific and measurable outcomes marked down from the beginning. It must have goals.
I worked with Connect Hearing to combine user frustrations with the client’s business needs to come up with several project goals.
The main clinic locator page needs to load fast – 2 seconds at the most
It should unobtrusively auto-detect a visitor’s current location
It should automatically display the 5 to 6 clinics closest to the user on the map and as info tiles
Clinic locations should be accurately displayed in any context
Map styles and custom pins should be sharp on Retina screens and match the company brand
Maps should not hijack the scroll wheel
Individual clinic pages should expose data like contact info, hours, location, and business type to Google and other search engines so third-party business listings are always accurate
Clinic info should be added and updated through the site’s CMS, ExpressionEngine – no turnkey, third-party store locator services
The best course of action was to rebuild the entire locator experience from scratch, and I took special care to make sure the new maps would not only look great but that the logic behind it all would support a seamless user experience, something that was even more important to Connect Hearing since their target market tends toward an aging population.
Thanks to smart caching and data modeling, the clinic locator and individual clinic pages are now blazing fast. The locator quickly determines the user’s general location and does so while the rest of the map continues to load, keeping things feeling snappy for the user.
All location data is now represented accurately on the maps with attractive, on-brand, high-resolution graphics, and the locator automatically clusters locations that would otherwise be crowded together on the map. And clinic info is encoded into each clinic page to make it easy for Google to import.
The visual highlights:
Much-improved visitor location auto-detection, sensible marker clustering, and nearest-first listing of the closest clinics.
On-brand custom markers and map styles with friendly info boxes showing the clinics in Winnipeg.
Major improvement on searching for clinics in Chicago.
]]>Kevin Smith“Putting your nose to the grindstone is a really easy way to cover up for an unhealthy business.”2016-04-29T19:28:00+00:002016-04-29T19:28:00+00:00https://kevinsmith.io/putting-your-nose-to-the-grindstone-is-a-really-easy-way-to-cover-up-for-an-unhealthy-businessMy first foray into entrepreneurship was surprisingly successful. At least it started that way.
My business partner and I hung our shingle as Hearsay in 2008 (now Hearsay Interactive), and we quickly began scheduling one project after another, making enough to live on almost immediately. And it felt really good for a while. The problem came when a huge flat-rate project dragged on four months longer than expected and we had no margins—in terms of time or money—to absorb the blow. When the dust settled, we looked up to see nothing on the books and a few hundred dollars left in our account. By this time it was 2010, and businesses were still reeling from the Great Recession. No one wanted a new site. Everyone was just barely holding on. We were stuck, so we swallowed our pride and started applying for anything that offered a paycheck.
In hindsight, of course, we had gotten lucky early on. The initial success blinded us to our ignorance about the business side of things. Turns out my business degree had been preparing me for a management position in the corporate world, but it hadn’t done much to get me ready to start and run a business of my own. My partner wasn’t in any better position, and eventually he sold me his shares of the business for a pittance and moved on.
I kept the company and took on side projects when they came along while I worked retail jobs and whatnot for a few years. As soon as my finances and connections looked promising, I jumped back into the solo development world again.
I’ve been at it almost a year and a half now, and this time around I’m determined not to make the same short-sighted mistakes. I’ve taken my licks and learned my lessons. I’m now seeking out advice from others who have seen success in similar areas, reading business books that come highly recommended, and staying flexible enough to change when a strategy isn’t working.
So I was delighted to see several fellow small business owners recently discussing a book called Profit First by Mike Michalowicz. The author’s way of handling business finances promised to address one of the biggest issues I faced 6 years ago: the seemingly never-ending feast or famine revenue cycle. He also talked about measuring the health of the business by profitability rather than busyness, which was very appealing. I bought a copy from the iBooks store that night and began digging in.
I started implementing the strategy as soon as possible, and I must say I’m very happy with the results. Finally, my business has a solid plan for paying me, saving for taxes, and for the first time, paying profit distributions. I’m no longer flying by the seat of my pants!
I’ve got a couple problems with the book, chiefly this: there’s simply not enough substance here to fill a book. What follows is part book review/part system summary. I think the system itself, distilled down to its fundamentals, could be helpful for some of my fellow entrepreneurs. I hope to do some of distillation here.
There are a few core concepts that form a very helpful strategy for structuring business finances, but it could’ve been covered in a single blog post. One could argue that some storytelling and positioning are necessary to really sell the system, but even then, a blog series or a small ebook would’ve been more than enough.
But a 191 page hardback? No sir. Less is more, especially in this case.
The book is rife with meandering off-topic stories, strained analogies, and awkward, inappropriate jokes that make it impossible to recommend to a fellow professional. Like this one:
If you don’t know your exact numbers, your mind goes wild and says crazy things (like. . . “Ahh. . . I am going broke. . . ahh. . . the only thing worse is Mike dressed up like Scarlett O’Hara. Ahh. . . what the hell am I thinking? Ahh!”). You can’t change what you don’t acknowledge, so you need to know exactly what you’re dealing with.
– Profit First by Mike Michalowicz, Chapter 6
And that example was tame. A good editor helps a business book like this stay focused. I got the impression several times while reading that this book was self-edited and self-published.
At its core, the system is a good one. My problem is the long and unfocused book. Even still, he has some gems in there; like the quote from his introduction that I used as the title for this blog post. It perfectly captures the predicament of a lot of failing small businesses.
The thesis of the book is this: the typical business model of Sales - Expenses = Profit is wrong, and it leads you to make all those bad financial decisions you’ve been making. The supporting argument is that as sales increase, expenses inevitably increase because the money’s there and hey, why not. If you don’t intentionally set money aside for profit, you’ll never have it.
No arguments there.
Michalowicz recommends flipping that equation on its head: Sales - Profit = Expenses.
It makes sense, though it seems like his equation is overly simplified. That’s because as you plod through the book, you’ll discover there are some expenses that are more important than other expenses. The equation should really look more like this:
As sales come in, take a certain percentage out for profit, then taxes, and then owner’s pay. Whatever’s left is all you have available for operating expenses, so you’d better make sure you’ve cut the fat. Pretty straightforward.
For some reason, Michalowicz spends an whole chapter laboriously stepping through what I can only imagine is the unabridged thought process that lead him to the realization that a business should use percentages to divvy up incoming money. It’s helpful to have the structure he eventually arrives at, but the complicated spreadsheets and mind-bending explanations in the book just aren’t necessary. It essentially comes down to these guidelines:
Set aside a small percentage for Profit, starting at only 1% or 2%. Cautiously and modestly increase the percentage over time.
Find the percentage of last year’s taxes paid compared to overall revenue. Set aside that percentage for Taxes. This should include the owners’ personal taxes if the business is set up as an entity with pass-through taxation.
Use the percentage of overall revenue that went to owners last year as a basis to determine the percentage to set aside for Owner’s Pay.
The remainder is for Operating Expenses.
If your business cannot stay afloat on what remains for Operating Expenses, you’ll have to drastically cut costs.
You could use complicated spreadsheets or accounting programs to try to keep track of all those guidelines, but it’s a lot easier to just set up separate bank accounts for them. Here again Michalowicz gets overly complicated, but I sorted through it all and got the essentials:
Keep your existing checking account as your Operating Expenses account.
Open up a new checking account at the same bank for Owner’s Pay. You’ll need check writing privileges and/or bill pay to pay yourself and any other owners, but don’t get a debit card with it. You’ll be too tempted to use it if you’re ever in a crunch.
Open up 2 savings accounts at a different bank for Profit and Taxes. You want to make it difficult to get this money, so don’t allow yourself a checkbook, debit card, etc. If it’s too easy, you’ll inevitably use it when you’re short in Operating Expenses, and you need to know from the outset that you won’t be able to do this. I set it up so that I can deposit checks through their mobile app but I’ll have to physically go to the bank to make a withdrawal in the form of a cashier’s check.
If your bank’s online interface allows giving your accounts nicknames, you might want to give them nicknames like “Taxes [20%]” as a helpful reminder of the percentage you’ll be putting in that account.
All your revenue should be deposited to the Operating Expenses account. Obviously moving money from that account to all the others every time you get a deposit would maddening and completely unsustainable, so Michalowicz recommends doing it every few weeks: on the 10th and 25th of every month. Just total up all deposits since the last time, split it up by the designated percentages, and move it to the appropriate accounts.
For some reason, the author felt the above explanation was worthy of the lion’s share of a chapter.
When the 10th of the month in a new quarter arrives, the very first thing you should do is take your profit distribution, pay your taxes, and re-evaluate.
Withdraw 50% of the balance in the Profit account, and split it up amongst the owners. Do something unique with it: enjoy the fruits of owning a successful business. Whatever you do, don’t put it back into the company. If you find yourself forced to do that with your personal distributions, it’s a signal there’s something rotten elsewhere in your business model.
Leave the remaining 50% in the Profit account as a rainy day fund for the company.
Withdraw from the Taxes account however much is necessary to pay the owners’ quarterly taxes.
Examine your current percentages and make sure they’re still working for the company. Adjust if necessary.
The above is exactly what I implemented for my company, and it’s working really well for me. Above all, I learned why it never felt like I was making financial progress with my business even though I was making enough to live on.
I wrote this post in part because I believe the system could be very helpful to a lot of folks in my position, but for a variety of professional and stylistic reasons, I just can’t recommend the book itself.
While it seemed to take forever for the author to get to his main idea, the system was fully explained by the time I reached the middle of the book. What remained were more convoluted tips and tricks and oddly placed motivational stories. To my mind, there’s really no reason to read past chapter 5.
Do yourself a favor and take a hard look at how your company is dealing with its finances. Use the above guidelines if they’re helpful. Make sure you’re billing more than just enough to get by. There’s nothing quite so empowering as having enough money to turn down work, so sock it away.
