Tag: Work

Hertz doubled the cost of my car rental. Here’s their process design trick.

This year, during my family holiday, I discovered how Hertz – and many other car hire companies – have crafted a process involving dark patterns to trick stressed travellers into paying far more than necessary for their car hire.

I’ve used Hertz several times over the years, particularly in the 2000s, so I expected good service, a decent car, and straightforward business.

Let’s start with the booking process. It was perfectly fine and straightforward online. The price seemed reasonable, and as the holiday approached, I was actually looking forward to seeing what car I would receive. I even found a potential car upgrade at a slightly lower price when I went back to check some details. Happy days, I thought.

Car collection was quick – no big queue – and the chap at the counter was affable. I was juggling kids and feeling the usual airport stress when he said, “You can have your wife as an additional driver for free, and you don’t have to worry about the excess.” I thought, “Oh wow, that’s great! Romana, do you have your licence?”

He handed me the rental slip, I signed, and at no point did I realise what had just happened.

Great, I thought.

The car, however, was a bit of a disappointment – a somewhat battered Jeep Compass plug-in hybrid. Not ideal when staying at a caravan site. But I coped, thinking it wasn’t worth the hassle to switch cars. Then, midway through the holiday, the engine check light came on. I called Hertz, and they said to drop off the car for a replacement. Brilliant. Except the woman at the counter seemed grumpy about it.

No worries. New car acquired, off we went. It was a Mini Countryman Cooper. Worse spec, but a much better car.

But this isn’t a car review. It’s about how Hertz deliberately created a system that, if you’re not paying attention, can significantly boost their profits.

By my estimation, a rental like mine would typically generate about €40 in profit for Hertz. But through clever design, they turned that into about €450 by breaking the usual patterns.

Step one: when the customer is stressed, offer things that sound helpful but are actually upselling excess waivers and other add-ons.

Step two: sneak in a Fuel Purchase option for €73. I brimmed the car right next to the airport when I returned it, but I won’t see that money back either.

Step three: hand the customer a docket with tiny print (which my ageing eyes now struggle to read without glasses), and ask them to sign here, here, and here – all while chatting away – and with no workflow difference when accepting or rejecting. €430 added there also.

A Hertz car rental slip full of small print.
My rental slip that caught me out.

So, because I was busy with the kids, wife, and general airport stress, I signed without paying proper attention. I never usually buy these optional extras.

With taxes, it was just a smidge under €600 in total.

But this time, I accepted the charge. I think I was tricked into it, but there’s nothing I can do. I contacted Hertz, complained, but was told, “No, you signed, you pay.”

And it’s a brilliantly designed system. When you decline, you sign. When you accept, you sign. You get used to signing every time. It’s easy to slip in an acceptance instead of a decline, and then you’re stuck.

So a rental agreement that was supposed to be under €500 turned into €1,133.35 once taxes were accounted for. I’m gutted. Money is already tight this year.

Well done, Hertz and your desk chap. You tricked me out of a substantial amount of money. You win. You’ve increase your profit on this rental by a factor of 10, so even if I don’t use you again for a decade, you’re still ahead. I rarely get tricked, but you managed it.

All I can do now is what I usually do when life hands me lemons—make lemonade, vent on my blog, and move on. I’ve paid my “stupid tax” for 2024. It won’t happen again.

It’s a cleverly designed process they’ve created, clearly aimed at getting people to pay extra at the desk. And it’s not just Hertz that does this—I just expected better from them. They’re supposed to be one of the better firms.

If you were designing the process with the customer not making mistakes in mind, you’d make them specifically write accepted.

The ultimate test as to whether this is sneaky is whether people ever find themselves doing the opposite, and not taking out such cover and extras when they do, in fact, want them.

Taller buildings let us design better towns

I lived in a range of places as a kid, partly because my father was a bit of an itinerant who didn’t know what he wanted in life, other than that I mustn’t live with my mother. Go figure.

Eventually I got to settle down with my grandmother, but in the process I learned a lot about life as a child in different places. Where I felt safe and where I did not.

I did not feel safe in large council estates surrounding cities. I did feel safe in a caravan park. I did not feel safe in a city centre. I did feel safe in a built up part of a large city, living in an apartment block.

Right now I live in a mid-sized house, with a decent garden, in the town of Widnes. It’s nice. I genuinely think Widnes is a very lovely place to live in. Where I am, I have easy access to nature, pleasant walks, parks, a GP, a train station, shops and am not too far from a major hospital. It’s getting close to the ideal 15 minute neighbourhood and I’d say it’s one of the better planned areas I’ve lived in. It’s higher density than some old neighbourhoods I’ve been in, but way lower than others.

But it’s not sustainable. It was built on countryside. Soon more houses will be built. These houses will be between 90m² and 150m². Not that big, really, and they’ll take a fair chunk of land up.

Meanwhile, I can think back to a place that I really enjoyed living in as a kid in Spain. It was rented by my father, so it wasn’t unaffordable, and it was bigger and nicer than any house I lived in here in Britain. It was clean, I didn’t suffer asthma there… almost perfect. And within a short walk we had bars, restaurants, a bodega, and sports facilities. I loved it. I could play safely in the playgrounds with friends, and even went to school there as we had a primary school on site!

Here’s a snip from Google Maps.

As you can see it’s a big old building. I estimate there are between 400 and 500 apartments, varying in size from approximately the three bedroom 90m² apartment we lived in and the larger end apartments which I believe are about 140m².

The caravan I lived in with my grandmother can be seen here although in other pictures it seems it’s been knocked down:

Let’s have a think about housing density here, however.

The caravan park I lived on, including internal roads but not the road to the caravan park used about 320m² per home. Some were more densely packed, some more loosely, but it’s a fair approximation. My current house sits on about the same, funnily enough. The apartment block, however, uses about 100m² of land per home and includes playgrounds, two swimming pools, three tennis courts, a basketball court, shops, bars and more! Use the slider below. Both images are at exactly the same zoom level, and see how 400+ homes compares with about 200 homes for facilities and space use. Our towns could be half the size they are, and rammed with leisure facilities, all at lower cost.

Halsnead Park in Whiston with approximately 210 homes Club del Mar in Alicante with approximately 400-500 homes

Why don’t many British people live in apartments then?

Trust.

That’s it. One word. Due to systemic issues in how the UK treats and manages high density housing, we’ve ended up with a situation where there is very little trust in large apartment buildings.

We have miserable stories like The Decks in Runcorn, where residents have been waiting a decade for resolution and have felt locked in their unsellable apartments.

Historically there have been a number of disasters often down to poor design of buildings so that if something bad did happen, people died. Grenfell is the classic disaster.

By Natalie Oxford – https://twitter.com/Natalie_Oxford/status/874835244989513729/photo/1, CC BY 4.0, https://commons.wikimedia.org/w/index.php?curid=59913134

You had this tower block with a single stairway, no decent quality fire extinguishing system built in, flammable cladding. So many people were let down by the way this country treats high density housing. It was a disaster.

Meanwhile, when I visited family in Poland I noticed there were three stairways in the apartment block – the central one and one at each end of the building. Each corridor also had fire doors at each end, with security access, making it safe but also giving people plenty of options for escape. At the bottom you had a playground, sports facilities, and nearby a park and woods along with good access to public transport and very close shops.

High density housing makes living easier and nicer. You don’t have a garden to tend constantly, and you get a decent amount of space for the money because you’re not handing over a pile of money to wealthy landowners.

But we can’t get from where we are to where we should be without addressing these systemic issues. Meanwhile, we can’t be denying young families genuinely affordable, quality housing. So that means we keep building on land that really shouldn’t be built on. Because we’re scared of towers. And I understand why.

We can build better neighbourhoods, with better facilities, and better lifestyles. It’s possible. I’ve seen how it’s done in other countries. Having family in Spain and Poland, and friends in many other countries has taught me a lot. Being poor up to the age of about 25 has also taught me that we can’t foist middle class solutions to working class problems either. That got us the Southgate estate. It stank of piss.

Don’t design in customer traps on your systems

You know that friend, the one who always promises to help you move flat, or help you fix your mower, but then doesn’t turn up? Yes. Or the guy who goes on a date and pays for the meal and somehow that comes with access to your body? Or the airline that makes you think you have to pay for a seat upgrade in order to take any baggage on board when actually you don’t?

The first two are obvious red flags. We all know people like this, and once we’re done excusing their behaviour we tend to move on. They’re often charming people at first. They have to be. They keep needing new friends and new partners, so they get really good at the introductory gab. But as I always say, the only perfect people in this world are people you don’t know very well yet.

To most of you I’m a suave, sophisticated man of the world. What? No? OK, well at least if you don’t know me I’m probably fairly mid. If I was on a charm offensive you might find me quite nice. Yet I snore, I go off on overly long rants about how a £70,000 car can have a misaligned number plate, and I walk into garage doors that I didn’t bother to put up all the way.

And so it is with companies. They’re often amazing at the introductory gabble. They’re smooth, funny on social media, and charming in their promises. But so many will let you down. Sometimes that’s your own fault. In a way, I kind of admire the Ryanair model because you actually know they’re going to play games and it’s out in the open. It’s a game! It’s a bit like that girl who you know is a heartbreaker but you’re gonna date her anyway because you’ll have fun for a while. Ryanair will be cheap. But sometimes you’re dealing with a business that should be well behaved, is very much business to business, and you’re thrown a curve ball.

Jabra’s survey

I have a Jabra headset I use for all my work meetings and honestly, it’s absolutely lovely. It’s clear, has good sound, and an excellent microphone. It’s also light enough that I can wear it all day long.

I also have the Jabra app, which is a bit annoying in needing frequent updates for something that does almost nothing. So when I saw a survey I thought I could give feedback on this, as well as the headset, so that they can improve.

I worked my way through the survey. It wasn’t especially long, but at the end, I couldn’t submit, because of this:

As you can see, the bit that says “I agree to terms and conditions and that Jabra, its parent company company can contact me by email or advertising…” is a mandatory field in the feedback form. So I’ve been asked to invest time in giving feedback yet here we go… in order to give that feedback there’s a price!

It’s like the Hotel California – it’s a classic case of social entrapment and a breach of the natural human social contract. Akin to being asked some questions at the supermarket then the surveyor saying “but if you don’t agree to let me stalk you, I’m throwing away everything you just told me.”

So I changed my address to one that included a tag* so that I could identify it as being from Jabra and submitted anyway. Because they need to know.

And you could say no sensible designer would build a survey like this. But it doesn’t matter. The survey was designed, even if by a person without the job title of “designer”. They were either careless, or cynical. I can’t be sure of which. And yes, my form entry was harsh. I was annoyed.

What do you think? Is it fair for companies to do this? Have you seen other examples that you’d like to share? Comments are open!

* many email platforms let you format an email with a + such as david+jabra@example.com, meaning you can see where your email went and block it easily if you wish to.

A new manifesto for the web

This blog post has now been re-arranged with the manifesto at the top, and the reasoning that led up to it beneath. Because, after all, placing the important content six or seven hundred words in is hardly being pure, is it?

Web Dogma 24. By me.

Content First. All articles, images, and graphics must be there to serve a purpose. Superficial or filler material is forbidden.

Real-world Photography. Photographs should not be bought from stock suppliers, but either provided by the subjects, or taken by a suitable photographer using only natural and available lighting and real life settings.

No AI. AI generated content is highly derivative, prone to errors, and could get you in trouble. It must not be used on any serious website to create content.

Simplicity in Design. Layouts must be clean and simple, with a focus on readability and user experience. Decorative elements that fail to enhance content comprehension are not permitted.

Environmental Care. A website should be as efficient and fast as possible, minimising the economic impact of running it. Where real life products are created, they should be on recycled paper using biodegradable inks.

No Sacred Cows. There are no firms, people, suppliers or institutions who cannot be investigated or criticised in the event that they behave badly or produce poor work.

Transparent Authorship. All articles and images must be clearly credited. Ghost-writing and anonymous contributions are forbidden.

No Advertisements. The website must be free of advertisements. Revenue should be generated through subscriptions, donations and grants.

Timelessness. Content should focus on timeless principles of the website’s subject matter, avoiding trendy or ephemeral topics.

Interactive and Inclusive. The website should encourage reader interaction through letters, feedback, and community events, fostering an inclusive community.

The story behind this

AI is making it much easier to produce content on the web. I could literally ask it to write this article and it’d do a half decent job of it. It might, of course, be a somewhat plagiarised equivalent, because of the way large language models work, but it would be an article.

They’re amazing. They’re also inauthentic and only relate to the past.

Search engines are getting in on this too – taking the content of websites and digesting it in order to provide what are AI generated responses, rather than sending you to the right websites:

Of course, Google created the problem, and is now offering a solution to the problem it created. How so? Let’s take a look at recipe websites. You’ve got a vegan visitor and you want a recipe for, say, homemade vegan sausages that everyone can enjoy. You’ve never made them before, so you search Google.

And you get not a personal blogger who’s landed on such a great recipe they wanted to share it, but a professional and well made site which isn’t a pleasure to use: https://elavegan.com/vegan-sausage-recipe/

If you don’t run an adblocker, the site is clearly set up to serve ads. Which in many ways, is fair enough – they need to make money, and Google provided this easy monetisation channel. By the time you get to the recipe itself you will see, get this, around twenty-two advertisements! And these sometimes rotate, so it could easily be more and rarely fewer. Meanwhile, I can hear the fan on my laptop starting to spool up as it handles rendering the video advertisements on a moving page.

And people are so, so tired of this. The reason they love TikTok and a lot of the newer social channels that come along is because, frankly, it often feels more authentic. It feels like real people giving advice and having opinions. It’s nice, honestly, although of course TikTok needs to make money too, so they do their thing. At least it’s easy to flick past their ads. People are getting their links inside WhatsApp groups. Emails are, incredibly, becoming a channel once more.

So to help people navigate past the advert dross, Google are, instead, going to serve up AI generated renderings. In fact, Bing are ahead of the curve here. Take a look at this block that you get on a Bing search for vegan sausage recipes:

They’re taking away the need to visit a site at all! Which is, I think, terrifying for sites that have entirely generated their revenue through an overdose of display ads.

But it’s hardly authentic either – and people have a deep distrust of AIs because they’re already noticing they can be really dumb or easily misled.

People have been finding old school search terms less useful and, instead, have been appending site names in order to help get them to more useful content. In fact, it is quite normal to see search terms appended with Reddit, or, in the case of recipes, “BBC” is quite a common suffix. Why? Because we know the purpose of these sites is less to sell a lot of adverts, and more to inform.

People crave authenticity

Designing to hold a reader on your website in order to maximise the value of the reader’s time to advertisers is grating. We all know website publishers have to make money. We all know that it’s hard to do so. Which is why I find myself now subscribing not to the big local newspaper, but to a small, independent one that sometimes scrimps on presentation, but does get deep down into real stories. It doesn’t tell me which tube dress is suddenly selling well in Primark, however.

I don’t think people would even mind some promotional work, but again, only if it’s honest and blended with other genuine, non paid content, just so long as it’s well advertised that it is indeed a paid piece.

The craving for authenticity isn’t just a problem for websites. It’s created new trends in videos, podcasts and reels. This video is a great deep dive into this phenomenon on YouTube:

Of course, to get to where he is, Tom Nicholas above has had to distribute his videos on the YouTube walled garden. Which is favoured by Google’s search engine. Google owns YouTube.

Google in effect has an enormous effect on how the web works, and even how it is designed. But true success now can’t depend on just getting search engine or social media generated traffic. It has to be by being a destination of usefulness for your readers. One so useful, that they feel they’re part of something. Connected to it. Tuned in together.

Which has caused me a lot of navel gazing, and I decided I needed a fundamental set of principles for websites that I think can be the starting point for what is good… And at one point I started to think of Dogme 95 and how influential it was to filmmaking, even if most people haven’t heard of it.

It’s worth noting that Dogme 95 didn’t become the way to make films. Many made in the style were not even particularly successful or critically acclaimed. In large part, because it’s an extreme position that is generally unsustainable. It’s an idea that can season the whole, not the One True Way. So if you head back to the top of this post, you’ll see Web Dogma 24 and that I’m using the same rationale.

But reality will intervene

I don’t think it’s possible to run anything with that sort of purity. Life and the need to make a living means that compromises will have to be sought. But it offers an idea of what’s possible.

What do you think? What would you add or take away? I’d love to hear your thoughts.

AI is predictive, not intelligent

I hope you haven’t missed the big AI revolution going on right now? No? Yes? YES? You have no idea about it? Where do you live? On Mars? Ah OK, you were kidding. AI is being talked about everywhere, new devices like the Humane AI Pin or the Rabbit R1 are very much about it, and everyone involved is scrabbling around like mad for venture capital to fund their incredible ideas.

Having access to these language models is immense, right? We can sit down, whip out our AI, and enjoy a better than life experience. We can ask it questions and often it can come up with remarkably lucid explains. It seems intelligent. Here’s a conversation I’ve just had with ChatGPT’s latest model, after I asked it to settle into a British conversational style:

Me: The weather’s nice isn’t it?
ChatGPT: Oh, absolutely! It’s a lovely day, isn’t it? Perfect for a stroll in the park or perhaps a spot of tea in the garden. How are you planning to enjoy the fine weather?
Me: I have to work, sadly. In fact I’m working right now.
ChatGPT: Ah, that’s a bit of a shame, isn’t it? But needs must, as they say. Hopefully, you can steal a few moments to enjoy the sunshine, even if it’s just a quick breather by the window. Anything I can help you with to make your workday a tad brighter?
Me: Give me a joke to help me get over my vague sense of imposter syndrome!
ChatGPT: Of course! Here’s a bit of light-hearted humour for you:

Why did the scarecrow win an award?

Because he was outstanding in his field!

Hopefully, that brought a smile to your face. Remember, everyone feels like an imposter at times, but you’re doing brilliantly!

It’s a decent conversation, although no British friend I know would chat like that. They’d be less polite and more vaguely abusive, to be honest. But it does seem intelligent. Then I can ask it a tricky question:

Me: I’m struggling with a maths question. I have a formula for a parabola, and I need a graph and the location of the y intercept: f(x)=3x²+7

ChatGPT:

No worries, I’m happy to help with that. Let’s break it down.

The given function is:

To find the y-intercept, we need to determine the value of the function when =0. Plugging =0 into the equation:

(0)=3(0)²+7=7

So, the y-intercept is at (0,7).

Now, let’s plot the graph of the function and visualize it. I’ll create a graph for you.

Graph by ChatGPT

Here you go! The graph of  shows a parabola opening upwards. The y-intercept, marked in red, is at the point (0,7). I hope this clears things up for you!

That’s so freaking clever!

Except… except… it’s not actually clever.

Let’s try asking it a question you can use on a seven year old child who would almost universally get the answer right:

As I was going to St. Ives, I met a man with seven wives who was going to St Ives. How many people are going to St Ives?

I think you’ll agree that this riddle is pretty simple. I’m going to St Ives, the man is going to St Ives, and the seven wives are going to St Ives. Look at this madness I got back:

The riddle goes as follows:

“As I was going to St. Ives, I met a man with seven wives, Each wife had seven sacks, Each sack had seven cats, Each cat had seven kits.”

The key detail in the riddle is the first line: “As I was going to St. Ives.” The person telling the story is the one going to St. Ives. The man with the seven wives and all the others are simply met on the way; the riddle doesn’t specify their destination.

Therefore, only the narrator is confirmed to be going to St. Ives. The answer is:

1 person (the narrator) is going to St. Ives.

And that reveals the weakness of large language models. There’s no inherent cleverness. What it does is to look for what it thinks you’re talking about then infer an answer using a combination of techniques, and it then picks what the most likely question is then creates a response based on what it thinks is correct. If the problem is very well known, it can do some really quite incredible things. But if the problem given to it is a novel variation of an old problem it will absolutely fall apart.

Life is full of novel problems. Don’t be misled by the clever bullshit attitude of a large language model, but at the same time, if you know its limitations, it can be a wonderful tool. Use it wisely and carefully, or there’s a risk it’ll make a fool of you. And yes, the post image at the top is, indeed, AI generated – because I have bugger all budget and this is a personal blog. Bite me!

Design Dilemmas: When bathrooms become battlegrounds

In the world of transgender rights, the battle for the bathroom has reached fever pitch, especially in the USA. Now, I’m not here to referee in this brawl – because on one side of the debate you have the ‘TERFs”, and on the other, the trans rights activists sometimes labelled “handmaidens”. Both are terms about as socially acceptable as weeing all over the toilet seat, and on the extreme fringes of both you find calls for violence. It seems every modern campaign group needs its villains and its sycophantic followers.

Both sides have got themselves worked up about issues that I’m pretty sure most transgender people would prefer to be handled with a little less shouting and little more thought. I have my own opinions, but that’s not for me to share here. What I’m looking at is how design can make the situation better, by picking on one such contentious issue: public bathrooms.

We’ve all got to admit that the concept of women only spaces wasn’t dreamt up at a lovely cocktail party. They often serve as a refuge from the stormy reality of life that many women face. The abused, the desperate, the traumatised who’ve experienced the worst aspects of a male dominated society are, quite rightly, entitled to seek out a safe space away from men.

Meanwhile, imagine you’re a transgender woman and are about to transition.  You’ve got to live life as a woman for at least a year or two before anything permanent happens. It’s an obvious safety precaution so decisions aren’t made in haste.

So out you go on your first night in full femme mode. Nature calls, and you head to the bathrooms. You aim for the ladies, because, well, that’s where you belong now. But for some women in there you might be as welcome as a fox in a chicken coop.

Head into the men’s toilets and many trans women are as vulnerable to the nasty side of men as the traumatised women who want their safe spaces. You don’t feel you belong there.

All you want to do is check your lipstick and adjust the heels.

And there’s a twist here. I’ve met some women who are burly and could beat the majority of men in arm wrestling. They may look pretty masculine too. They’re now sometimes finding themselves being shouted at for heading into the ladies’ bathroom!

About designing loo layouts. The Brits generally build them with two doors going in, and stalls that are pretty private or, as is increasingly common, the toilets are individual spaces with floor to ceiling walls.

Americans, however:

They love a gap! You could pass a handbag through that! Nobody wants to make eye contact with someone taking a dump.

So how about work is done to make sure bathrooms accommodate everyone? Picture a row of stalls which are just little rooms with proper walls. Private, no tinkling sounds being shared. Just a nice, lockable door for each person with a sign saying “Thinking Room”.

We can have scents and nice music for everyone, and ladies can compliment the prospective ladies for their make-up skills. Remember – the more people there are around the more likely they are to hear a shout for help. Predators hide in shadows, after all.

Just a thought. Ultimately then, this isn’t a potty problem. It’s a design problem. And that means we can design our way out of this mess, and make sure everyone is happy. Or are we going to just keep arguing until we’re just too old to care what’s going on in the next door toilet cubicle?

Comments. Please. Just remember the one rule on my site. Be kind.

A little change on this website

In the past, this website was sort of my place to keep family and friends in touch. I first set it up in 2005, just before going away to Peru on travels. It was a great. Facebook had been invented but wasn’t generally available or popular yet, other platforms weren’t well thought through or scattered, Twitter didn’t exist. But installing WordPress on my cheap Yahoo web hosting turned out to be one of the easiest ways to publish content and share it with the world.

But then the day came when I upset some far right people, and I realised some content had to go away. My trust in the world diminished. A man shouted abuse aimed at me at our house one night. Nobody else in the family witnessed this, incredibly – the kids were asleep, my wife was brushing her teeth with an electric toothbrush, and all I did was simply close the bedroom window and ignore the man. No point feeding a fire.

That was the moment when I realised that being hyper online, sharing everything, and also trying to fight to protect people from the malevolent sides of society came at a cost. Before that I’d been campaigning against Brexit, and continued to do so.

A year after that abusive night – I had a heart attack. I document some of that here, and here. I don’t recommend it, but I did realise that for most of my life I lived under a constant strain. As a kid I attended seven or eight (I actually lost count!) schools before I was twelve. I’d lived in multiple countries, and most of that with an abusive, violent and frequently drunk father. Each school was a new challenge – standing out, but then also being reasonably bright, and a bit weird was tough. But I learned about people, what makes them tick, and also that the only thing that really worked once a bully had decided you were a victim was to fight back. You could win a battle with words, or you could win it physically. It worked.

It always worked.

And it has to stop. I’m too old, too tired, too spread thin to fight and, more importantly, to take the damage. Obviously I haven’t been in a physical fight in a very very long time, but verbal fights? Arguments? Online battles with the world? Plenty. And I see many people just quietly standing at the side. They might agree, but they’re not going to fight that battle.

I disagree with ignoring injustice, but I’m also at that stage where I think that fighting small battles is pointless when there’s a broader war going on. So instead, from today, I’m turning my mind to making this blog about the intersections between design, technology, publishing and ethics.

Why? Because this is the stuff I know about these days. That’s all. And I hope I can make it interesting and useful to others.

The astonishing power of modern computing

Being very old (or at least, that’s how I feel being in tech!) means that after coming up to nearly forty years in technology, I’ve seen some changes.

My first computer at home, that I owned, that I could truly call my own, was a Dragon 32. It was a small, 32KB computer using the rather lovely little 6809 processor. This CPU ran at 2MHz and the system as a whole allowed me to learn a heck of a lot about computer science as a geeky teenager who was busy ignoring sports.

It didn’t exactly prepare me for my first coding job, working on an IBM MVS mainframe computer handling the payroll of 70,000 people, and distributing over £1bn a year. In fact, it prepared me terribly, because mainframes back then were similar to desktop computers today – powerful, with a ton of security features, online storage and a lot of data pipelines. But still, I knew various algorithms, knew what compilers were (though I’d only used an assembler really, but it got me half way there) and how to do a lot with a little.

That mainframe, when I started, could do about two hundred megaflops.

And today, in the picture above, I have an M5Stack on my desk. It uses the Xtensa LX7 processor. As you can see from the picture it’s absolutely tiny. And it can handle the integrated digital camera, WiFi, Bluetooth, USB… everything, really. The little CPU can run at low speeds and high speeds as appropriate to workload, and can reach over 1GHz.

Let’s put things into perspective.

The 6809 was addressing 32KB. The S3 Stack has 16MB of Flash and 8MB RAM. So 750 times the onboard storage of my Dragon 32. You can insert a 16GB microSD Card. You could put 200KB onto a C60 cassette tape back then. So that’s 80,000 times the unloadable storage! Running a lot faster too.

In terms of CPU performance, I’m struggling to find comparable benchmarks, but if we look at a 6809 you used to get 2-3 clock cycles per simple instruction – so you’d max out at around one million per second. Complex instructions would be far worse, and moving memory around was slow and expensive. The LX7 seems to get through around 0.8 instructions per clock cycle, so we can see it can max out at over a billion instructions per second. Or at least a couple of thousand times faster than the 6809.

But here’s the thing… the little computer I’m holding in my hand is at least as capable as the  contents of the data centre I started to work at in 1987.

And that, my friends, is why I feel very very old in this sector! Things have changed so much.

Yet at the same time, they haven’t. UX didn’t have a name back then, but careful thought into the design of a screen could save people a huge amount of time or reduce data errors. Saving CPU cycles then meant less of a bill (we paid by CPU time on mainframes) and could save tens of thousands a year for our team. Today, it may feel as if the CPUs are endlessly powerful, but using lower power techniques can dramatically cut down on resource use – meaning less environmental damage. I know at work we’ve taken systems running on 32GB, top spec web servers down to 512MB machines! It’s incredibly satisfying to reduce resource consumption by so much.

Let’s look at it another way. The sophisticated corporate payroll, for 70,000 people that I worked on in the eighties and nineties could easily be run today, if suitably considered, on the tiny little computer I’m holding in my hand here.

Which leads me to the real point of this piece – we as software engineers and designers have a duty to minimise resource usage. Yes, that enormous library you’ve included in your project might save you an hour of time today, but if it increases the memory and power consumption of your system so much that you need loads of cloud computing to support your system then it’s not a job well done – it’s a kludge you can hide. We have an ethical imperative to do better in systems development.

Also, low power computing is awesome and fun. Try it!

The productivity paradox

I came across this chart recently and it gave me pause for thought as someone whose life work is designed to improve productivity. It shows that from 2008, although we know technologies have been growing in power, productivity growth suddenly dropped away from the trendline following the global financial crisis. Why?

This is where I fall back on an old story as a developer trying to make things more efficient. I worked with my colleagues to bring a client company the equivalent of about ten hours a day of labour saving due to using a well structured database for better data integrity over their previous ‘loose’ system. As they expanded, each branch would save at least an hour of time, and they were no longer capacity bound at each branch where more customers would linearly increase administrative work in a difficult recruiting environment. This would provide a return on investment (ROI) of less than a year.

It worked exceptionally well and the person we’d worked with said their staff were even worried redundancies might be on the way! This was never going to be the case in a growing company, because there’s an infinite amount of work to do.

All was good.

Then for some reason, our contact’s role was diminished and they eventually left the company. And the knowledge of the improvement left as well. The board’s innate distrust of anyone and their perception of us as no more than a cost base resulted in a great difficulty for us to help them without dramatically increasing costs by gold-plating every spec and decision.

This left the client unable to gain for advantage by exploiting our skills. It was difficult for us to see, and we could do little for them other to look for cost savings when, in reality, they needed to implement cost savings by using us more. Instead they opted for a recruitment based strategy to feed their growth and we parted ways as they shifted the system to a maintenance partner.

In all my thirty years of development experience I’ve seen this again and again. Instead of sticking with a solid set of developers who have learned your business details, chop and change to save money. Treat developers as a cost base and you lose so much knowledge that you’ll fail to gain productivity nearly as quickly as you could treating it as a partnership.

I don’t feel this is unique. I see lots of projects following an arc of big development then a cost reduction period which ends up losing the momentum of ongoing improvement.

Solving the productivity paradox

A strategy I now suggest when selling is to spend 60% of what you thought of on your project in year 1, then 40% in year two, 30% in year three, and then 20% a year throughout the product lifecycle. Software isn’t like building a bridge where things stabilise then stay them same for thirty years, sadly.

Fundamentally, I think corporate culture around custom software became both cautious and speculative following the global financial crash of 2008 and only invested in technology if it felt like it would create some impressive unicorn of an offering. We saw lots of developers tied up with venture capital chasing big things like blockchain and AI whilst ignoring the importance of daily incremental improvement that adds up to a huge return over time.

The answer then, is that if you have access to capital, you should stop chasing unicorns and start thinking about continuous pragmatic development aimed at dealing with all the little productivity blocks in your firm. Look to Kaizen style methodologies to help with structuring your processes around this.

What do you think? Feel free to comment!

Off-Cloud Backup for Heroku apps – a possible answer

The Heroku platform is an absolutely fantastic way to have to not bother with devops within a small development company. We’ve been using it at interconnect for years now, and whilst it’s not entirely perfect, it takes away one set of headaches and does so at a reasonable cost.

All the services offer backups, and the VMs are built from scripts and are essentially read only. So if something catastrophic happened to one of our databases, we can roll back a day and be OK. Except… let me explain my fears around data.

Trust issues with providers

In our very earliest years we used a VPS provider that used Plesk. Everything was solid and stable until one day, we got a report that a site had been hacked. Then another. It turned out that a vulnerability had exposed our sites to being hacked. And they were. This resulted in a big old clean up operation and restoration from backups. Except the daily backups we’d been paying for turned out to be weekly. So the backups we had were three days old. Ever since then, I’ve preferred to have a way of pulling backups separately to a server under my own control, unless the provider is Kumina, because I know the people so well that I’m 100% certain they’re as paranoid as I am and they’ve never ever let me down. But in the era of hustle culture bros who move fast and break things, you need a safety net.

Creeping corruption

My next fear is corruption you don’t notice immediately. I can well imagine that if all the meta data for the posts on a site before a certain date got wiped out, most people wouldn’t notice for ages. Imagine you’ve got a site with 200,000 posts, and various elements of the first 100,00 were damaged – the long tail matters to these sites and suddenly it’s all gone. Well, thank heavens for backups!

Except, of course, most cloud providers don’t provider substantial generational backups. Instead, they keep a few days or a week or so. And that’s your lot. If you need to go back months you’d better hope a developer in the company left a dump on their laptop somewhere – except of course that very very few developers keep dumps of production systems on their laptops – it’s bad practice and only tends to happen in exceptional circumstances and should be deleted soon after use.

How we fix it today

In the end, I asked one of my Linux oriented colleagues, Gianluigi, to create a service that would connect to Heroku’s API and then download every database, and sync every S3 bucket. It worked, with some limitations. More recently, because he’d left but remains a good friend, he helped me with a crash course in Linux sysadmin basics and I was able to extend and improve some bits. The system is a service written in PHP that does all the work. I then asked another colleague internally, Jack, to extend things to cover the PostgreSQL databases we also now used and to create a dashboard so that I could monitor the backups easily without resorting to logging into the backups servers.

The dashboard also doesn’t run on the backups servers. I needed to keep the backups as safe as possible – they’d be a great honeypot for a hacker, so they’re onioned away, and the backups service isn’t reachable from outside. Instead, it messages the dashboard with information about the backups taken. The dashboard also provides details on application and framework versions, for security monitoring and making sure updates have been applied appropriately, and it also sends me a daily summary email showing me storage space available and what was backed up in the previous 24 hours.

Here are a few screenshots of the system, with some censoring, but I hope you catch how it works from what you see.

To commercialise, or not?

And now to one of the reasons why I’ve decided to write about this. In the past, I created the first version of Search Replace DB – a quick script and algorithm I knocked up to parse a database and search and replace items in it. A fast, dangerous tool that I released as free open source code. Other people took it and commercialised it into successful products. We didn’t. And with the code being integrated into wp-cli and most devs would use that in preference (myself included!), except in those tricky situations where command line access wasn’t possible – mostly on cheap hosts. I think we were right to release the code, but where we failed was in realising the commercial possibilities. And that’s left me a little torn.

So now I’m torn – it’s not easy to set up services in Linux, but once you do, these things just run and run. It’s also not going to be the easiest thing to work with, so I anticipate support costs being quite high. It’s proper server level work. And I certainly don’t feel inclined to build a SaaS that acts as a conduit for people’s backups. It’s just too risky to have a central pool of lots and lots of backups, and people find them lurking on S3 buckets all the time. So I want to put this out to the community. Is this something you’d find useful? Let us know in the comments below. If we did release it, the code would be open source, but access to the latest versions would be restricted.

I’m looking forward to hearing your thoughts!