WordPress Performance, Make it 3x Quicker!

I’d started to notice that my site could often be slow to load – other sites on the same server weren’t suffering the same way, so I wanted to document a simple way in which one can identify performance issues on the site. This is one of them.

A little while ago I reported that my site, since some WordPress upgrades, had started to slow down. I’d wondered whether it was WP becoming increasingly bloated, or some other problem.

Well, it took me a while to get back to the issue (babies and a booming business don’t help!) it’s continued to get worse and worse, until a recent change has improved things… but only marginally, as shown by the Pingdom chart below:

Not looking good…

This is dreadful, really – daily average of 4,000ms responses just aren’t acceptable where, two years ago, I was getting 800ms.

So, now the process starts.  The recent small improvement came after installing our Spectacu.la Advanced Search Plugin, which runs a regular database optimisation to help keep things nippy, but it was still dreadful.

Is it Pluginitis?

My first suspicion is always that of plugins (and sometimes themes, if they’re complex).  In our office we have a term called ‘pluginitis’ which refers to the problem of a site having too many plugins installed, many of which are poorly written.  I hate to say it, but when clients call to ask for a plugin to be installed that we’ve never tested we go through it and, 90% of the time, discover serious performance or security flaws that will cause long-term issues.

And this site here is old – I’ve been running a WP install for four and a half years with nothing more than upgrades and, like an old PC that’s been upgraded too many times, that causes issues with old drivers and code.  Same can apply to WordPress.  So let’s see what we can do to improve things.

First stage is to disable as many plugins as possible so as to isolate the issue.  I’m using a division based approach – ie, I’m going to disable half of my plugins to see what happens.  If I get full performance back, then the problem lies in that half.  I can then reactivate half the plugins and see what happens.  If the performance is still good, the problem is in the other half.  I think you can see where I’m going here.

I’m also going to go for plugins that aren’t written by us. Not because I’m biased (ok, maybe a little) but because I know all of ours are carefully tested for performance – many are run on major sites such as the Telegraphs blogs site.  Speed is of the essence.

I’m also going to skip plugins like Akismet, because anything that’s essentially ‘core’ is usually going to be reasonably performant – at least on a small site like this one.

It’s worth noting that I could easily delve into SQL statements and code efficiency – but that’s only interesting to developers – if you’re simply a WordPress user, performance is interesting but what you can do to find problems is somewhat more limiting.

Plugins being disabled:

Add to feed – a simple plugin, but sometimes simple plugins miss simple tricks.

Headspace2 – I have my suspicions about this plugin as it’s massive. Could be fine, may not be.  Only way definite way to know – measure it.

Search Meter – a nice plugin to see what people are searching for, but is it adding load somewhere?

Social Bookmarks – it shouldn’t cause issues, but you can never be sure.

wp-typography – I love what it does for the typography on the site, but it’s also running a lot of javascript.

First results:

I do use YSlow to test the site, but one of the problems is that it’s hard to get a large enough series of data to be statistically relevant.  It’s good for seeing the extra load (and why I knew the amount of javascript was an issue) but for longer term analysis it’s flawed.

So, we go back to Pingdom and look at the one day chart.  As I type this it’s now an hour since disabling the plugins above – so let’s see what’s happened:

A dramatic improvement!

As you can see, in this afternoon alone there’s been a dramatic improvement – from around 2500ms per visit to 1230ms per visit.  In one single step I’ve halved the load time of the page.

What we don’t know so far is whether that’s because the page got smaller to load or whether it’s down to a reduction in database load – but that’s really for another article.  What this is all about is trying to document how I’m improving the responsiveness of the site in a way which relatively non-technical folk can follow.

What I’ll do in the next feature is to turn off some more plugins to measure the impact they had.  I’ll also be interested to see if the spikeyness of the response times has varied much – are they caused by simple server load, or is there something else at play?

I will then start to switch plugins on again in a structured way in order to measure which was causing the heaviest loads on the site.

Keep watching!

Of WordPress Training and Learning

Over at Interconnect IT, where I work, we’ve found that demand for our WordPress Training Courses has shot up over the past year or so.  And it’s certainly interesting to see where the work comes from.

Curiously, it’s very polarised – we either work with smaller one man band companies like Jason Nevin who runs a house removals company site who are looking to switch technologies, or big corporates and government agencies such as Shop Direct, The US Mission to the UN and the Department for Business, Innovation and Skills.

I’ve been thinking about why there’s such a big dip in the middle of our client base – 90% of our business is groups with over 500 staff or with fewer than three.

The Gap

Small, one man companies tend to involve highly motivated individuals.  They understand the important of their skills, and they know that in the tech sector they have to always stay up to date.  Large corporates tend to understand this too, and have allocated training budgets to make sure they keep up to date.  Government departments are often a little slower with new technology, but they too need to keep efficient or tax payers will kick up a stink.

And WordPress is a very efficient platform for running many informational websites.

But why are medium sized companies not coming to us in nearly such large numbers as micro enterprises and large corporates?  Let’s see…

Budgets

I’ve noticed that medium companies often have people who are extremely good at what they do, but I do find that there are often significant skills gaps.  I’ve dealt with someone from a school (medium) who didn’t realise that you don’t have to close an application in Windows in order to see another application.  Each time she cut and paste she’d open one document, select the text, close the document, then open the receiving application and paste the text there.  Productivity, as you can imagine, was pretty poor.

I think a lot comes from budgets – many firms around the 50 people size aren’t always able to make good money.  They have to be careful where they spend it – they’re not big enough to have training departments who make sure everyone is up to date, and they’re usually busy.

Culture

Another thing with smaller firms, I believe, is that they’re often started by individuals with a lot of skills – these are flexible folk who can teach themselves and learn quickly from books and the internet.  As they grow they try and hire similar people, but eventually there comes a point where a lot of staff are there for the job.  They need training because they aren’t going to go to the trouble of autodidactism.  They have a job to do, and they’re going to do that and no more or less.

At best, they’ll get on-the-job training.

Our Marketing

We don’t do a lot of it, to be quite frank.  But perhaps our website, approach and costs simply don’t ring true with people in medium sized companies looking for WordPress training?  Do we need to get advertising in business magazines, such as those sent out by the FSB?

Summary

I suspect that the truth is that a company always needs to look at the skills of staff, but as they reach a certain size they have too much going on to give it much thought.  Once they break through that difficult 50 man barrier things seem to start to change again.

But how do we change this?  I’m not sure we can, easily.  Governments often give generous grants, and around here we have Skillworks which helps a little, and we do get local approaches – however, the person paying still has to pay a significant contribution and specialist training is never all that cheap.

Do you have any ideas of how skills can be improved in small to medium sized enterprises?  Should we be marketing our courses more proactively?  What about scheduled classroom courses that keep things at a lower cost, albeit needing more time from the attendees?  I’d love to hear your thoughts – especially in the field of WordPress.

Is WordPress Slowing My Site Down?

I noticed on my pingdom stats recently that this site has been slowing down recently. But why? Is it WordPress? Some plugins? Time for research!

I don’t really look at server response times too much, because generally it’s a bit dull, really.  If our server is really sick then the automatic alerts and text messages from Pingdom tell me to get fixing.

Anyway, tonight I just thought I’d have a look at some graphs.  And hey, there’s a bit of a shock… my own site, this one here, is getting slower.  Here’s the graph:

 

Now, what you’ll notice is that generally everything looked nice and brisk until early August 2009 when there was a marked deterioration in performance.  Then it plodded along just fine until December 2009 when it got worse again.

Is It WordPress Bloat?

I wondered what had happened to cause this – I’m especially shocked that a page load is now so slow.  I haven’t changed the theme in use for at least a year, and the plugins are generally ones I use elsewhere.  So I did a bit of thinking, and then it dawned on me when I found this list of release dates at WordPress.org:

The dates correlate exactly with the worsening performance of this site.

Now, correlation absolutely does not relate to causality.  At upgrade time I usually take a moment to review plugins, upgrade the theme, etc.  It could be that an upgrade or change to one of the many plugins in use here that has caused the slowdown.  I know it’s not traffic to my site – that’s been flat for ages, with only the occasional blip.  The server is a dedicated machine running about fourty sites that we manage, but the load is generally quite consistent and impact on different websites is fairly well managed.

But I’m a speed freak.  I like it when a website zings into view.  I love fast cars.  I love computers that don’t dither.  I love telephones that respond instantly.  So I hate that response times are now below 1s.  That’s not on!

So I’m going to try to find the cause of this slowdown – I’ll be running some experiments on this site, profiling queries, checking the database over and so on.  And I’ll document it all as a way of showing you how we at Interconnect IT do our performance tuning.  Whether I document it here or there I haven’t decided yet – I think it would make a good case study for the site.  We’ll see!  Watch both sites…

note 1 – deactivated Twitme on 03/02/2010

What it’s Like to Present and Attend at WordCamp UK

Over the weekend just gone I made two planned presentations at WordCamp UK 2009 down in Cardiff. I also threw in a quick 45 minutes of show and tell on the Caribou Theme that runs this site and is available for download from Spectacula.

Over the weekend just gone I made two planned presentations at WordCamp UK 2009 down in Cardiff.  I also threw in a quick 45 minutes of show and tell on the Caribou Theme that runs this site and is available for download from Spectacula.

I also got to mix with some very interesting, talented and cool people that know a heck of a lot of stuff about what we’re working with.  In this conference were, potentially, the next generation of web creators.  People who will make things happen.  And this year, more than last, there was a real buzz at the potential of WordPress, its markets, and its uses.

Presenting at WordCamps

Presentation
Presentation

I’ve never actually done a formal presentation in front of more than about ten people before in my life, and even then only perhaps four five in my life.  I’m a techie – I would do technical discussions and demos, but never with Powerpoint and a laser pointer.  I did do an unconference show and tell at Barcamp Liverpool last year where about twenty to thirty people turned up, but that wasn’t planned… it just kind of happened.

But I could also appreciate the benefits of putting myself out there in front of a room full of my peers.  So in a fit of enthusiasm I volunteered for two presentations – WordPress in the Enterprise, and WordPress for News and Media.  I expect one or even both might be dropped by the organisers.  I have no history or background in public speaking.

Both got accepted.

Damn!

But it had some great potential too.  I could play with approaches and actually ‘test’ the results.  So what did I learn?  Read on:

1. L-Shaped Rooms are Tricky

The main room for the event turned out to be L-shaped… or, a better description, V-shaped, with the presented at the bottom of the V.  At this event three rooms were in operation, a large L-shaped room with up to 150 people, a medium sized rectangular room for up to 70 people, and a small boardroom type for about 15 people.

I had expected my first presentation on the Enterprise to be the tricky one – it’s not a fascinating subject.  But it was in the medium sized room, and it proved very easy to get engagement with the audience.  In the L-shaped room you’re trying to look in two different directions.  It’s almost impossible.

2. Consider an Assistant for Demos

One can work the computer, the other can talk.  Saves awkward silences, and it’s something I’m going to try in a future talk.

3. Get in Early

I did one of the first, and the very last, formal presentations of the event.  I noticed that in the first everyone was wide awake and very enthusiastic.  By the end of the conference people were flagging.  Getting and keeping attention becomes trickier at this stage.  You also have the advantage that nobody ever wonders off from the conference at the very beginning – it’ll never be fuller!

4. Start Funny

In the Enterprise talk I started with a humorous quote and in the News & Media I started with a pithy quote.  The funny one got the mood lifted and people in a cheerful mood.  It gave me a chance to relax and settle into the presentation.

5. It’s a Great Audience

I was dealing with fellow geeks.  People in the same situation as me.  It was, frankly, the best audience I can imagine.  The few presentations I’ve done before have been up in front of a board of hardened and cynical directors, or senior management, or people who have tough deadlines to meet.  This was a whole lot more relaxed.  Nobody’s going to consider firing you because of a minor mistake.

6. Get Engagement

I noticed that speakers who asked for shows of hands, asked questions of the audience and so on generally had a better applause at the end than those who didn’t.  It doesn’t take much to engage your audience, but I’ll admit that it’s trickier when you can only look directly at half of them at any one point.

7. Be Prepared

At conferences opportunities come up.  Have business cards, listen to people, smile a lot.

8. Freebies

You can’t believe how the mood of a room lifts when you hand out gifts.  Good gifts though.  I remember the really rubbish calculators we got given in my ICI Systems days.  What geek in the world needs a calculator?  So I handed out the penknives we had made for Spectacu.la and they went down a treat.

9. Matt

I finally met Matt Mullenweg at the weekend.  I’d promised him a beer months ago in reconciliation following our (now seemingly minor) argument over WordPress’s take on the GPL.  So I bought him a pear cider and had a good chat.  He’s an affable chap, easy going, says ‘awesome’ a lot (but he’s American, so that’s normal) and has clearly listened to the concerns of WP developers about how they’ll make any money.

Funny hat tho’ ;-)

Summary

An ace time, basically.  I’d like to say hi to everyone I met, but I’m scared of missing someone – so instead, let’s just say I look forward to chatting and, hopefully, working with some of you in the not so distant future.

Here’s to WordCamp UK 2010!

Five Things Bing Does Better than Google

MS have, at last, come up with what appears to be a competent rival to Google. Here’s five ways in which it beats Google.

Microsoft (MS), quite frankly, gets a lot of grief in the internet world.  Sometimes it’s fair (I never like MSN, for example, from way back in the mid nineties) and often a little unfair.

But Live Search simply wasn’t up to the job.  It didn’t work well.  And I know that people that found IE defaulting to it would either work out how to change it, or simply type Google.com into the address bar.  In other words, many tried it, but it didn’t find the answers they wanted.  The algorithm has been slowly improved with time, but the damage was done.  MS knew they had to relaunch.

Bing, they felt, was the answer.  And in some ways, it’s a better and more productive tool than Google:

Bing - pretty pictures to cheer you up
Bing - pretty pictures to cheer you up

1. It’s Prettier

While I’ve heard many question the function of the landing page photo, I personally really like it.  It’s attractive, well designed, and brings a little bit of beauty into the day.  You can’t sit and surf pretty images at work, so if they’re there as part of the ‘wallpaper’ of a daily tool then that’s a lift we all need.

2. Infinite Image Search

The infinite scroll facility of the image search makes it a quicker tool to use.  Chunking of text related searches makes sense, because we can scan a page relatively slowly, but with images the human eye can scan a huge amount of visual information incredibly quickly which means that Bing’s constantly scrolling visual tool is way ahead of Google’s image search.

3. Video Previewing on Video Search

bingvideo
Bing Video - content owner's nightmare or benefit?

Searching for video content can often be a slow and painful process.  In Bing, when you get a series of videos up on screen you can simply hover your mouse pointer over a video to preview the first 30s and get a feel for the video, rather than visiting the site and waiting for a slow load.  The previews are poor quality, in order to get quick loading, but they’re good enough.  I feel this is one of Bing’s most effective innovations.

One thing where they may struggle is that if you click the video and that video has an embed option, you get it on the Bing site, rather than going through to the source site.  So a YouTube video search result doesn’t send you off to YouTube.  Content owners may not like this.

4. Site Preview

When you hover over a search result, you’ll see a small orange marker appear over to the right.  Hover over that and up pops a preview of the content you’re looking for.  Again, saves a wasted visit as it lets you scan a little bit of content for relevance – something that’s quicker this way than clicking on yet another unnecessary site.

5. It’s Not Google

Bing is, purportedly, a recursive acronym that means Bing Is Not Google.  But there’s something important in that – Microsoft is a highly profitable, focussed company that has the resources to provide an alternative to Google.  This is important – without solid competition Google will cease to innovate appropriately.  MS suffered a similar fate on the desktop – they were too dominant and rivals couldn’t compete.  Apple’s OS9 was dreadfully dated when sat next to a Windows machine of the same era, yet Windows had significant flaws.  It’s only lately with Windows 7 that MS have really started to get their act together properly – because OSX finally gave it some decent competition in certain sectors.

When you start seeing articles on how to change from Google to Bing on Firefox, you know something’s happened.

It Can Get Better

Microsoft Seadragon, with it’s deep zoom and mobile capabilities, and Photosynth technologies could be tied into the image search, for example.  As cheap processing power expands and more and more images are geotagged, this could form an astonishing visual search capability.  A shame it won’t be coupled with Google Street View – imagine what that could be like?

Search is going to become more relevant and more powerful with time.  Developers (our own Interconnect IT included) are busy creating a lot of powerful geocoded databases which will allow for some amazing mashups.  If Google and MS start fighting for dominance in this space the opportunities for users and information suppliers are vast.  Are you looking into it?

mySQL Database Search & Replace With Serialized PHP [Updated]

Ever needed to migrate a database to a new server or website (especially with WordPress and other PHP applications) and been stuck because when you do a search and replace some of the data seems to get corrupted?

Please note that a newer version of this code is now available from my company’s Spectacu.la site over at http://spectacu.la/search-and-replace-for-wordpress-databases/

Ever needed to migrate a database to a new server or website (especially with WordPress and other PHP applications) and been stuck because when you do a search and replace some of the data seems to get corrupted?

Serialized PHP Arrays Cause Problems

In PHP one of the easiest ways of storing an array in a database is to use the serialize function.  Works a treat, but the downside is that you’re not storing data with a cross platform method.  In many product development environments this would get you a stern talking to, but in the world of web development where deadlines are tight and betas are the norm, this seems to be overlooked somewhat.

So what we have are tables full of data that can’t be easily edited by hand.  For example:

;a:3:{s:5:"title";s:17:"This Week\'s Poll";s:18:"poll_multiplepolls";s:0:"";s:14:"multiple_polls";N;}

Say you had thousands of records like the one above, and the word ‘multiple’ needs to be changed to ‘happy’.  Two bits would change – poll_multiplepolls would now read poll_happypolls and multiple_polls would read happy_polls.  In both cases you would have three characters fewer to deal with.

Fine, you may think, but you can only do the change by hand because where it says s:18:"poll_multiplepolls" it now has to say s:15:"poll_happypolls" – see the difference?  S18 spells out the length of the following string, and it has to be changed to s:15

I’ll say right now, that that was a pain.  For simple arrays I wrote the straightforward PHP Serialization fixer code, which got me out of many a pickle – do the search and replace without worrying, and then run the script.  Fixed about 90% of problems.

Multidimensional Array Problem

Sadly those 10% of problems left were a real pain.  I needed something more robust.  Something more powerful.  And finally today it was a Bank Holiday in the UK – that means no phone calls… I could have a quiet day of coding and concentrate on the best solution to this problem.

What I’ve done is to write a database search and replace utility in PHP that scans through an entire database (so use with care!) which is designed for developers to use on database migrations.  It’s definitely not what you’d call an end-user tool, though I may sanitize it at some point and turn it into an easy to use WordPress plugin.  Thing is – this is dangerous code – sometimes I think it’s better to make it deliberately a bit tricky, don’t you?

It’s not that bad though – if you can manually install WordPress, you can easily configure the database connection settings.

What the code does is to look at the database, analyse the tables, columns and keys, and then starts reading through it.  It will attempt to unserialize any data it finds, and if it succeeds it will modify that data then reserialize it and pop it back in the database where it found it.  If it finds unserialized data it will still carry out the search and replace.

Use in WordPress

In most WordPress migrations you tend to have the primary problem of changing the domain name entries in content, settings and widgets – you simply need to put in the $search_for string the old domain address (including the http if it’s there) as seen on the database, and the new one into $replace_with.  Then put this script onto your server, and run it by visiting it in your browser or inputting the appropriate command line – depending on your server configuration.

Other things you may want to check are for plugins or themes that have made the mistake of storing the full server path to the installation – cFormsII does this, for example.  You will need to find out your old and new server paths and use those, in full, for another iteration of this script.

After less than a second of running, you should have a freshly edited database.  It may take a little longer on slow or share hosting, or if you have a very large database, but on my laptop I can manage around 60,000 items of data per second.

I’ve just used the script to migrate, in its entirety, with content, settings, 87 widgets (yes, really!) and hundreds of images to my localhost server.  It took moments, and the site is perfectly preserved.

Search and Replace Database download.
download file

Search and Replace Database download

BIG WARNING: I take no responsibility for what this code does to your data. Use it at your own risk. Test it. Be careful. OK? Here in the North we might describe the code as being as “Rough as a badger’s arse.” Never felt a badger’s arse, but I’ll take their word for it.

Slow Posting Performance in WordPress 2.7

An interesting problem today – WordPress posting on this site was sloooooow.  Some other back-end tasks were also incredibly slow.

Adding a post or page would take about 30 to 60 seconds.  Unacceptable.  I did the usual job of deactivating plugins, and even resorted to a different theme for a few minutes but no, it was all still very slow.

That’s when the work starts.  On this server we haven’t opened up port 3600 to allow remote connection with mySQL Administrator, so instead it’s straight into phpMyAdmin.

The process is then pretty simple – for each table you have an Operations tab.  Here you can see some information about the state of the table as well as quick links at the bottom to allow you to run various tools.  The process I use is to backup the site then run a Repair then an Optimize on the _posts (typically tables are prefixed WP_) , _postmeta and _options tables.

Why the Slowdown?

Well, I didn’t bother investigating properly – without full server access it’s often hard to get full timings.  You can put traces into code and look at logs, or you can very quickly run the Repair process.  But what I do know is that databases can slow down for inserts more easily than for reads.  This is usually because something’s become messed up in an Index, or the whole table’s become inefficiently organised over time.  By repairing and optimizing you get to quickly tidy things up and restore the performance of the site.

For a minor site like this, it’s about the best approach to take.  If you’re running something like Facebook you’ll probably want to investigate things properly – but then you’ll have the money to be able to do so as well!

Handy Tip for CSS Reloading

CSS not refreshing in the browser can be a pain – Mark Jaquith has come up with an elegant solution to this for WordPress coders.

I saw this rather handy tip on CSS reloading on Mark Jaquith’s blog today.  Such a simple approach, but a great one for when you’re making CSS changes and need browsers to pick up on it immediately, rather than some hours down the line.

Some changes are critical.  Now you can version your css by hand, but you don’t have to.

The basis of the change is to use the code:

<link rel="stylesheet" href="<?php bloginfo('template_url'); ?>/style.css?v=<?php echo filemtime(TEMPLATEPATH . '/style.css'); ?>" type="text/css" media="screen, projection" />

But for the detail, go read Mark’s post.

Broadcom B57nd60x 10.10.0.0 performance problems (Dell XPS especially)

CPU spikes were making my laptop less fun to use – a nice online guide and a quick bit of sleuthing with MS’s sysinternals tools and I soon had the answer. The laptop is now faster, quieter and has better battery life.

Excuse the title, but may as well make it easy to find.  I’d been experiencing problems with performance, whilst networking, with my Dell XPS M1330.

CPU Spikes

Basically, the CPU usage was spiking on a regular basis.  I could feel when playing games, and it was annoying.  It had started relatively recently, and the precise cause was unknown.  However, a bit of Googling and I found Mark Russinovich’s excellent overview of using Sysinternals Process Explorer and Kernrate to track down the root of this kind of spiky CPU usage.

And my problem was exactly the same.  Same driver, same version – the B57nd60x 10.10.0.0 driver was gobbling up CPU at a frightening rate.  However, although he’d reported the problem, at the time there was no solution and a new driver wasn’t available on the Dell website.  A year later, the driver still isn’t available – the Dell driver is resolutely stuck at v 10.10.0.0 – so, no fix.

Excessive CPU consumption = poor battery life

But it’s annoying seeing your CPU running constantly at 20%.  It also has an impact on battery life.

So I looked around a little further and found an updated driver to download at the the broadcom site, for version 11.7.3.0 – surely this would have a fix, as Broadcom were aware of the problem thanks to Mark’s excellent work.

And it worked – the screenshot below shows the impact – the first third or so shows the CPU usage with the old driver, and then it drops dramatically:

broadcom_driver_performance

That Dell haven’t updated their driver pack in over a year is something of a support fail – it makes the XPS M1330, at least in certain circumstances, somewhat less of a great PC than it could be.  And it’s also poor for the PC community – a lot of criticism is made of Windows being something that slows down over time.  It’s rarely the fault of Microsoft – often it’s driver issues, but finding the latest drivers isn’t easy for everyone, and it’s quite technical to solve.  MS could possibly make driver management a simpler system, but the PC makers could help by keeping up to date driver packs – especially for laptops which are rarely modified.

PHP Serialization Fix for WordPress Migrations (& other applications like Expression Engine)

Serialization of data loaded into an SQL table is a dreadful thing and makes WordPress migrations harder than they should be, but it happens and so we must deal with it. I’ve knocked up a rough and ready bit of code which does its best to resolve the problem.

When you move a WordPress blog from one folder to another, or from one site to another, you normally use the export/import functionality.

This is fine for normal blogs, but say you’ve developed a new website and set it up on your local machine – the URL for the site may be something like http://localhost/devsite and the live URL will be something like https://davidcoveney.com – you won’t want to set up all the theme options, site options, plugin options and so on all over again.

A different kind of migration - public domain from Wikipedia Commons
A different kind of migration – public domain from Wikipedia Commons

Instead, a theoretically simple approach is to do a database dump, a search and replace for all references to server paths and URLs, and then reimport that data in the new location.

Should work, but it often falls apart.

What happens is that in WordPress, its themes and its plugins, a lot of data is stored using a method known as serialization.  Now, in my opinion this breaks all known good practice around data – it’s language specific, it’s not relational even though it often could be, and it’s hard to edit by hand.

One particular problem is that if you change the length of the data in a serialised string you have to change the length declared in the generated string.

That’s very painful when you have hundreds of the fields.

So, because I’d found this painful I decided to knock together a quick application to at least reduce the amount of editing I had to do.  You just do your search and replace, forget about the serialized string lengths, upload your data to the new database, and run this script.

Warning: I haven’t got it to work for widgets and cForms II yet, but the latter has some export functionality anyway, which takes that particular pain away if you plan ahead.  In the meantime, feel free to play with the attached file.  You use it at your own risk, of course.

To use it, download the file linked in this post, extract it, open the file, edit the connection settings, tell it the table you want to scan through, the column, and the unique key field.  If you somehow manage to have more than one unique key to deal with (you shouldn’t, but then it surprises me what people manage to code up), then you’ll have to modify the code accordingly.  Once done, make sure you have a backup of that table, and execute the php – either at the command line or through the browser. License is WTFPL, and if you’d like to improve the code, please do and I’ll host the new version.

Serialization-fixer.zip download.
download file

Serialization-fixer.zip download

BIG WARNING: I take no responsibility for what this code does to your data. Use it at your own risk. Test it. Be careful. OK? Here in the North we might describe the code as being as “Rough as a badger’s arse.” Never felt a badger’s arse, but I’ll take their word for it.