David Coveney

PHP Serialization Fix for WordPress Migrations (& other applications like Expression Engine)

When you move a WordPress blog from one folder to another, or from one site to another, you normally use the export/import functionality.

This is fine for normal blogs, but say you’ve developed a new website and set it up on your local machine – the URL for the site may be something like http://localhost/devsite and the live URL will be something like http://davidcoveney.com – you won’t want to set up all the theme options, site options, plugin options and so on all over again.

A different kind of migration - public domain from Wikipedia Commons

A different kind of migration – public domain from Wikipedia Commons

Instead, a theoretically simple approach is to do a database dump, a search and replace for all references to server paths and URLs, and then reimport that data in the new location.

Should work, but it often falls apart.

What happens is that in WordPress, its themes and its plugins, a lot of data is stored using a method known as serialization.  Now, in my opinion this breaks all known good practice around data – it’s language specific, it’s not relational even though it often could be, and it’s hard to edit by hand.

One particular problem is that if you change the length of the data in a serialised string you have to change the length declared in the generated string.

That’s very painful when you have hundreds of the fields.

So, because I’d found this painful I decided to knock together a quick application to at least reduce the amount of editing I had to do.  You just do your search and replace, forget about the serialized string lengths, upload your data to the new database, and run this script.

Warning: I haven’t got it to work for widgets and cForms II yet, but the latter has some export functionality anyway, which takes that particular pain away if you plan ahead.  In the meantime, feel free to play with the attached file.  You use it at your own risk, of course.

To use it, download the file linked in this post, extract it, open the file, edit the connection settings, tell it the table you want to scan through, the column, and the unique key field.  If you somehow manage to have more than one unique key to deal with (you shouldn’t, but then it surprises me what people manage to code up), then you’ll have to modify the code accordingly.  Once done, make sure you have a backup of that table, and execute the php – either at the command line or through the browser. License is WTFPL, and if you’d like to improve the code, please do and I’ll host the new version.

Serialization-fixer.zip download.

download file

Serialization-fixer.zip download

BIG WARNING: I take no responsibility for what this code does to your data. Use it at your own risk. Test it. Be careful. OK? Here in the North we might describe the code as being as “Rough as a badger’s arse.” Never felt a badger’s arse, but I’ll take their word for it.

  • http://sean.gravener.net Sean Gravener

    You’re a life saver, Dave. I used this script to fix serialization issues in Expression Engine… many thanks.

    • http://davidcoveney.com Dave

      Thanks Sean – glad to hear it was useful for you. When you put these things out you never know whether it’ll help people or not – gives a nice warm fuzzy feeling to hear that it does :-)

  • http://sean.gravener.net Sean Gravener

    You should label it not only to fix WordPress, but as a general fix for corrupted serialized database records – being able to define the db, table, index and column makes it flexible.

    Perhaps the next step would be allowing a user to define an array of table columns instead of one at a time… and perhaps echoing a status at the end :)

    Maybe even a clever name…

    In any case, it’s a big help as is.

    • http://davidcoveney.com Dave

      Aye – good point. I wrote it to solve a very specific problem, using snippets I’d found elsewhere but wrapped up in a single script.

      I’m planning to write a script that lets you do a search and replace on a table or even entire database, deserializing and reserializing as necessary. Ultimately, there needs to be a simple way for people to easily and quickly change values in serialized data. Alternatively, PHP developers could just learn some good database practices, but I’m not holding my breath ;-)

  • http://sean.gravener.net Sean Gravener

    “Ultimately, there needs to be a simple way for people to easily and quickly change values in serialized data”

    Agreed!!

  • http://www.burtonkent.com Burton Kent

    Interesting! I just wrote a script to copy a wordpress install to a new domain, and am running into the exact same problem. The script works except for widgets.

    Any idea exactly what part of widgets is being broken? Maybe I can figure it out for both of us.

    Thanks!

    • http://davidcoveney.com Dave

      Burton – I think the problem is down to nested arrays – it’s just too hard a problem for my simple regex fix to solve.

      What’s needed instead is a script to read a database, pick up what’s serialized php, unserialize it, carry out the search and replace, and then re-serialize the data and put it back into the database. But that’s a considerably bigger job.

      My script is, essentially, a sticking plaster to fix most of the problems. You could do the widgets by hand – at least then there’s not too much to do. Everything else seems to be OK, though I’ve noticed some plugins like cFormsII do this too, so that’s another manual job.

  • Pingback: Checklist For Moving Domains - WordPress Tavern Forum

  • Pingback: Automatic WordPress Migration « BAM Weblog

  • Pingback: mySQL Database Search & Replace With Serialized PHP [Updated] » David Coveney

  • http://skcdev.com/ Scott Kingsley Clark

    WOW! Thank you so much for this! I can’t believe it worked so well, I’m not even using it for WP specifically, but a plugin that stores some values in a serialized array and I had edited it in the db and adjusted the count of the string, but saving it completely busted open the array from being unserialized. Running it through this filter fixed it right up! Thanks again for your contribution Dave!

  • http://www.wordimpressed.com/ Devin Walker

    We need cForms II and Widgets to be moved also! This is one of the most painful parts of any WordPress migration… any solutions?

    • http://davidcoveney.com Dave

      It’s the whole point of the plugin, to allow widgets to move. You should get the latest version from spectacu.la rather than here though.

      However, cForms II is ropey as hell, you may struggle with that one. There’s a php file to change but I can’t remember it I’m afraid.

  • http://kucrut.org/ kucrut

    Thanks, you just saved me :)

  • http://www.wsjfc.org.au Stuart Travers

    I added some customisation that let me put multiple tables/columns into an array and processed them in a loop – as a result this just saved me a stack of time and restored a broken wordpress installation to full working order in seconds! Thanks Dave.

    • http://davidcoveney.com Dave

      Hi Stuart – over at http:///interconnectit.com we’ll soon be releasing a new version of our search and replace plugin that lets you choose tables, runs faster, and is full of lovely.

  • manuel

    You are the man!!! Great!! Thanks a lot!

  • Brice Burgess

    Thank you my man!

  • http://grandjunctionmortgagebroker.com/ RC Thompson

    Hey thanks Dave.
    Appreciate the help!

  • http://blogestudio.com Pau Iglesias

    Hi, I released a similar PHP script but working with database dumps and need to make global replacements, this script repair serialized length values:

    https://github.com/Blogestudio/Fix-Serialization

    Regards
    Pau

    • David Coveney

      Hi Pau – thanks for that looks useful.

      If you’re doing migrations with WP, another option is to use the Search/Replace script we wrote at work for databases, which built on my work above: http://interconnectit.com/124/search-and-replace-for-wordpress-databases/

      My script here is really just for remedial fixes where it’s all gone wrong and I can’t think what to do next!

  • http://blogestudio.com Pau Iglesias

    Thanks David, good script to make changes directly in running databases, I will try it :-)

  • http://fishfindersworldwide.com/ Jonnyandandrea

    HI there, just wondering if there are any updates to this posts? I see the last comments are a while ago?!

  • Hannah West

    Hi, I moved a WordPress blog and after several failed attempts I managed to get everything working at the new url (doing this for a nonprofit that just got their domain name after hosting in a subdirectory of my site for a while). Except of course the images. I am so apprehensive about doing anything more, but need to get this done and behind me and the responses you’ve gotten in the comments look like it’s safe for me to try (I’ve worked with WP a lot, but not very sophisticated when it comes to database management and PHP). Before I go for it, I was just wondering where the best place is to download your serialization fixer? Here, or at one of the other places mentioned in the comments? Thanks so much!

    • davecoveney

       Hi Hannah – you should probably use the WP Search and Replace tool over at http://interconnectit.com/124/search-and-replace-for-wordpress-databases/ which is a user friendly (but still high-risk) tool.

      The important thing is to make sure you backup your system first!

      • Hannah West

         Thanks Dave. I did use Search and Replace. It did a great job with everything else, but my images are still not showing up and I can’t upload new ones for whatever reason. Do you think your tool will help me restore the images, or is there another strategy I should use instead?

      • Hannah West

        I’ll try again and let you know how it goes…

  • mike

    Im having an problem that happens to me always.
    On every project i start i work locally and once im done i export my database and search replace all localhost url’s to the new url.
    then in cpanel i create a new database user name and password and modify wp-config.
    the website works fine all pages working just one thing wont work and i cant find a solution:
    all my Theme Options wont show
    my theme options settings and s widgets wont show up and i have to redo this process from the beginning so each time i will have to redo all the settings.
    i have read a lot about it:
    http://codex.wordpress.org/Mov
    and even used this tool you posted:
    http://interconnectit.com/prod
    but still don’t know what am i doing wrong.
    same thing happens when i try to install a site on local host from a backup.
    is there a tutorial on how to do this the right way or can someone please please help.
    Thanks in advance

    • http://davidcoveney.com/ David Coveney

      It could well be an encoding issue – the very latest version 3 of search/replace has some major changes in how it handles encodings. Give it a test. Also if you have sample data it could be very very useful to us in testing and finding the basic problems.

      Also, do take a look at Shannon’s fix above – that could well help you out.

  • http://www.frothingthefrap.com/ Shannon Black

    major issue with this i finally sorted :)

    $__ret =preg_replace(‘!s:(d+):”(.*?)”;!e’, “‘s:’.strlen(‘$2′).':”$2″;'”, $sObject );

    should be

    $__ret =preg_replace(‘!s:(d+):”(.*?)”;!e’, “‘s:’.strlen(stripslashes(‘$2′)).':”$2″;'”, $sObject );

    serialized php strings don’t count special escaped characters as 2 letters but rather one.

    example: s:1:”n”

    • Xis

      I was working for hours repairing my database!!! THANK YOU -> this fix worked for the repair script!!!

    • http://davidcoveney.com/ David Coveney

      Hi Shannon – I’ll see about incorporating your fix – thanks,

  • Jake

    Her’s one that works with SQL dump: http://pastebin.ca/2444926

  • Pingback: Anonymous

  • Jeff Hunsaker

    Awesome utility. Thanks. One thing for folks to also be aware of (that tripped me up) is to check and update the number of elements in the serialized data. That the value outside the brackets: a:125{} Mine was incorrect and throwing off the engine. I simply counted the “s:” occurrences in the data, updated the “a:” value, and saved.

  • RedhopIT

    GoLive plugin could handle the migration (all steps) and update the serialized data properly:

    http://codecanyon.net/item/wp-golive/7793291