I’m bringing it up here because I’ve decided to trial the plugin out here on my own site. It was designed to work in conjunction with a webinars project, allowing visitors to have an active discussion, in real time, on a WordPress site. It can be dropped into almost any theme, and adds nicely to the standard WP comments functionality.
Threaded comments are a powerful way to turn your WordPress site into a mini discussion forum. Adding live commenting can now turn it into a chatroom full of ajaxey goodness.
I’d started to notice that my site could often be slow to load – other sites on the same server weren’t suffering the same way, so I wanted to document a simple way in which one can identify performance issues on the site. This is one of them.
Well, it took me a while to get back to the issue (babies and a booming business don’t help!) it’s continued to get worse and worse, until a recent change has improved things… but only marginally, as shown by the Pingdom chart below:
Not looking good…
This is dreadful, really – daily average of 4,000ms responses just aren’t acceptable where, two years ago, I was getting 800ms.
So, now the process starts. The recent small improvement came after installing our Spectacu.la Advanced Search Plugin, which runs a regular database optimisation to help keep things nippy, but it was still dreadful.
Is it Pluginitis?
My first suspicion is always that of plugins (and sometimes themes, if they’re complex). In our office we have a term called ‘pluginitis’ which refers to the problem of a site having too many plugins installed, many of which are poorly written. I hate to say it, but when clients call to ask for a plugin to be installed that we’ve never tested we go through it and, 90% of the time, discover serious performance or security flaws that will cause long-term issues.
And this site here is old – I’ve been running a WP install for four and a half years with nothing more than upgrades and, like an old PC that’s been upgraded too many times, that causes issues with old drivers and code. Same can apply to WordPress. So let’s see what we can do to improve things.
First stage is to disable as many plugins as possible so as to isolate the issue. I’m using a division based approach – ie, I’m going to disable half of my plugins to see what happens. If I get full performance back, then the problem lies in that half. I can then reactivate half the plugins and see what happens. If the performance is still good, the problem is in the other half. I think you can see where I’m going here.
I’m also going to go for plugins that aren’t written by us. Not because I’m biased (ok, maybe a little) but because I know all of ours are carefully tested for performance – many are run on major sites such as the Telegraphs blogs site. Speed is of the essence.
I’m also going to skip plugins like Akismet, because anything that’s essentially ‘core’ is usually going to be reasonably performant – at least on a small site like this one.
It’s worth noting that I could easily delve into SQL statements and code efficiency – but that’s only interesting to developers – if you’re simply a WordPress user, performance is interesting but what you can do to find problems is somewhat more limiting.
Plugins being disabled:
Add to feed – a simple plugin, but sometimes simple plugins miss simple tricks.
Headspace2 – I have my suspicions about this plugin as it’s massive. Could be fine, may not be. Only way definite way to know – measure it.
Search Meter – a nice plugin to see what people are searching for, but is it adding load somewhere?
Social Bookmarks – it shouldn’t cause issues, but you can never be sure.
wp-typography – I love what it does for the typography on the site, but it’s also running a lot of javascript.
First results:
I do use YSlow to test the site, but one of the problems is that it’s hard to get a large enough series of data to be statistically relevant. It’s good for seeing the extra load (and why I knew the amount of javascript was an issue) but for longer term analysis it’s flawed.
So, we go back to Pingdom and look at the one day chart. As I type this it’s now an hour since disabling the plugins above – so let’s see what’s happened:
A dramatic improvement!
As you can see, in this afternoon alone there’s been a dramatic improvement – from around 2500ms per visit to 1230ms per visit. In one single step I’ve halved the load time of the page.
What we don’t know so far is whether that’s because the page got smaller to load or whether it’s down to a reduction in database load – but that’s really for another article. What this is all about is trying to document how I’m improving the responsiveness of the site in a way which relatively non-technical folk can follow.
What I’ll do in the next feature is to turn off some more plugins to measure the impact they had. I’ll also be interested to see if the spikeyness of the response times has varied much – are they caused by simple server load, or is there something else at play?
I will then start to switch plugins on again in a structured way in order to measure which was causing the heaviest loads on the site.
Over at Interconnect IT, where I work, we’ve found that demand for our WordPress Training Courses has shot up over the past year or so. And it’s certainly interesting to see where the work comes from.
I’ve been thinking about why there’s such a big dip in the middle of our client base – 90% of our business is groups with over 500 staff or with fewer than three.
The Gap
Small, one man companies tend to involve highly motivated individuals. They understand the important of their skills, and they know that in the tech sector they have to always stay up to date. Large corporates tend to understand this too, and have allocated training budgets to make sure they keep up to date. Government departments are often a little slower with new technology, but they too need to keep efficient or tax payers will kick up a stink.
And WordPress is a very efficient platform for running many informational websites.
But why are medium sized companies not coming to us in nearly such large numbers as micro enterprises and large corporates? Let’s see…
Budgets
I’ve noticed that medium companies often have people who are extremely good at what they do, but I do find that there are often significant skills gaps. I’ve dealt with someone from a school (medium) who didn’t realise that you don’t have to close an application in Windows in order to see another application. Each time she cut and paste she’d open one document, select the text, close the document, then open the receiving application and paste the text there. Productivity, as you can imagine, was pretty poor.
I think a lot comes from budgets – many firms around the 50 people size aren’t always able to make good money. They have to be careful where they spend it – they’re not big enough to have training departments who make sure everyone is up to date, and they’re usually busy.
Culture
Another thing with smaller firms, I believe, is that they’re often started by individuals with a lot of skills – these are flexible folk who can teach themselves and learn quickly from books and the internet. As they grow they try and hire similar people, but eventually there comes a point where a lot of staff are there for the job. They need training because they aren’t going to go to the trouble of autodidactism. They have a job to do, and they’re going to do that and no more or less.
At best, they’ll get on-the-job training.
Our Marketing
We don’t do a lot of it, to be quite frank. But perhaps our website, approach and costs simply don’t ring true with people in medium sized companies looking for WordPress training? Do we need to get advertising in business magazines, such as those sent out by the FSB?
Summary
I suspect that the truth is that a company always needs to look at the skills of staff, but as they reach a certain size they have too much going on to give it much thought. Once they break through that difficult 50 man barrier things seem to start to change again.
But how do we change this? I’m not sure we can, easily. Governments often give generous grants, and around here we have Skillworks which helps a little, and we do get local approaches – however, the person paying still has to pay a significant contribution and specialist training is never all that cheap.
Do you have any ideas of how skills can be improved in small to medium sized enterprises? Should we be marketing our courses more proactively? What about scheduled classroom courses that keep things at a lower cost, albeit needing more time from the attendees? I’d love to hear your thoughts – especially in the field of WordPress.
I noticed on my pingdom stats recently that this site has been slowing down recently. But why? Is it WordPress? Some plugins? Time for research!
I don’t really look at server response times too much, because generally it’s a bit dull, really. If our server is really sick then the automatic alerts and text messages from Pingdom tell me to get fixing.
Anyway, tonight I just thought I’d have a look at some graphs. And hey, there’s a bit of a shock… my own site, this one here, is getting slower. Here’s the graph:
Now, what you’ll notice is that generally everything looked nice and brisk until early August 2009 when there was a marked deterioration in performance. Then it plodded along just fine until December 2009 when it got worse again.
Is It WordPress Bloat?
I wondered what had happened to cause this – I’m especially shocked that a page load is now so slow. I haven’t changed the theme in use for at least a year, and the plugins are generally ones I use elsewhere. So I did a bit of thinking, and then it dawned on me when I found this list of release dates at WordPress.org:
The dates correlate exactly with the worsening performance of this site.
Now, correlation absolutely does not relate to causality. At upgrade time I usually take a moment to review plugins, upgrade the theme, etc. It could be that an upgrade or change to one of the many plugins in use here that has caused the slowdown. I know it’s not traffic to my site – that’s been flat for ages, with only the occasional blip. The server is a dedicated machine running about fourty sites that we manage, but the load is generally quite consistent and impact on different websites is fairly well managed.
But I’m a speed freak. I like it when a website zings into view. I love fast cars. I love computers that don’t dither. I love telephones that respond instantly. So I hate that response times are now below 1s. That’s not on!
So I’m going to try to find the cause of this slowdown – I’ll be running some experiments on this site, profiling queries, checking the database over and so on. And I’ll document it all as a way of showing you how we at Interconnect IT do our performance tuning. Whether I document it here or there I haven’t decided yet – I think it would make a good case study for the site. We’ll see! Watch both sites…
Over the weekend just gone I made two planned presentations at WordCamp UK 2009 down in Cardiff. I also threw in a quick 45 minutes of show and tell on the Caribou Theme that runs this site and is available for download from Spectacula.
Over the weekend just gone I made two planned presentations at WordCamp UK 2009 down in Cardiff. I also threw in a quick 45 minutes of show and tell on the Caribou Theme that runs this site and is available for download from Spectacula.
I also got to mix with some very interesting, talented and cool people that know a heck of a lot of stuff about what we’re working with. In this conference were, potentially, the next generation of web creators. People who will make things happen. And this year, more than last, there was a real buzz at the potential of WordPress, its markets, and its uses.
Presenting at WordCamps
Presentation
I’ve never actually done a formal presentation in front of more than about ten people before in my life, and even then only perhaps four five in my life. I’m a techie – I would do technical discussions and demos, but never with Powerpoint and a laser pointer. I did do an unconference show and tell at Barcamp Liverpool last year where about twenty to thirty people turned up, but that wasn’t planned… it just kind of happened.
But I could also appreciate the benefits of putting myself out there in front of a room full of my peers. So in a fit of enthusiasm I volunteered for two presentations – WordPress in the Enterprise, and WordPress for News and Media. I expect one or even both might be dropped by the organisers. I have no history or background in public speaking.
Both got accepted.
Damn!
But it had some great potential too. I could play with approaches and actually ‘test’ the results. So what did I learn? Read on:
1. L-Shaped Rooms are Tricky
The main room for the event turned out to be L-shaped… or, a better description, V-shaped, with the presented at the bottom of the V. At this event three rooms were in operation, a large L-shaped room with up to 150 people, a medium sized rectangular room for up to 70 people, and a small boardroom type for about 15 people.
I had expected my first presentation on the Enterprise to be the tricky one – it’s not a fascinating subject. But it was in the medium sized room, and it proved very easy to get engagement with the audience. In the L-shaped room you’re trying to look in two different directions. It’s almost impossible.
2. Consider an Assistant for Demos
One can work the computer, the other can talk. Saves awkward silences, and it’s something I’m going to try in a future talk.
3. Get in Early
I did one of the first, and the very last, formal presentations of the event. I noticed that in the first everyone was wide awake and very enthusiastic. By the end of the conference people were flagging. Getting and keeping attention becomes trickier at this stage. You also have the advantage that nobody ever wonders off from the conference at the very beginning – it’ll never be fuller!
4. Start Funny
In the Enterprise talk I started with a humorous quote and in the News & Media I started with a pithy quote. The funny one got the mood lifted and people in a cheerful mood. It gave me a chance to relax and settle into the presentation.
5. It’s a Great Audience
I was dealing with fellow geeks. People in the same situation as me. It was, frankly, the best audience I can imagine. The few presentations I’ve done before have been up in front of a board of hardened and cynical directors, or senior management, or people who have tough deadlines to meet. This was a whole lot more relaxed. Nobody’s going to consider firing you because of a minor mistake.
6. Get Engagement
I noticed that speakers who asked for shows of hands, asked questions of the audience and so on generally had a better applause at the end than those who didn’t. It doesn’t take much to engage your audience, but I’ll admit that it’s trickier when you can only look directly at half of them at any one point.
7. Be Prepared
At conferences opportunities come up. Have business cards, listen to people, smile a lot.
8. Freebies
You can’t believe how the mood of a room lifts when you hand out gifts. Good gifts though. I remember the really rubbish calculators we got given in my ICI Systems days. What geek in the world needs a calculator? So I handed out the penknives we had made for Spectacu.la and they went down a treat.
9. Matt
I finally met Matt Mullenweg at the weekend. I’d promised him a beer months ago in reconciliation following our (now seemingly minor) argument over WordPress’s take on the GPL. So I bought him a pear cider and had a good chat. He’s an affable chap, easy going, says ‘awesome’ a lot (but he’s American, so that’s normal) and has clearly listened to the concerns of WP developers about how they’ll make any money.
Funny hat tho’ ;-)
Summary
An ace time, basically. I’d like to say hi to everyone I met, but I’m scared of missing someone – so instead, let’s just say I look forward to chatting and, hopefully, working with some of you in the not so distant future.
MS have, at last, come up with what appears to be a competent rival to Google. Here’s five ways in which it beats Google.
Microsoft (MS), quite frankly, gets a lot of grief in the internet world. Sometimes it’s fair (I never like MSN, for example, from way back in the mid nineties) and often a little unfair.
But Live Search simply wasn’t up to the job. It didn’t work well. And I know that people that found IE defaulting to it would either work out how to change it, or simply type Google.com into the address bar. In other words, many tried it, but it didn’t find the answers they wanted. The algorithm has been slowly improved with time, but the damage was done. MS knew they had to relaunch.
Bing, they felt, was the answer. And in some ways, it’s a better and more productive tool than Google:
Bing - pretty pictures to cheer you up
1. It’s Prettier
While I’ve heard many question the function of the landing page photo, I personally really like it. It’s attractive, well designed, and brings a little bit of beauty into the day. You can’t sit and surf pretty images at work, so if they’re there as part of the ‘wallpaper’ of a daily tool then that’s a lift we all need.
2. Infinite Image Search
The infinite scroll facility of the image search makes it a quicker tool to use. Chunking of text related searches makes sense, because we can scan a page relatively slowly, but with images the human eye can scan a huge amount of visual information incredibly quickly which means that Bing’s constantly scrolling visual tool is way ahead of Google’s image search.
3. Video Previewing on Video Search
Bing Video - content owner's nightmare or benefit?
Searching for video content can often be a slow and painful process. In Bing, when you get a series of videos up on screen you can simply hover your mouse pointer over a video to preview the first 30s and get a feel for the video, rather than visiting the site and waiting for a slow load. The previews are poor quality, in order to get quick loading, but they’re good enough. I feel this is one of Bing’s most effective innovations.
One thing where they may struggle is that if you click the video and that video has an embed option, you get it on the Bing site, rather than going through to the source site. So a YouTube video search result doesn’t send you off to YouTube. Content owners may not like this.
4. Site Preview
When you hover over a search result, you’ll see a small orange marker appear over to the right. Hover over that and up pops a preview of the content you’re looking for. Again, saves a wasted visit as it lets you scan a little bit of content for relevance – something that’s quicker this way than clicking on yet another unnecessary site.
5. It’s Not Google
Bing is, purportedly, a recursive acronym that means Bing Is Not Google. But there’s something important in that – Microsoft is a highly profitable, focussed company that has the resources to provide an alternative to Google. This is important – without solid competition Google will cease to innovate appropriately. MS suffered a similar fate on the desktop – they were too dominant and rivals couldn’t compete. Apple’s OS9 was dreadfully dated when sat next to a Windows machine of the same era, yet Windows had significant flaws. It’s only lately with Windows 7 that MS have really started to get their act together properly – because OSX finally gave it some decent competition in certain sectors.
Microsoft Seadragon, with it’s deep zoom and mobile capabilities, and Photosynth technologies could be tied into the image search, for example. As cheap processing power expands and more and more images are geotagged, this could form an astonishing visual search capability. A shame it won’t be coupled with Google Street View – imagine what that could be like?
Search is going to become more relevant and more powerful with time. Developers (our own Interconnect IT included) are busy creating a lot of powerful geocoded databases which will allow for some amazing mashups. If Google and MS start fighting for dominance in this space the opportunities for users and information suppliers are vast. Are you looking into it?
Ever needed to migrate a database to a new server or website (especially with WordPress and other PHP applications) and been stuck because when you do a search and replace some of the data seems to get corrupted?
Ever needed to migrate a database to a new server or website (especially with WordPress and other PHP applications) and been stuck because when you do a search and replace some of the data seems to get corrupted?
Serialized PHP Arrays Cause Problems
In PHP one of the easiest ways of storing an array in a database is to use the serialize function. Works a treat, but the downside is that you’re not storing data with a cross platform method. In many product development environments this would get you a stern talking to, but in the world of web development where deadlines are tight and betas are the norm, this seems to be overlooked somewhat.
So what we have are tables full of data that can’t be easily edited by hand. For example:
Say you had thousands of records like the one above, and the word ‘multiple’ needs to be changed to ‘happy’. Two bits would change – poll_multiplepolls would now read poll_happypolls and multiple_polls would read happy_polls. In both cases you would have three characters fewer to deal with.
Fine, you may think, but you can only do the change by hand because where it says s:18:"poll_multiplepolls" it now has to say s:15:"poll_happypolls" – see the difference? S18 spells out the length of the following string, and it has to be changed to s:15
I’ll say right now, that that was a pain. For simple arrays I wrote the straightforward PHP Serialization fixer code, which got me out of many a pickle – do the search and replace without worrying, and then run the script. Fixed about 90% of problems.
Multidimensional Array Problem
Sadly those 10% of problems left were a real pain. I needed something more robust. Something more powerful. And finally today it was a Bank Holiday in the UK – that means no phone calls… I could have a quiet day of coding and concentrate on the best solution to this problem.
What I’ve done is to write a database search and replace utility in PHP that scans through an entire database (so use with care!) which is designed for developers to use on database migrations. It’s definitely not what you’d call an end-user tool, though I may sanitize it at some point and turn it into an easy to use WordPress plugin. Thing is – this is dangerous code – sometimes I think it’s better to make it deliberately a bit tricky, don’t you?
It’s not that bad though – if you can manually install WordPress, you can easily configure the database connection settings.
What the code does is to look at the database, analyse the tables, columns and keys, and then starts reading through it. It will attempt to unserialize any data it finds, and if it succeeds it will modify that data then reserialize it and pop it back in the database where it found it. If it finds unserialized data it will still carry out the search and replace.
Use in WordPress
In most WordPress migrations you tend to have the primary problem of changing the domain name entries in content, settings and widgets – you simply need to put in the $search_for string the old domain address (including the http if it’s there) as seen on the database, and the new one into $replace_with. Then put this script onto your server, and run it by visiting it in your browser or inputting the appropriate command line – depending on your server configuration.
Other things you may want to check are for plugins or themes that have made the mistake of storing the full server path to the installation – cFormsII does this, for example. You will need to find out your old and new server paths and use those, in full, for another iteration of this script.
After less than a second of running, you should have a freshly edited database. It may take a little longer on slow or share hosting, or if you have a very large database, but on my laptop I can manage around 60,000 items of data per second.
I’ve just used the script to migrate, in its entirety, with content, settings, 87 widgets (yes, really!) and hundreds of images to my localhost server. It took moments, and the site is perfectly preserved.
BIG WARNING: I take no responsibility for what this code does to your data. Use it at your own risk. Test it. Be careful. OK? Here in the North we might describe the code as being as “Rough as a badger’s arse.” Never felt a badger’s arse, but I’ll take their word for it.
An interesting problem today – WordPress posting on this site was sloooooow. Some other back-end tasks were also incredibly slow.
Adding a post or page would take about 30 to 60 seconds. Unacceptable. I did the usual job of deactivating plugins, and even resorted to a different theme for a few minutes but no, it was all still very slow.
That’s when the work starts. On this server we haven’t opened up port 3600 to allow remote connection with mySQL Administrator, so instead it’s straight into phpMyAdmin.
The process is then pretty simple – for each table you have an Operations tab. Here you can see some information about the state of the table as well as quick links at the bottom to allow you to run various tools. The process I use is to backup the site then run a Repair then an Optimize on the _posts (typically tables are prefixed WP_) , _postmeta and _options tables.
Why the Slowdown?
Well, I didn’t bother investigating properly – without full server access it’s often hard to get full timings. You can put traces into code and look at logs, or you can very quickly run the Repair process. But what I do know is that databases can slow down for inserts more easily than for reads. This is usually because something’s become messed up in an Index, or the whole table’s become inefficiently organised over time. By repairing and optimizing you get to quickly tidy things up and restore the performance of the site.
For a minor site like this, it’s about the best approach to take. If you’re running something like Facebook you’ll probably want to investigate things properly – but then you’ll have the money to be able to do so as well!
CSS not refreshing in the browser can be a pain – Mark Jaquith has come up with an elegant solution to this for WordPress coders.
I saw this rather handy tip on CSS reloading on Mark Jaquith’s blog today. Such a simple approach, but a great one for when you’re making CSS changes and need browsers to pick up on it immediately, rather than some hours down the line.
Some changes are critical. Now you can version your css by hand, but you don’t have to.
CPU spikes were making my laptop less fun to use – a nice online guide and a quick bit of sleuthing with MS’s sysinternals tools and I soon had the answer. The laptop is now faster, quieter and has better battery life.
Excuse the title, but may as well make it easy to find. I’d been experiencing problems with performance, whilst networking, with my Dell XPS M1330.
And my problem was exactly the same. Same driver, same version – the B57nd60x 10.10.0.0 driver was gobbling up CPU at a frightening rate. However, although he’d reported the problem, at the time there was no solution and a new driver wasn’t available on the Dell website. A year later, the driver still isn’t available – the Dell driver is resolutely stuck at v 10.10.0.0 – so, no fix.
Excessive CPU consumption = poor battery life
But it’s annoying seeing your CPU running constantly at 20%. It also has an impact on battery life.
And it worked – the screenshot below shows the impact – the first third or so shows the CPU usage with the old driver, and then it drops dramatically:
That Dell haven’t updated their driver pack in over a year is something of a support fail – it makes the XPS M1330, at least in certain circumstances, somewhat less of a great PC than it could be. And it’s also poor for the PC community – a lot of criticism is made of Windows being something that slows down over time. It’s rarely the fault of Microsoft – often it’s driver issues, but finding the latest drivers isn’t easy for everyone, and it’s quite technical to solve. MS could possibly make driver management a simpler system, but the PC makers could help by keeping up to date driver packs – especially for laptops which are rarely modified.
