Magento enterprise upgrade

A backend point of view

So, a client approaches you and asks, ‚Äòcan we update our Magento store?’

Obviously, you immediately answer with a ‚Äúyes‚Äù. Not because they are a long-standing, devoted client or because you really like them, but because it is the right thing to do, to keep all your stores up to date with the latest versions. There will be security updates, new features, an improved loading speed, and more on the latest version of Magento that you and your client can both benefit from upgrading to. Since April the 21st this year Google made some very large changes to the way their search engine favours responsive sites (and more; read here), they hold the monopoly, plus who is to say that the other search engines won’t follow suit? So let’s get everything upgraded and updated, everything from the platform to the design responsiveness.

The background

We’ve been working with one of our larger clients for the last 4 years and we’ve grown quite fond of this project, we always do. For the more junior developers (now fully fledged) this project was a learning experience. For others, a chance to master their craft. We enjoy working on this project as much as the rest, building different ‚Äúbits and bobs‚Äù that are helping the staff do what they do best: be one of the UK’s top retailers.

The store was sitting happily on Magento 1.10 Enterprise Edition. Along the years we had added a lot of features and optimisations. It’s been a journey and it was time for a new path.

The challenge

When we started this new project, Magento 1.14.0.1 was very recently released and after some talks with the client we decided that we would upgrade.

Any upgrade on any platform can be scary, especially when you have lovingly added modules and custom built add-ons to your wonderful Magento site. Also, the bigger the “gap” between the two versions is, the greater the possible risk. Even in the “middle”, around versions 1.12 and 1.13 there were a lot of changes at the core level, especially with the indexers and with the URLs. Putting any generalised fears aside, we were confident that we could complete the Magento Enterprise upgrade without any problems.

The road

One of the first things we had to do was to go through all the modules installed and assess which ones are compatible with the new version, which ones we have to upgrade, if we need to find some replacements, or to build them (we quite often chose the latter, building in-house what was now obsolete). Then we had to carefully plan out the process.

We tested the upgrade with repeated trial and error on a Vagrant simulated environment, we routinely checked the variables, what was working and what was not, going back and refining the plan. If you continue reading on, you can read the 15 steps which we then followed to complete the upgrade on the live site.

Here comes the technical bit:

The final “map” we created was something like this:

• 1. Upload a landing page on the live site.
  In other words, take the site down for the upgrade. We built a nice and clean landing page with the client’s logo, their contact details and a message apologising for the disruption of services to the customers inviting them to come back in a few hours to visit the brand new website.
  The page has a double-role actually: to prevent visitors from triggering the upgrade scripts and to prevent any new orders from being lost from the upgraded version.
• 2. Disable the crons.
  The project has some other cron jobs, except the default Magento one: products feeds generation, 3rd party sales integration, etc. Obviously, during the upgrade everything had to be turned off as early in the process as possible.
• 3. Increase the timeouts for nginx and php-fpm
  This is really important, by default there is a timeout in the communication between php-fpm and nginx of 30 seconds. This setting, along with the php’s ‚Äúmax_execution_time‚Äù makes it impossible for a page to run for more than 30 seconds. While this is very good/decent setting for a live environment, is a really bad idea for an upgrade of that magnitude;
  Now, the new value is more of a guessing game. It depends on how big the catalog is, how many orders/quotes are in the DB, CMS pages, settings, etc. For this project we went with 2 hours (7200 seconds).
  So, we’ve added the proxy_read_timeout and fastcgi_read_timeout settings in the nginx conf file (in our Vagrant box it was in /etc/nginx/conf.d/default.conf), set them to 7200, and the request_terminate_timeout (in /etc/php-fpm.d/www.conf for php-fpm), set it to 7200s as well and restarted the services (service nginx restart and service php-fpm restart).
  Without these changes, the upgrade starts when you ‚Äúhit‚Äù your site in the browser and your browser will return a ‚Äú502/Bad Gateway‚Äù message, if you’re on nginx/php-fpm.
• 4. Export the live database and import it on the new location.
  We had a new location on the same server for the upgraded project. To save time, we already had the files deployed in place along with the live site from before, the only thing left to be done was to upgrade the database. So, as the next step, was to export the live database and to import it in a separate one, used by the upgraded project.
• 5. Disable the cache.
  Since it wasn’t possible to do this from the admin without triggering the upgrade, we ran a SQL query against the database: ‚ÄúUPDATE core_cache_option SET value=0‚Äù.
  Also, we had to make sure the directories var/cache and var/full_page_cache are empty.
• 6. Disable Solr search.
  Along with the upgrade, the client wanted to implement a 3rd party for search that is not fully compatible with the “solr” search engine, so we had to turn this off.
• 7. Trigger the upgrade scripts.
  This is just a fancy name for “visit the new site in your browser”. What actually happens, is the new Magento files figure out that the database is an old one, by checking its core modules versions against the core_resource table and it starts to run the upgrade scripts from the version in core_config, to bring everything up to date with the version declared in app/code/core/Mage_or_Enterprise/Module/etc/config.xml. Of course, this is the most time-consuming part of the upgrade. This IS the actual upgrade.
• 8. Run the URL migration script.
  Almost two hours later after step 7…
  Because the upgrade was from a version older than 1.13, Magento provides some URLs migration scripts, placed in the shell/ directory. So, before doing the usual “cache refresh and reindex”, we ran the ./url_migration_to_1_13.php script that took care of the redirects that were in place on the 1.10 project.
• 9. Enable canonical URLs.
  Although the setting was there in the previous version, we used a 3rd party module for that. The main reason for that was because the client needed some things to work a bit different than the default Magento ones. But with this upgrade, we built our own module to extend the default Magento functionality to suit the client needs.
• 10. Enable the basket pop-up module.
  That’s the ‚Äúinternal‚Äù name we have for a module that doesn’t actually redirects you to the cart page (or back to the product page, depending on the settings) when you click the ‚Äúadd to cart‚Äù button for a product. A modal window ‚Äúpops-up‚Äù (hence the name) that informs the customer that the desired product was added to basket, present them with some related products (that’s configurable from the admin, between related, cross-sells and no products) and offers them the possibility to go directly to the basket page or continue shopping.
• 11. Trigger full reindex.
  This is an obvious next step, after such a big upgrade, right? 🙂
• 12. Change back the timeouts in nginx / php-fpm
  Set back the values to 30s in both services.
• 13. Re-enable the crons.
  Remember when we disabled the cron jobs on step 2? Now it’s time to re-enable them and make sure they reflect the new directory the project is deployed.
• 14. Enable access for the client.
  Sent the “green light” to the customer to visit the newly upgraded website on the temporary URL and check if everything is works as expected and do their changes for CMS pages/static blocks, if any.
• 15. Set the new project live.
  Changed the urls in the core_config_data table for the web/unsecure/base_url and web/unsecure/base_url paths with the live site’s URL. Enable the cache.

Once step 14 was done, the site was live and we could see visitors coming on the website by checking the nginx access log. It wasn’t long after and the first successful transaction went through just fine. That is the main goal of an eCommerce website, to actually sell stuff.

Please note that the above steps are stripped down, there’s no need to go into detail with the ‚Äúin-between‚Äù steps of our internal workflow, or custom builds, each project will be unique in its own way.

Final notes

After the site was live, we had to tweak some Redis cache engine settings, but this sounds like a good reason for another article. The site now performs very well, having response times (we’re using NewRelic for monitoring) of (on average) 300ms for 300rpm. For a Magento with a catalog of that size, that is something.

For any more details, questions or to chat, why not drop me a message in the comments below or call us in the office!