When I joined Y-Proximité, after picking up the many warning signs described in this series of posts, I felt the need to start migrating our multi-tenant (1 DB = X customers), white-label “CMS” application built for site networks. Today it hosts about 2,000 sites/landing pages.
You can read in some places that a rewrite is evil. I decided to tell the story of our migration (politically less abrasive than a rewrite), which took 2 years. Here is my logbook of the migration to “Yprox v2”, covering several topics.
- I. Migration: The reasons
- II. Migration: The continuous migration process
- III. Migration: Taking stock
What we gained from the migration
A “better” structure, introducing microservices
Click for an expanded version of the architecture
I want to stress that this was a migration: the code/logic was not rewritten “from scratch”, since the core already stood on a solid foundation.
That said, many modules went through major refactoring and deep restructuring. In the end, here is the before/after comparison of the number of lines of code (LOC), as reported by the cloc tool.
| Language | V1 | V2 |
|---|---|---|
| PHP | 138,000 | 75,000 |
| JS | 312,000 | 20,000 |
| HTML+TWIG | 182,000 | 25,000 |
This comes down to:
- Moving the JS libraries to NPM, and publishing them with Gulp
- Refactoring the PHP business modules
- Removing dead code
- Dropping features that cost far too much to maintain compared to how much they were used (e.g. a “homemade” e-commerce module)
Some business tasks were moved out to other repositories. For some of them, we have a “queue” and “worker” system (Silex, running on Docker) at Iron.io that performs those tasks outside our infrastructure.
Functional and unit tests
We increased our test coverage with unit tests, class tests (60 specs, 368 PHPSpec examples) and functional tests (449 Behat scenarios).
Between you and me, quoting coverage figures is meaningless if the tests are badly designed, but it gives you a baseline to “reassure” yourself. We’re still a long way from what Sylius does on the testing front. To back us up, we upgraded Travis to get 5 parallel builds.
A more “modern” design
Click to enlarge
Granted, it’s an admin template bought on a template marketplace, but it does the job. Our designer & front-end team, swamped as well, were barely involved, which we sometimes regretted.
Migration = fallow period? Not necessarily.
Although it held back some of our internal users’ requests, more than 20 new features saw the light of day during the migration:
- New business modules
- Modules rewritten in Vue.js, with new features
- A REST API consumed by partners
- HTTP2 support for the back office
Be warned: keeping the product moving during a migration (+ bug fixes) comes at a significant cost in migration time. We had to keep up a sometimes intense pace for 2 years (starting 05/2014), with about 3 to 4 full-time people depending on the period.
User resistance: beware of contagion.
Over 2 years, we “lost” the buy-in of some internal users, who wonder “why this migration?” and “is it really necessary from a user’s point of view?”. Resistance to change deserves a post of its own, and this won’t be it ;-). Make sure these vocal criticisms/fears don’t spread to your development team, or the team will start doubting the real value of its work too. Find early adopters who won’t see the change as “imposed” and “useless”.
Although we did run workshops, support and education for our internal users should have been pursued with much more energy, so we could be transparent about the migration’s progress (hard to estimate, since we were also maintaining the existing product and adding new features).
We lacked a Product Owner to gather user feedback and improvement ideas for the tool we were reworking. Users sometimes felt a bit ignored, given the size of the JIRA backlog (>320 tickets, the “it’s never going to get done anyway” syndrome).
Conclusion
Only embark on this kind of heavy migration (130,000 lines of PHP code) when you have the backing of the business teams on the ground and of the existing team. Otherwise, you’ll run into walls that may prove fatal to such a long-haul effort.
NB: this is an early review, made possible by continuous migration. As I write these lines, we are validating a final environment where everything has been migrated (front and back).
NB2: I have deliberately left out everything we introduced on the side (industrialized development, business metrics).