A few years ago, Black Friday, the discount fever dream, meant frenzied shoppers sparring over bargain television sets in-store. However, increasingly, the mania has migrated online, with a staggering 1000% increase in internet searches for Black Friday deals this year compared to last, according to research conducted by MyVoucherCodes.
This means more business for MetaPack, the leading provider of eCommerce delivery management technology - acting as a bridge between 465 carriers and 5,000 delivery services. Among the services MetaPack offers to clients is identifying the most appropriate delivery service for a particular retailer, managing the delivery experience, and taking on some of the otherwise time consuming processes in the retailer’s warehouse.
Over the Black Friday weekend and Cyber Monday, the company's workload kicks into hyperdrive, with transaction volumes ‘several times higher’ than during a normal period. CIO UK spoke to MetaPack CIO, Steve Homan, about how the company handles this intense time period.
“We get multiple tracking events for all the parcels that come through, and we'll process about 600 million parcels annually and 13 million over the Black Friday weekend," says Homan. "Pretty much, if you buy something online not from Amazon, Metapack probably played a role in it."
For Metapack, Black Friday kicks off an extremely intense period stretching for between 5-6 days, which forms part of a longer intense period that can last until mid-February, following the festive period and January sales. It’s a time that for Metapack, becomes more frenetic each year as the business and customer base grows.
A major demand of the Black Friday period is preparation in terms of technology scaling. “We look at the forecasts from our retailers, and then add a bunch on, because A) they may sell more, and B) we want a contingency,” says Homan.
The key metric is the volume of transactions the system can handle. “We'll work that out into number of transactions per second, and we'll model that across our estate, and then we'll start to understand, 'are we comfortable that we're hitting all the right levels'?” Homan explains.
This involves examining any potential scaling limitations, such as those set by big technology vendors. This also involves working around big retailers' 'freeze periods' - where no material changes will be made to the development process, in terms of engineering.
But how do they perfect the systems in place to handle the surge? "We have a big philosophy about empowering our engineering team to solve a problem," says Homan. "So we give them a problem to solve, and then we give them a degree of flexibility in making it happen."
He says they work with a 'build it, ship it, run it' model, meaning that the team has the freedom to run what they build. However, accountability means that they must been on call to ensure all is running smoothly. "If you're a developer who builds something that goes wrong at two in the morning, they get out of bed and fix it," says Homan. Systems are constantly monitored for error, and if there is a flaw, this is immediately flagged through PagerDuty.
These concerns then flow up the food chain in a cascading escalation. "Eventually, if we get to a really serious problem, which very rarely happens, my phone starts to ring," says Homan.
But this is not sufficient for the couple of weeks spanning Black Friday and the festive run-up. During this time, members of the senior tech team, including Homan, are on-site 24/7. "It's really good because it means we're really living what we're doing," he says, adding that during this time it's important to make the team feel valued, by, for example, upping the calibre of snacks on offer.
Homan says they'll also offer employees compensatory perks if they've had to spend a Saturday in the office like a meal with a partner, or a family day out. "It's always the simple things that people like," he says.
For Homan, the drive is to achieve periods of relative calm, but circumventing the prospect of a serious failure. "We've got a phrase, 'boring Mondays', which is our mantra," he says. "We just love a boring Monday where we've had a weekend, nothings happened, no-ones had to do anything, and it's all been quite relaxed."
He says ideally, they'd like every day to be reminiscent of a 'Boring Monday'. "We're spending tons of time, to chase down every detail to make sure everything's permanently fixed so it never goes wrong again," he says.
However of course, this can't always be the case, with incidents arising along the way. "One of the major challenges is developing something in testing and then when exposed to real world volume, you realise that something's not working," says Homan. "Then it's like having to find needle in the haystack to discover what that is."
But despite the frenzy, Homan still has a favourable view of the Black Friday period. "I think it's really useful in a way that it marks a lot of homework from a technology perspective," he says. "It creates this really unique period of 'what's the most extreme you could do?' and radically applying that back into the business, which is really interesting."
It's one of the features that marks MetaPack apart. "It's so rare you'd get that in other businesses, obviously the peak might be 20 or 30% more, not by several factors," says Homan.