10x'ers: Evolutionary Leaps and Bounds
May 22, 2010
By Jeff Freund
Sometimes evolution comes in small incremental changes and other times it will be in leaps and bounds.
Back in the fall, we were working on a huge content import as part of a customer implementation. We were able to sustain content loading at over 10 times previous peak rates! There was a significant amount of work required to achieve this throughput - from parallelization of the data loading process on Amazon EC2, to reworking the procedure to prevent our search indexing servers from getting overloaded, to seamlessly adding new storage capacity.
This past week we did it again - we hit another "10x" moment. Our engineering team reworked some internals on our Lucene based search servers in pursuit of better operational stability. In the process, we ended up increasing the throughput of the search servers by a factor of 10! Each search server can now handle 10 times the traffic as before, and do it without any of the stability issues we have faced over time. Yippee!
Any time you create an order of magnitude change like these examples, it is a big deal. This is true in technology and any other area of the business. Just think about these what-ifs:
- What if you cut implementation times to 1/10th the time?
- What if your application was 10x faster for users to do tasks?
- What if you generated 10x the number of leads?
- What if you traffic increased 10x?
- What if your service was 1/10th the cost to deliver?
- What if your company was 10x as large?
Some of these may be far fetched, and others will simply occur as a business grows and evolves. But each one is certainly a game changer.
Evolutionary leaps are core to the concept of this blog - http://www.localpeaks.com/Local_Peaks.html - and a 10x'er like these may very well be that leap to the next higher peak.