Seamlessly Refresh OneTick Databases with Catalogic ECX Posted on Sep 30th, 2019 by Brian Sietsma

Categories: ECX

OneTick is a time series database that is primarily used for providing both historical and real-time services, including complex event processing and historical tick data analysis. OneTick is designed for the capture, storage, retrieval and analysis of data and needs to be able to handle inbound data at an extremely high-frequency (oftentimes with milli-, micro-, or nano-second granularity). Because of this, OneTick is widely used in the Finance world to improve trading performance through superior data management. OneTick users include hedge funds, proprietary trading firms, banks, brokers, technology and information providers, universities and more. Use cases include backtesting and quantitative research, high frequency trading signal generation, venue analysis, risk and portfolio analytics, and other generic time series analysis.

Dealing with this type of high-frequency data presents a problem when it comes to development and testing. If you have a need where multiple developers require a dedicated instance of the OneTick database to develop against, it becomes very difficult to present fresh data for these developers. Because the OneTick application does not support NAS for the data storage, it is not possible to share tick data across multiple environments. The workaround, which would include manually copying and mounting the drive containing the OneTick database to each individual development server, would take a considerable amount of time (depending on the size of that OneTick database), and by the time the database is available for development, the data is stale.

With Catalogic Software's ECX solution, this process becomes simple. ECX is a virtual appliance that manages your existing storage (Pure Storage, NetApp, or IBM Spectrum Virtualize and Accelerate) and provides the ability to automate and orchestrate application consistent storage snapshots. Most ECX customers use ECX to help with SQL or Oracle use-cases like database refreshes, Test-Dev, DevOps, or even DR, but ECX is also able to perform backup and recovery of InterSystems Caché or SAP HANA databases and VMs in VMware environments.

So how does ECX simplify the OneTick pain point discussed earlier? In addition to the ability to backup and restore individual VMs, ECX can also perform what we call an “instant disk restore.” This means that using the storage snapshots that ECX automates, it can instantly mount a cloned drive from one VM to another. When ECX performs a backup job of a particular VM, it is essentially taking a snapshot of the storage volume that the VM sits on. Then, during an instant disk restore, the end user selects the drive that contains the OneTick database, and ECX clones the volume and then mounts that clone onto the destination server. To make it even better, because ECX is working with cloned storage volumes, the end user can create 2… 5… 10… 25 copies of that drive and it will not require additional local disk space, because they are considered zero-footprint copies. Typically, mounting a 1TB drive onto 5 different servers would require 5TB of local storage. This is not the case when working with ECX. The “Disk Restore” is also extremely fast, because ECX is not mounting an entirely new version of the database, it is instead simply mounting a clone from a snapshot.

One particular ECX user, who is using ECX for this OneTick use-case, is using scheduled restore jobs to automate daily refreshes of the OneTick database onto a half-dozen Test-Dev servers. As part of these scheduled restore jobs, they are also taking advantage of ECX's “allow to overwrite and force clean-up of pending old session” feature. This eliminates any need for manual cleanup, etc. because this feature tells ECX to automatically clean up the previously mounted version of the drive (dismount the disk, delete the cloned volume from storage), and simply replace it with the most recent version of production data for testing. Now, on a daily basis, their developers arrive at the office with a fresh version of the OneTick database waiting for them.

If you or someone in your organization is experiencing the pain of providing instances of your OneTick database to your development team for testing, and want to know more about how Catalogic Software's ECX can help, check out our website.

You can request a live demo, or even get a 30-day trial copy to try it yourself. We'll be happy to help you get things set up.