Finding the Needle.txt in the NetApp Haystack Posted on Aug 19th, 2019 by Brian Sietsma

Categories: DPX, NetApp, RestoreManager

On a recent business trip, my Account Executive and I made a pitstop in the local NetApp office. Our purpose was to discuss our current product offerings and see how Catalogic Software’s products can integrate with and enhance NetApp’s stickiness in customer environments. Because as great as NetApp Storage is, there are always gaps that need to be filled. As we talked, one such gap became clear. NetApp is known for its snapshot capabilities. You can apply snapshot schedules, you can replicate to a SnapVault or SnapMirror, even push to an Altavault location. However, as you build out this architecture, it becomes exceedingly difficult to know what data lives where, and on what snapshots.

This issue usually presents itself when IT admins receive a simple request: “I lost one of my files, it must have been deleted accidently and I need it back.” Maybe the user accidently deleted the file, or overwrote a previous version by mistake, or maybe he/she just moved the file to the wrong sub-directory without realizing it. Sounds easy enough, but for IT Admins, this common request could lead to major headaches and hours of lost time, because finding the file is like searching for a needle in a haystack. If you are effectively utilizing NetApp’s snapshot capabilities, you know that you have multiple versions of that file in your snapshots, but there is no easy way to find that file. There is no central catalog that lets you search for and restore specific files. The Admin ends up spending hours manually searching through directories on Snapshots for that missing file. Or sometimes organizations simply deny the restore request because they don’t have the time to go looking.

As you can imagine, this problem is concerning to many admins. Maybe even concerning enough for them to begin to consider alternatives to NetApp that do a better job here. That is why, as we discussed this topic in the local NetApp office, the NetApp sales reps began to show their excitement when we brought up our RestoreManager product.

RestoreManager is a virtual appliance that communicates directly with your NetApp cluster(s), and using NetApp SnapDiff APIs lets you easily locate and recover files on NetApp storage copies. Using what we call a Crawler, RestoreManager collects the SnapDiff API information and with that data builds an online central catalog of your metadata for primary and/or secondary NetApp systems. This means that we also index your SnapMirror and SnapVault destination volumes. Then, because we have all of that information stored locally in an ElasticSearch database, you can search through snapshots using multiple filters and wildcards. When you find the file(s) or folder(s), you drive the restore right from within RestoreManager. The ElasticSearch database also allows RestoreManager to be highly scalable for large NetApp file environments.

Once you search for and identify the specific file that you need to restore, you simply select the file and click the “Restore File” button. This then gives the user the ability to restore the file from any snapshot that contains that file. Say for example you accidently overwrote a file and didn’t realize you did so. Perhaps several days go by and your Snapshot schedule has gone through a few cycles. With RestoreManager, you can restore the file using a snapshot that was taken before the inadvertent overwrite occurred.

The added benefit of the SnapDiff metadata collection is that all file and snapshot information is included in the crawl. This allows end-users the ability to apply filters for more targeted searches. Possible filters include:

  • By file name, parts of the name or file path, with wildcards being permitted
  • By data type or file ending: jpg, xls, doc, ppt, etc.
  • By deletion period
  • By creation period
  • By file size
  • Etc.

The basic search capability is great, but for large NetApp environments, these filters give the user the ability to narrow the focus to more easily find the specific file you are looking for. The extensive catalog also gives RestoreManager the ability to provide additional data analytics and reporting capabilities in the Kibana Dashboards module. Analytics and reporting will be covered in an upcoming post.

Using RestoreManager in conjunction with Catalogic Software’s DPX backup product allows for a complete file recovery solution. Because of its local, online catalog, RestoreManager is perfect for short term retention and operational recovery, including all the examples described earlier. But perhaps your organization would like to offload to tape or to the cloud for long-term retention. That data no longer lives on NetApp storage, and in turn cannot be restored using RestoreManager. That is where DPX comes in. Using scheduled and managed NDMP backups of your NetApp filer, DPX can push your data to tape or to the cloud, and then provide a searchable file index that features single file restore capabilities. This is particularly useful for data compliance or legal situations like an audit. See example below:

Using scheduled and managed NDMP backups of your NetApp filer, DPX can push your data to tape or to the cloud

You would be glad to know that licensing for RestoreManager is simple and affordable. It is priced per controller node and is tiered by NetApp system size. It does not matter how much data is stored, how many users you have, or the number of files. Licensing is subscription-based and is priced per month with a minimum 12-month purchase, with additional licensing available for any additional months on top of that first year. If you are interested in learning more about RestoreManager, visit our RestoreManager web page.

If you would like to see RestoreManager in action, watch a pre-recorded product demonstration. Or even better, contact us for a one-on-one demo with one of our product experts. You can even download a trial copy and test it for yourself.

Stop spending hours manually searching through all of your snapshots just to find that single file that someone accidently deleted! Instead, reach out to Catalogic Software and allow RestoreManager to search for that needle in a haystack for you.