Salesforce – Data Archival Needs and Strategies
Salesforce being a cloud based multi-tenant application needs to closely monitor its resource usage. Be it server processing time, transactions or data storage, all resources need to be monitored for justified usage. One of the essential limit is data storage limit.
What happens if Data storage limit runs out?
Well, Salesforce just doesn’t immediately stop users to insert new data if data storage reaches 100%. Salesforce allows users to continue using their applications and create/ update data up to a grace limit (usually around 110%, but may vary). Once the grace limit is breached, Salesforce will not allow users to perform any create/ update operation and throw errors if users try to do so.
Benefits of Data Archival
As we have established that data archival is essential in Salesforce, let’s understand what are benefits of implementing data archival for salesforce consultant California :-
- Optimum data storage usage– optimum use of data storage provided by salesforce
- Reduced cost– as we can keep reusing data storage for more relevant data and keep non relevant data in backup locations
- Consistent application performance– with growing data the application can also become slow and nonresponsive. Event standard functionalities like listview, reports can start exhibiting degraded performance. With appropriate data archival processes, only relevant data will be persisted within Salesforce, hence enabling better performance of listviews, search, queries, reports
- Compliance– many industries are governed by various compliance rules (for e.g. Insurance, Finance etc.) and need to adhere to all the defined data retention guidelines/ policies
Planning for disaster is planning to avoid it
It is very evident that an appropriate data archival strategy needs to be designed and incorporated. Salesforce doesn’t profess any specific data archival mechanism, but recommends designing an appropriate data purge mechanism.
- BigObjects (Pilot):Salesforce has recently launched a new data storage mechanism called “BigObjects”. It is bulk data storage option within Database layer of salesforce. As per forums, this is an HBase data storage platform which means it’ll be able to handle huge loads of data (probably >100M records) and can be used for bulk data processing. However, at this stage there are limited details around this feature
- Cloud based archival platforms: with ever increasing demand for backup and archival needs, some major cloud players have provided data archival and backup platforms viz. Amazon Glacier, Google Cloud storage(Nearline) etc.
- Cloud based archival services:some innovative players have started offering cloud based backup and archival services BackupiFy, Spanning, OwnBackup Each product has myriads of features ranging from data comparison, snapshots, automatic backups, user based backups, one click restore etc.
- External On-Prem archival:often organizations may choose to utilize an in-house tool to manage backups and data archival (a database within local network)
- Scheduled backups: Now, this is included purely for informational purposes. Salesforce provides administrators ability to get backup of their Salesforce data (can choose specific salesforce objects). This can come handy for ad-hoc backup needs without using any third party tool. Once data backup processing is completed, Salesforce stores data backup files in a shared FTP location and emails FTP location to administrator
- Manual data backup via Salesforce: Administrators can manually download data from Salesforce using Apex dataloader or any other third-party tools
- Archival Custom object (least preferred):Create a custom object to store archived data. The only benefit this approach provides is ability to improve performance, as it keeps a check of data rows in the main object
In this way salesforce consultant can monitor the benefits of data archival thoroughly. Irrespective of what approach you finalize for your organization, identifying one is very important. Not planning for data archival in a cloud based SaaS app is like waiting for the disaster to happen. The biggest problem for data archival is that it is mostly underestimated. Once, data size becomes huge, it is quite difficult to ideate and implement an appropriate data archival strategy, even further problematic to test it.