InfinSnap for S3
InfinSnap for S3 enhances S3 with snapshot capabilities. Most applications treat S3 like a hierarchical file system and read multiple objects from S3, e.g. read all files in a directory for a machine learning training run, or an analytics query. In these types of uses, bucket wide snapshots are much more valuable than S3 object versioning.
Benefits of InfinSnap for S3
- Consistent snapshot view of data for applications including ML Training
- Ingest programs can continue to create objects in the bucket – read applications see a consistent view
- Reproducible ML Training is finally possible, because you can re-run the training against the same snapshot
- Test multiple models against the same snapshot of data to quantify model performance
- ML Platforms such as ML Flow and SageMaker Experiments currently store hyper-parameters and results of experiments as the metadata for each experiment. The addition of InfinSnap snapshot id to this metadata completes the metadata and makes reproducible ML experiments feasible
InfinSnap for S3 is offered as a SaaS offering on AWS Marketplace. It is currently available free of charge. Simply sign up and one-click enable InfinSnap for your S3 bucket. The following screen capture video demonstrates the AWS Marketplace subscription, InfinStor service account signup and InfinStor service configuration by creating IAM Role.
InfinSnap for S3 Demo
Demonstrates the power of InfinSnap for S3, bucketwide fine grained snapshots of S3 buckets. A file called testfile with content ‘contentA’ is uploaded to the bucket, then deleted, and then another file by the same name testfile with different content contentX is uploaded to the same bucket. We then browse the bucket at different points in time and look at the state of the bucket with no file present, testfile present with contentA, no file present again, and finally testfile present with contentB. InfinSnap for S3 provides ultimate flexibility for managing content in S3 by adding snapshot capability.