Garnet: A new Remote Caching solution

by

Introduction to Garnet

Garnet is Microsoft’s innovative response to the needs of modern remote caching. This tool reflects a significant leap in performance and efficiency, taking inspiration from the ShadowFax research on high-throughput, low-latency distributed key-value stores. Also, after Redis’s recent licensing changes and demand for an even more efficient low-latency and high-throughput caching platform, this was required!

My professional experience has revealed challenges in managing Redis’s replication and clustering, especially outside cloud-based solutions. Given its compatibility with Redis clients, Garnet presents a compelling option for such environments.

API Compatibility

Garnet has a new thread-scalable system architecture. They designed network, processing, and even storage that supports in-memory, disk, and cloud, all from scratch. It uses RESP API to achieve the highest compatibility with Redis-based codebases. Garnet is not intended to be a 100% perfect drop-in replacement for Redis, rather it should be regarded as a close-enough starting point for you to ensure compatibility for features that matter to you. Garnet does work unmodified with many Redis clients (compatible with StackExchange.Redis very well), so getting started is very easy.

Differences you can expect when using Garnet:

  1. It is multi-threaded, ao MSET is not atomic by default. For an atomic version of MSET, you would need to express it as a transaction (stored procedure).
  2. It does not support the Redis functions or modules. Instead, it has its own C# based extensibility mechanisms that are optimized for high performance and ease of use.
  3. Garnet does not support Lua scripting. They are working to make a fast and reliable adoption of it.
  4. Garnet respects the FIFO ordering of request-responses. However, when used with larger-than-memory data, and if you opt in to using the scatter-gather version of IO (using the EnableScatterGatherGet [–sg-get] option) for increased disk performance, then even though results are still returned in FIFO order, the read operations may be executed out-of-order to earlier write operations in the same input operation sequence.
  5. When Garnet is used with append-only-file (AOF) turned on, by default the server does not wait for commit before returning success to the user. This can be adjusted using the WaitForCommit [–aof-commit-wait] option, while the frequency of commit can be tuned using the CommitFrequencyMs [–aof-commit-freq] option.
  6. You can disable the object store if your workload only consists of raw string operations, using the option DisableObjects [–no-obj]. The storage tier is disabled by default, and you can enable it using EnableStorageTier [–storage-tier]. You can disable the pub-sub feature using the option DisablePubSub [–no-pubsub].

Garnet’s Core Innovations Inspired by ShadowFax

1. Optimized Network Layer:

  • Shared Memory Network Design: Drawing from ShadowFax, Garnet implements a shared memory model in its network layer. This design allows for more efficient handling of data, as it reduces the need for copying data between different memory areas, thus improving throughput and reducing latency.
  • Efficient TLS and Storage Interactions: Garnet optimizes TLS processing by handling it directly on the IO completion thread. This approach minimizes thread switching overheads, a common bottleneck in network-intensive applications. By doing so, Garnet can keep the data closer to the network, leveraging CPU cache coherence effectively.

2. Advanced Storage Design:

  • Two-Tier Tsavorite Storage Architecture: Garnet features a dual Tsavorite storage model. The ‘main store’ is finely tuned for high-speed string operations, utilizing efficient memory management techniques to reduce garbage collection overheads. This leads to a more efficient use of available memory and reduces latency in data access.
  • Versatile Object Store: In addition to the main store, Garnet includes an ‘object store’ designed for managing complex objects and diverse data types. This store leverages the robust .NET library ecosystem for implementation, providing a flexible platform for handling structures like sorted sets, hashes, lists, and geographical data efficiently.

3. Networking and Data Management Integration:

  • Seamless Network-Storage Interaction: Garnet’s architecture ensures a close integration between its network layer and storage components. The shared memory design not only benefits network throughput but also positively impacts the way data is handled and stored, ensuring that data movement across the server is optimized for speed and efficiency.

4. High Performance and Scalability:

  • Inherited ShadowFax Performance Attributes: Garnet inherits key performance attributes from ShadowFax, particularly in handling high volumes of operations per second and maintaining efficiency under high-load scenarios. Its architecture allows for scalability without significant degradation in performance, making it well-suited for large-scale, distributed environments.

Technical Specifications and Performance of Garnet

1. Advanced Networking Capabilities:

  • Shared Memory Inspired Network Layer: Garnet’s network layer, influenced by ShadowFax’s shared memory design, enhances throughput and minimizes latency. This efficient approach allows for rapid data processing and transfer, essential for high-performance caching.
  • Optimized TLS Processing: By executing TLS processing directly on the IO completion thread, Garnet reduces the overhead associated with thread switching. This leads to more efficient use of CPU resources and faster data encryption and decryption processes, crucial for secure data transmissions.

2. Innovative Storage Mechanism:

  • Two-Tiered Tsavorite Storage Design: Garnet employs a two-tiered Tsavorite storage architecture. The ‘main store’ is tailored for string operations, optimizing memory usage to reduce the impact of garbage collection. The ‘object store’ complements this by focusing on complex objects and varied data types, utilizing the .NET ecosystem for efficient data management and storage. This dual approach allows Garnet to handle a wide range of data structures with ease.

3. Enhanced Data Management Techniques:

  • RESP API and Tsavorite Storage API Integration: Garnet’s rich RESP API surface is built on top of a streamlined Tsavorite storage API. This combination enables Garnet to support a wide array of read, upsert, delete, and atomic read-modify-write operations, executed through asynchronous callbacks. This design provides Garnet the flexibility to insert logic efficiently at various points during each operation, ensuring a responsive and dynamic caching solution.
  • Advanced Concurrency Control: Garnet’s adoption of two-phase locking for handling multi-key transactions showcases its commitment to data integrity and consistency. This method ensures that all parts of a transaction are processed in a coordinated manner, maintaining database consistency without sacrificing performance.

4. Performance Metrics:

  • High Throughput and Low Latency: Inherited from ShadowFax, Garnet shows exceptional performance metrics, particularly in handling high throughput and low latency operations. This makes it particularly suitable for scenarios where rapid data access and modification are critical.
  • Scalability in High-load Environments: Garnet’s architecture and storage design allow it to maintain performance consistency even in high-load environments. Its ability to scale efficiently makes it a viable solution for large-scale applications requiring robust caching mechanisms.

Comparative Analysis: Garnet vs. Redis

1. Performance Metrics:

  • Throughput and Latency:Garnet, drawing on advances from ShadowFax research, boasts superior throughput and lower latency compared to Redis, especially under heavy loads. This is a result of its shared memory network design and efficient handling of TLS processing on the IO completion thread, which reduces thread-switching overheads. Redis, while performing well in many scenarios, may not match the high-volume data throughput and low-latency capabilities of Garnet in high-load situations.
Read more
  • Scalability: Garnet’s architecture allows for better scalability in distributed environments compared to Redis. Garnet’s dual Tsavorite storage design, combined with its efficient network layer, enables it to manage larger volumes of data and a higher number of concurrent operations more effectively than Redis.

2. Data Management and Storage:

  • Data Structures and Storage Flexibility: Garnet’s two-tiered storage approach, consisting of the main store for strings and an object store for complex data types, provides a more versatile data management solution compared to Redis. While Redis offers a variety of data structures (such as strings, hashes, lists, sets, and sorted sets), Garnet leverages the .NET library ecosystem to optimize storage and access for these complex data types.
  • Concurrency and Transaction Management: Garnet uses two-phase locking for multi-key transactions, ensuring data consistency and integrity. This approach contrasts with Redis, which primarily uses single-threaded processing. While Redis is efficient in its execution model, Garnet’s method offers enhanced concurrency control, beneficial for complex transaction scenarios.

3. Advanced Features and Extensions:

  • Custom Data Types and Operations: Garnet’s object store, optimized for complex objects and custom data types, allows for more advanced operations and data handling capabilities than Redis. This feature, combined with Garnet’s extensive RESP API surface, provides a broader range of functionalities.
  • Networking and Security: Garnet’s shared memory inspired network design offers advantages in data throughput and security processing over Redis. While Redis provides robust features, Garnet’s approach to integrating TLS processing and data transfer results in more efficient network communication.

4. Open Source and Community Involvement:

  • Licensing and Contributions: Both Garnet and Redis are open-source tools; however, Garnet’s adoption of the MIT License may offer a simpler and more straightforward framework for contributions and usage in various projects compared to Redis’ licensing structure.

Conclusion

Garnet represents a major stride forward in remote caching technologies. Its incorporation of ShadowFax-inspired technologies positions it as a powerful tool in data management, capable of handling the demanding requirements of modern applications.

Resources:

Leave a Comment