Klaw in 2022: simplifying Apache Kafka data governance

It hasn’t been long since the initial release of Klaw on 29 September 2022, but even in this short time we’ve worked to enhance the Apache Kafka data governance experience.

In the initial release, we aimed at keeping the technology consistent and resilient across the new environment instead of risking breaking things. Our conservative approach paid off as nothing, in fact, broke, and we achieved our priority objectives: name change and integration with different versions of Apache Kafka.

The next item on the agenda was to make sure that your data remains safe. That’s why the next release was all about security. We want to ensure that Apache Kafka governance is safe and authenticated everywhere. At Aiven, we’re proud to say your data is safe at rest and in motion, and we wanted to apply this high standard to Klaw as well.

Major new features

Klaw now includes SASL authentication protocols SCRAM-SHA-256 and SCRAM-SHA-512 for connecting securely to Apache Kafka clusters.

You can now use JWT-based authentication to connect to the Klaw Cluster API. This authentication between APIs makes the connection more secure.

You can now configure Klaw to connect to multiple clusters with different SSL certificates. This way, you can govern larger Apache Kafka systems (with Apache Kafka, Schema Registry/Karapace, Kafka Connect, etc.) with just one instance of Klaw. This hugely simplifies your Kafka governance, removing the need for multiple Klaw instances per component and ensuring a smoother developer and DevOps experience.

The Browse topics screen has a completely new look and feel after we switched to React for UI. This gives a much more user-friendly and intuitive interface for managing your Kafka topics.

Azure AD based Authentication and Authorization: Your organization can now leverage an existing Azure AD infrastructure to manage access to Klaw. You can also retrieve role and team information from Azure AD tokens.
Advanced topic configuration on Topic requests, allowing you to configure different topic options like compacted topic, apply retention, compression, and so on in the same request.
You can now create service accounts using new Aiven ACL requests (Aiven specific change)

Other improvements

Major improvements

• Improved documentation (in progress) and releases mentioned at https://klaw-project.io
• Github CI actions, Code of conduct, code styling and formatting plugins, security policy, contributing guidelines, templates on PRs and Issues, packaging and installation instructions
• Support for Apache Kafka flavors like Aiven for Apache Kafka is introduced
• Updated unit tests and integration tests on several service classes
• Code refactoring to make the code more robust and readable
• Better exception handling and enhancements

Minor improvements

• The ClusterApi git project has been moved into Klaw core project as a module
• Vulnerabilities related to Actuator, Cluster Api authentication, etc are addressed
• Springboot and Apache Kafka upgrade
• Introduced Custom validator on topic requests allowing the validation logic to be isolated and easily managed
• Introduced Makefile for easy build and deploy
• Deep linking of URLs in Klaw web
• Alerts on code scanning, secret scanning, Dependabot and security advisories
• Spring security upgrade, removed the deprecation of WebSecurityConfigurerAdapter
• Support for docker

What to expect in the next three months

We are only getting started with Klaw and we are excited about future additions to the framework! Over the next few months, our development efforts will evolve around upgrading the framework, adding to Klaw’s React UI, and introducing a lot of new features.

Here’s a more detailed list for your convenience:

Framework

• Spring boot upgrade to 3.x
• Upgrade to JDK 17

Features

• Schema Promotion from one environment to another
• Support for PEM format for Kafka cluster SSL connectivity
• Introduce RegEx patterns on topic names in topic requests
• Introduce a new request type: Reset consumer offsets
• React UI
• Topic Requests
• Subscription Requests (Producer/Consumer)
• Schema Requests
• Connect Requests

Our thanks

The two minor releases wouldn’t have been possible without the help of our contributors. We’d like to thank each and every one of them:

• Mirjam Aulbach
• Mathieu Anderson
• Samuli Suortti
• Murali Basani
• Aindriu Lavelle
• Sergey Nuyanzin
• Josep Prat
• Ahmed Sobeh
• Dave Beech
• Stefan Arentz
• Sebastian Kaiser
• Harshini Rangaswamy

Wrapping up

Klaw has come a long way in the months since its acquisition by Aiven, and we’ve already enhanced it with significant improvements. Keep an eye out for the next release!

In the meantime, here are some resources to browse:

• The Klaw project page
• Klaw documentation
• Klaw on GitHub

To get the latest news about Aiven and our services, plus a bit of extra around all things open source, subscribe to our monthly newsletter! Daily news about Aiven is available on our LinkedIn and Twitter feeds.

If you just want to find out about our service updates, follow our changelog.