apache/flink

This system is a distributed stream and batch processing engine that provides a comprehensive data processing framework. It features a modern DataStream V2 API and a robust Table/SQL API for defining complex data pipelines. The platform supports a wide array of connectors for file systems, databases, and external services, alongside state management, metrics, and testing utilities.

This period focused on deepening Hadoop ecosystem integration by implementing a comprehensive compatibility layer for MapRed and MapReduce APIs, alongside adding support for Hadoop Writable types. Concurrently, the project significantly expanded its observability by introducing multiple new metrics reporters for systems like Prometheus, Datadog, and InfluxDB, while also refactoring the client submission architecture and modernizing the codebase with JUnit 5 and improved build tooling.

80 changes

This period focused on modernizing the Table API with a new type inference framework and SQL parser overhaul, while simultaneously refactoring the File Source and Sink connectors to support new APIs and features like file compaction. The work also included significant improvements to the Kubernetes integration, Web Dashboard, and comprehensive test coverage for these new components.

63 changes

This period focused on migrating the RPC layer from Akka to Pekko and introducing a new ChangelogStateBackend for state management. The team also established stricter architectural constraints using ArchUnit and enhanced the SQL Gateway API, while simultaneously expanding test coverage across connectors and file systems.

61 changes

This period focused on the introduction of the DataStream V2 API, providing a new execution environment and context model for stream processing. It also expanded Flink's ecosystem with a new ForSt state backend, native S3 filesystem support, and OpenTelemetry metrics integration, alongside significant test coverage improvements for the new APIs.

15 changes

apache/flink was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point the surveyor at a repository you know and see whether you agree with it.