If you aren't convinced that having your build definition as code rather than on a server is better, IntelliSense and Azure DevOps might change your mind.
Look at an overview of streaming concepts and learn how to ingest Kafka Events with Spark Structured Streaming and enrich events with a machine learning model.