KAFKA - ERROR Failed to write meta.properties due to (kafka.server.BrokerMetadataCheckpoint)

 

I set up Kafka source code read environment on Windows10, and encourted with error below, so what is wrong, I am a fresh man to learn kafka

[2021-04-29 19:57:42,957] ERROR Failed to write meta.properties due to (kafka.server.BrokerMetadataCheckpoint)
java.nio.file.AccessDeniedException: C:\Users\a\workspace\kafka\logs
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
[2021-04-29 19:57:42,973] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.nio.file.AccessDeniedException: C:\Users\a\workspace\kafka\logs
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)

and another ERROR below

[2021-04-29 19:57:43,821] ERROR Error while writing to checkpoint file C:\Users\a\workspace\kafka\logs\recovery-point-offset-checkpoint (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: C:\Users\a\workspace\kafka\logs
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
[2021-04-29 19:57:43,823] ERROR Disk error while writing recovery offsets checkpoint in directory C:\Users\a\workspace\kafka\logs: Error while writing to checkpoint file C:\Users\a\workspace\kafka\logs\recovery-point-offset-checkpoint (kafka.log.LogManager)
[2021-04-29 19:57:43,833] ERROR Error while writing to checkpoint file C:\Users\a\workspace\kafka\logs\log-start-offset-checkpoint (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: C:\Users\a\workspace\kafka\logs
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
[2021-04-29 19:57:43,834] ERROR Disk error while writing log start offsets checkpoint in directory C:\Users\a\workspace\kafka\logs: Error while writing to checkpoint file C:\Users\a\workspace\kafka\logs\log-start-offset-checkpoint (kafka.log.LogManager)

I downgraded to the Kafka 2.8.1 version (Index of /kafka/2.8.1) and after that everything is working perfectly.

Share

Improve this answer

This is a common error when log retention happens.

Kafka doesn't have good support for windows filesystem.

You can use WSL2 or Docker to work around these limitations - 

This seems to be a bug affecting some Kafka versions. Try installing kafka_2.12-2.8.1.tgz version. It will solve your problem.

I have tried using Kafka kafka_2.12-3.0.0.tgz and the same issue occurred. Seems 2.12-3.0.0 have issues with windows.

Try to downgrade the version to kafka_2.12-2.8.1.tgz. This resolved all issues and working fine.