To optimise the performenc , after result validation, it is better let the intermediate output format as avro, need to do the setting in myenv.yaml
outputDefinition:
type: "avro"
disableParallelism: false
To optimise the performenc , after result validation, it is better let the intermediate output format as avro, need to do the setting in myenv.yaml
outputDefinition:
type: "avro"
disableParallelism: false