spark stream java_使用Spark Java通过http流式传输视频文件

我正在尝试将视频文件流过静止,我正在尝试像这样实现类似于Jersey的东西:

ResponseBuilder builder = Response.ok(out.toByteArray());

builder.header("Content-Disposition", "attachment; filename=" + fields.get("filename"));

response = builder.build();

} else {

response = Response.status(404).

entity(" Unable to get file with ID: " + id).

type("text/plain").

build();

}

return response;

}

这是我的文件上传和下载/流媒体(下载半工作,文件大小已损坏):

我真的需要帮助这个人,谢谢

UPDATE

改变:

ByteArrayOutputStream out = new ByteArrayOutputStream();

至:

ServletOutputStream out = res.raw().getOutputStream();

更新2

好吧,我终于让它工作了,视频在浏览器中播放,但现在得到一个Jetty io.EofException,我关闭了流但是,仍然必须简单.

以下是前后:

并从浏览器下载文件,但如何直接在浏览器中流式传输?

之前(没工作)

//download a video/ trying to stream it right in the browser if possible

get("/post/:id", (req, res ) -> {

res.raw().setContentType("application/octet-stream");

String id = req.params(":id");

ObjectId objectId = new ObjectId(id);

BasicDBObject query = new BasicDBObject();

query.put("_id", objectId);

//DBObject video = collection.findOne(query);

GridFS gridfile = new GridFS(db, "videos");

GridFSDBFile gridFSDBFile = gridfile.findOne(query);

res.raw().setHeader("Content-Disposition", "attachment; filename=" + gridFSDBFile.getFilename());

InputStream inputStream = gridFSDBFile.getInputStream();

ServletOutputStream out = res.raw().getOutputStream();

// ByteArrayOutputStream out = new ByteArrayOutputStream();

int data = inputStream.read();

while (data >= 0) {

out.write((char) data);

data = inputStream.read();

}

out.flush();

out.close();

return out;

});

AFTER(这很好用,但是文件异常结束):

get("/post/:id", (req, res ) -> {

//what's the difference between these 2?

res.raw().setContentType("video/mp4");

res.type("video/mp4");

String id = req.params(":id");

ObjectId objectId = new ObjectId(id);

BasicDBObject query = new BasicDBObject();

query.put("_id", objectId);

GridFS gridfile = new GridFS(db, "videos");

GridFSDBFile gridFSDBFile = gridfile.findOne(query);

res.raw().setContentLengthLong(gridFSDBFile.getLength());

InputStream inputStream = gridFSDBFile.getInputStream();

ServletOutputStream out = res.raw().getOutputStream();

int data = inputStream.read();

while (data >= 0) {

gridFSDBFile.writeTo(out);

data = inputStream.read();

}

// out.flush();

out.close();

return 200;

});

上传:

post("/postvideo/:username", (req, res) -> {

MultipartConfigElement multipartConfigElement =

new MultipartConfigElement("/tmp");

req.raw().

setAttribute("org.eclipse.jetty.multipartConfig",

multipartConfigElement);

String username = req.params(":username");

double[] location =

new double[2];

double lattitude =

Double.parseDouble(req.queryParams("lat"));

double longitude =

Double.parseDouble(req.queryParams("lon"));

location[0] = lattitude;

location[1] = longitude;

InputStream inputStream = req.raw().getPart("file").getInputStream();;

Part uploadedFile = req.raw().getPart("file");

// File file = new File(uploadedFile.getName());

GridFS gridFS = new GridFS(db, "videos");

GridFSInputFile gfsFile = gridFS.createFile(inputStream);

gfsFile.put("location", location);

gfsFile.put("username", username);

gfsFile.put("contentType", req.raw().getContentType());

gfsFile.put("filename", uploadedFile.getSubmittedFileName());

collection.insert(gfsFile);

gfsFile.save();

return 201;

});

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
使用Spark Streaming整合Kafka可以实现实时流式数据处理。下面是一个简单的Java代码示例: ```java import org.apache.spark.SparkConf; import org.apache.spark.streaming.Duration; import org.apache.spark.streaming.api.java.JavaInputDStream; import org.apache.spark.streaming.api.java.JavaStreamingContext; import org.apache.spark.streaming.kafka010.ConsumerStrategies; import org.apache.spark.streaming.kafka010.KafkaUtils; import org.apache.spark.streaming.kafka010.LocationStrategies; import java.util.Arrays; import java.util.Collection; import java.util.HashMap; import java.util.Map; public class KafkaSparkStreamingExample { public static void main(String[] args) throws InterruptedException { String brokers = "localhost:9092"; String groupId = "group1"; String topics = "topic1"; // Create context with a 2 seconds batch interval SparkConf sparkConf = new SparkConf().setAppName("KafkaSparkStreamingExample"); JavaStreamingContext streamingContext = new JavaStreamingContext(sparkConf, new Duration(2000)); // Create Kafka parameters map Map<String, Object> kafkaParams = new HashMap<>(); kafkaParams.put("bootstrap.servers", brokers); kafkaParams.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); kafkaParams.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); kafkaParams.put("group.id", groupId); kafkaParams.put("auto.offset.reset", "latest"); kafkaParams.put("enable.auto.commit", false); Collection<String> topicsSet = Arrays.asList(topics.split(",")); // Create direct kafka stream JavaInputDStream<String> messages = KafkaUtils.createDirectStream( streamingContext, LocationStrategies.PreferConsistent(), ConsumerStrategies.<String, String>Subscribe(topicsSet, kafkaParams) ).map(record -> record.value()); // Process each message in stream messages.foreachRDD(rdd -> { rdd.foreach(message -> System.out.println(message)); }); // Start the computation streamingContext.start(); streamingContext.awaitTermination(); } } ``` 在这个示例中,我们首先定义了Kafka的broker地址、消费者组ID和要消费的主题。然后,我们创建了一个Spark Streaming的JavaStreamingContext对象,并定义了每个批次的时间间隔为2秒。 接下来,我们定义了Kafka参数Map和要消费的主题集合,并使用KafkaUtils.createDirectStream()方法创建了一个JavaInputDStream对象。 最后,我们对每个批次中的每个消息进行处理,将其打印到控制台上。最后,我们启动了Spark Streaming上下文并等待它终止。 这只是一个简单的示例,你可以根据自己的需求进行修改和扩展。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值