The best way to deal with large files in Python style

<span style="font-family:Arial Black;font-size:12px;color:#000099;">with open(...) as f:
    for line in f:
        <do something with line></span>

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

Click here for origin.

Another alternative: by making use of yield.

<span style="font-family:Arial Black;font-size:12px;">def read_file(fpath): 
    BLOCK_SIZE = 1024 
    with open(fpath, 'rb') as f: 
        while True: 
            block = f.read(BLOCK_SIZE) 
            if block: 
                yield block 
            else: 
                return</span>
Click here for origin.

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
In Elasticsearch 7.7, the recommended way to query from multiple indexes using the Transport Java API is to use the MultiSearchRequest API. Here's an example code snippet for querying from multiple indexes using the MultiSearchRequest API: ``` // Create a client object TransportClient client = new PreBuiltTransportClient(Settings.EMPTY) .addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300)); // Create a MultiSearchRequest object MultiSearchRequest multiSearchRequest = new MultiSearchRequest(); // Add multiple search requests to the MultiSearchRequest object multiSearchRequest.add(SearchRequestBuilders.searchRequest("index1").source(query)); multiSearchRequest.add(SearchRequestBuilders.searchRequest("index2").source(query)); // Execute the MultiSearchRequest MultiSearchResponse multiSearchResponse = client.multiSearch(multiSearchRequest).actionGet(); // Process the response for (MultiSearchResponse.Item item : multiSearchResponse.getResponses()) { SearchResponse searchResponse = item.getResponse(); // Process the search response for each index } // Close the client object client.close(); ``` In this code snippet, we first create a TransportClient object and a MultiSearchRequest object. We then add multiple SearchRequest objects to the MultiSearchRequest object, with each SearchRequest targeting a different index. We then execute the MultiSearchRequest using the client object, and process the response for each index. Note that the above code snippet uses the deprecated TransportClient API. It is recommended to use the Java High Level REST Client or the Java Low Level REST Client instead.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值