This week at Dozer #6

最新推荐文章于 2024-10-03 20:57:12 发布

GetDozer

最新推荐文章于 2024-10-03 20:57:12 发布

阅读量66

点赞数

文章标签：数据库开发语言 rust

本文链接：https://blog.csdn.net/2301_76777067/article/details/130386359

版权

原文：https://getdozer.io/blog/this-week-at-dozer-6

🌟 Welcome to this week’s update on Dozer! We are thrilled to announce the release of version v0.1.18, which brings a range of new features and improvements. Our team has been working diligently to enhance the overall functionality, and we are eager for you to experience all the new additions. Here are the updates for this week.

Key enhancement of v0.1.18 🚀

Support for Loading Dozer Config from a URL #1440 🌐:

In this release, Dozer introduces the ability to load configuration files from a remote URL. This feature simplifies configuration management and deployment. Now, you can run the following command to load the remore dozer config file:

dozer -c remote_url/dozer_config.yaml

Configurable Secondary Index #1461 🔧:

Secondary indexes can now be customized, offering more control over the database structure and optimization. To get gist of this feature, we will use a data sample from the following repository: https://github.com/getdozer/dozer-samples/tree/main/sql/join

Before implementing the secondary index configuration, the following dozer configuration would throw a 500 error:

connections:
  - config : !LocalStorage
      details:
        path: data
      tables:
        - !Table
          name: trips
          prefix: /trips
          file_type: parquet
          extension: .parquet
    name: ny_taxi


endpoints:
  - name: trips
    path: /trips
    table_name: trips
    index:
      secondary:
        create:


cache_max_map_size: 10000000000

query the data:

POST localhost:8080/trips/query
{
    "$filter": {
        "hvfhs_license_num": "HV0003"
    },
    "$order_by": {
        "trip_miles": "desc"
    }
}

the query result will throw an error:

Failed to query cache: Plan error: Matching index not found. Try to add following secondary index configuration:
- index: !SortedInverted
    fields:
    - hvfhs_license_num
    - trip_miles

Now, you can create and configure the secondary index as follows:

g : !LocalStorage
 details:
 path: data
 tables:
 - !Table
 name: trips
 prefix: /trips
 file_type: parquet
 extension: .parquet
 name: ny_taxi


endpoints:
 - name: trips
 path: /trips
 table_name: trips
 index:
 secondary:
 create:
 - index: !SortedInverted
 fields:
 - hvfhs_license_num
 - trip_miles


cache_max_map_size: 10000000000

After configuring the secondary index, you will get a query result like this:

[
    {
        "hvfhs_license_num": "HV0003",
        "dispatching_base_num": "B03404",
        "originating_base_num": "B03404",
        "request_datetime": "2022-01-01T00:13:51.000Z",
        "on_scene_datetime": "2022-01-01T00:19:52.000Z",
        "pickup_datetime": "2022-01-01T00:19:52.000Z",
        "dropoff_datetime": "2022-01-01T01:34:54.000Z",
        "PULocationID": 138,
        "DOLocationID": 265,
        "trip_miles": 63.33,
        "trip_time": 4502,
        "base_passenger_fare": 166.66,
        "tolls": 26.55,
        "bcf": 5.87,
        "sales_tax": 0.0,
        "congestion_surcharge": 0.0,
        "airport_fee": 2.5,
        "tips": 0.0,
        "driver_pay": 127.05,
        "shared_request_flag": "N",
        "shared_match_flag": "N",
        "access_a_ride_flag": " ",
        "wav_request_flag": "N",
        "wav_match_flag": "N",
        "__dozer_record_id": 8283,
        "__dozer_record_version": 1
    },
    ...
]

Other Fixes and Improvements 🛠️:

A variety of fixes and improvements have been made in this release, including UUID Postgres type support (#1458), optional AccessFilter::fields (#1436), and improved error messages for CustomTypeNotSupported (#1459).

New Contributor Spotlight 🌟

We would like to extend a warm welcome to @nurikk, who made their first contribution in #1460. We appreciate your support and look forward to your future contributions!

Dozer v0.1.18 is available now. Check out the release notes here.

Looking Forward & Spoiler Alert 🌈

As we continue to make strides in improving Dozer, we remain committed to enhancing its features and functionality. Your feedback is invaluable to us, so please don’t hesitate to reach out with any suggestions or ideas. You can always open an issue for feature request OR start a discusisuon thread under our GitHub Q&A category. Together, we can make Dozer the best it can be!

Dozer Cloud will be out soon with rich set of features, enabling developers to to build and deploy Data APIs with more ease! ✨ 🧙‍♀️