java异步框架feed,feed-sdk-java: Feed SDK (Java) 是 eBay 开源的 Java 开发包，用于下载和过滤通过公共 Feed API 提供的大型项目 Feed 文...-CSDN博客

FeedSDK.svg?branch=master

Feed SDK

Java SDK for downloading and filtering item feed files

Table of contents

Summary

The item feed files provide a rich set of data regarding active ebay listings. The feed files for any supported marketplace can be downloaded through the feed API.

However, since the volume of data is large, the download operation is performed in chunks, which may be cumbersome.

The feed SDK abstracts all the complexities and provides a very simple interface to -

Setup

The SDK can be added as a maven dependency or the entire repository can be cloned/forked and changes can be made.

You are most welcome to collaborate and enhance the existing code base.

Add as maven dependency

com.ebay.api

feed-sdk

1.0.1.beta-RELEASE

Setting up in the local environment

For setting up the project in your local environment

Clone the repository

Run mvn clean install.

This should generate an uber jar with the following naming convention

feed-sdk-{version}-uber.jar

Downloading feed files

Since the feed API supports feed files which may be as big as 5 GB, there is a capability which was built into the system to download the file in chunks of 100 MB.

The SDK abstracts the complexity involved in calculating the request header 'range' based on the response header 'content-range'.

To download a feed file which is -

bootstrap : (feed_scope = ALL_ACTIVE)

L1 category 1 : (category_id = 1)

marketplace US : (X-EBAY-C-MARKETPLACE-ID: EBAY_US)

FeedRequest.FeedRequestBuilder builder = new FeedRequestBuilder();

builder.feedScope("ALL_ACTIVE")

.categoryId("1")

.siteId("EBAY_US")

.token()

.type(FeedTypeEnum.ITEM);

Feed feed = new FeedImpl();

GetFeedResponse response = feed.get(builder.build(), null);

The GetFeedResponse.filePath denotes the location where the file was downloaded.

Customizing download location

The default download location is the current working directory.

The download location can be changed by specifying the optional 'downloadDirectory' parameter in the feed method.

For example, to download to the location /tmp/feed -

feed.get(builder.build(), "/tmp/feed");

Unzipping feed files

Since the feed file is gzipped, it needs to be unzipped before post processing tasks can be run on it.

To unzip a downloaded feed file -

Feed feed = new FeedImpl();

String unzippedFilePath = feed.unzip(filePath)

Filtering feed files

Available filters

The SDK provides the capability to filter the feed files based on :-

List of leaf category ids

List of seller usernames

List of item locations

Price range

Level two Categories (the token scope should include api_scope and buy.item.feed scope)

Level three Categories (the token scope should include api_scope and buy.item.feed scope)

Item IDs

EPIDs

GTINs

On successful completion of a filter operation, a new filtered file is created.

To filter a feed file on leaf category ids would require these 3 lines of code -

FeedFilterRequest filterRequest = new FeedFilterRequest();

filterRequest.setLeafCategoryIds();

Response response = feed.filter(, filterRequest);

The Response.filePath denotes the location of the filtered file.

Combining filter criteria

The SDK provides the freedom to combine the filter criteria.

To filter on leaf category ids and seller user names for listings in the price range of 1 to 100

FeedFilterRequest filterRequest = new FeedFilterRequest();

filterRequest.setLeafCategoryIds();

filterRequest.setSellerNames();

filterRequest.setPriceLowerLimit(1);

filterRequest.setPriceUpperLimit(100);

Response response = feed.filter(, filterRequest);

Filtering by level two category id

filterRequest.setLevelTwoCategories();

Response response = feed.filter(, filterRequest);

Filtering by level three category id

filterRequest.setLevelThreeCategories();

Response response = feed.filter(, filterRequest);

Schemas

This section provides more detail on what information is contained within the instances that are returned from the SDK method calls.

GetFeedResponse

The ApiResponse instance is returned from the Feed.get() method.

private Integer statusCode;

private String message;

private String filePath;

List errors;

Field name

Description

statusCode

Integer: 0 indicates a successful response. Any non zero value indicates an error

message

String: Detailed information on the status

filePath

String: Absolute path of the location of the resulting file

errors

List: Detailed error information

Response

This Response instance is returned from Feed.unzip and Feed.filter methods.

private Integer statusCode;

private String message;

private String filePath;

Field name

Description

statusCode

Integer: 0 indicates a successful response. Any non zero value indicates an error

message

String: Detailed information on the status

filePath

String: Absolute path of the location of the resulting file

Logging

Logging is configured using slf4j and rolling log files are created in the current directory.

Ensure that appropriate permissions are present to write to the directory

The current log file name is : feed-sdk-log.log

Rolling Log files are created per day with the pattern : feed-sdk-log.{yyyy-MM-dd}.log

Log files are retained for a period of 7 days

Total log file size is capped at 100 MB

Individual log file size is capped at 3 MB, at which new file will be created

Customizing log file location

The log file location can be changed overriding the LOG_HOME variable, and passing as VM argument.

For example, to change the log file location to /var/log

-DLOG_HOME="/var/log"

Usage

The following sections describe the different ways in which the SDK can be used

Using command line options

All the capabilities of the SDK can be invoked using the command line.

To see the available options and filters , use '-help'

java -jar feed-sdk-1.0.0.beta-RELEASE-uber.jar -help

usage: Feed SDK CLI

-help display help

-dt the date when feed file was generated

-c1 the l1 category id of the feed file

-scope the feed scope. Available scopes are

ALL_ACTIVE or NEWLY_LISTED

-mkt the marketplace id for which feed is

being request. For example - EBAY_US

-token the oauth token for the consumer. Omit

the word 'Bearer'

-c2f list of l2 categories which are used to

filter the feed

-c3f list of l3 categories which are used to

filter the feed

-lf list of leaf categories which are used

to filter the feed

-sellerf list of seller names which are used to

filter the feed

-locf list of item locations which are used

to filter the feed

-pricelf lower limit of the price range for

items in the feed

-priceuf upper limit of the price range for

items in the feed

-epidf list of epids which are used to filter

the feed

-iepidf list of inferred epids which are used

to filter the feed

-gtinf list of gtins which are used to filter

the feed

-itemf list of item ids which are used to

filter the feed

-dl,--downloadlocation override for changing the directory

where files are downloaded

-cl,--credentiallocation directory where the credentials file is

located

-authscopes,--oauthscopes list of scopes

-env Environment Type. Supported Environments are SANDBOX and PRODUCTION

For example, to use the command line options to download, unzip and filter feed files

Using token

java -jar feed-sdk-1.0.0.beta-RELEASE-uber.jar -dt 20180701 -c1 1281 -scope ALL_ACTIVE -lf 46310 177789 -mkt EBAY_US -c3f 177792 116381 -pricelf 2 -priceuf 100 -locf US CN -token xxx

Using credentials file

java -jar feed-sdk-1.0.0.beta-RELEASE-uber.jar -dt 20180801 -c1 11700 -scope ALL_ACTIVE -mkt EBAY_US -pricelf 2 -priceuf 100 -locf US CN -cl

Using config file driven approach

All the capabilities of the SDK can be leveraged via a config file.

The feed file download and filter parameters can be specified in the config file for multiple files, and SDK will process them sequentially.

The structure of the config file

{

"requests": [{

"feedRequest": {

"categoryId": "1",

"marketplaceId": "EBAY_US",

"date": "20180708",

"feedScope": "ALL_ACTIVE",

"type": "ITEM"

"filterRequest": {

"leafCategoryIds": ["11675", "3226"],

"itemLocationCountries": ["US", "CN"],

"priceLowerLimit": 10.0,

"priceUpperLimit": 100.0

"levelThreeCategories": ["1313", "13583", "39"]

}]

}

An example of using the SDK through a config file is located at

Using java method calls

Samples showing the usage of available operations and filters.

Examples

All the examples are located here

Performance