To many requests binance

Binance: Too many requests; current limit is 1200 req/min #134

Comments

kascheri12 commented Aug 26, 2018

I got the following error/warning message when creating a tax report for 8/1/2018-8/26/2018.

Trade History Query Warning

During trade history query we got: Binance API request https://api.binance.com/api/v3/myTrades?symbol=PIVXBTC&fromId=0&recvWindow=10000×tamp=1535324978160&signature=de264. for myTrades failed with HTTP status code: 429, error code: -1003 and error message: Too many requests; current limit is 1200 requests per minute. Please use the websocket for live updates to avoid polling the API.. History report is probably not complete.

Should there be a setting in rotkehlchen to limit the basic requests per minute for each exchange’s API?

It seems that different tiers of membership at each exchange might allow for a greater number of available requests/min. It would be best if rotkehlchen would support faster polling, when possible.

The text was updated successfully, but these errors were encountered:

LefterisJP commented Aug 28, 2018

Where did you see this warning? Where was it shown?

Binance does indeed have query limits as explained in their API here

kascheri12 commented Aug 28, 2018

This was shown as a popup in rotkehlchen.

Thank you for the API link.

kascheri12 commented Sep 9, 2018

Is there any way to limit the requests per minute from rotkehlchen so that the error doesn’t happen?

LefterisJP commented Sep 9, 2018

It keeps happening for you? What action are you taking when it happens?

kascheri12 commented Sep 13, 2018

Doesn’t seem to happen anymore. If it happens again I’ll open another issue.

kascheri12 commented Sep 22, 2018

I got a similar error again.

kascheri12 commented Dec 28, 2018

Hi @LefterisJP, I am again receiving a similar error about limit of requests per minute but this time I noticed the symbol (ex. above «STORMBTC») in the URI query string is not one that I have used.

Why is Rotkehlchen querying the exchange for symbols that have no transactions?

LefterisJP commented Dec 28, 2018

Hmm I see what you mean. This limitation comes due to the way the binance API is written. I have to iterate through all existing markets to see which ones you have history for. Perhaps they have added a new endpoint since the time I wrote this. I will check.

LefterisJP commented Dec 28, 2018

LefterisJP commented Dec 28, 2018 •

@kascheri12 I can try to handle the 429 response for binance and wait on it instead of failing like above. But how the heck did you manage to go over 1200 requests per minute? Can you perhaps describe exactly what you did or even send me logs?

Assuming you used the tax report query per the screenshot, that should use the myTrades endpoint which has a weight of 5 for each query. So that leaves us 1200 / 5 = 240 requests per minute. For the symbols for which you have lots of trades we need to make multiple requests (only 500 trades are returned per request). So you must really have had a lot of binance trades and a very good connection to get that.

I think the right solution here is to handle 429 for binance and add a waiting factor.

kascheri12 commented Dec 29, 2018

@LefterisJP I have sent you an email with the goods.

These are the steps taken for the test case:

• Run from latest master branch; macOS 10.14.2
• Login and wait for dashboard to load
• Select Tax Report
• Select May 1, 2018 00:00 & May 31, 2018 00:00 as begin/end dates
• Select Generate Report
• Wait, receive 429 response, select Close

I don’t think 150Mbps is excessively high bandwidth. I’m using an i7 MacBook Pro. I do believe I have a lot of (probably unsuccessful/unnecessary) Binance trades.

It’s unfortunate that the Binance API has this limitation. It makes for lots of wasted resources for all parties involved.

LefterisJP commented Dec 29, 2018

Hey @kascheri12 I wrote a patch that should handle the 429 error: #248

I made a mock test for it since I could not reproduce the 429 error response no matter how fast I spammed binance. Looking at your logs it seems that you can for some reason spam faster than I can. One page of logs (

100 requests) takes 5 milliseconds for you while for me 12. I have an old computer 🙂

I also increased the limit of binance responses to 1000 from the default of 500. Hope they honor it. I tested with smaller limit of 5 and they do.

Unfortunately they don’t honor startTime and endTime so the filtering happens on the rotkehlchen side. We query all trades that ever happened.

Thank you for testing Binance. I have only 70 trades in there and only in one pair so there may be things rotkehlchen misses. Testing with external APIs that depend on you having funds on the exchange is really hard. Wish more exchanges follow bitmex’s example and offer Testing API endpoints.

Источник

To many requests binance

Table of Contents generated with DocToc

Public Rest API for Binance (2021-05-12)

General API Information

• The base endpoint is: https://api.binance.com
• If there are performance issues with the endpoint above, these API clusters are also available:
  • https://api1.binance.com
  • https://api2.binance.com
  • https://api3.binance.com
• All endpoints return either a JSON object or array.
• Data is returned in ascending order. Oldest first, newest last.
• All time and timestamp related fields are in milliseconds.

HTTP Return Codes

• HTTP 4XX return codes are used for malformed requests; the issue is on the sender’s side.
• HTTP 403 return code is used when the WAF Limit (Web Application Firewall) has been violated.
• HTTP 429 return code is used when breaking a request rate limit.
• HTTP 418 return code is used when an IP has been auto-banned for continuing to send requests after receiving 429 codes.
• HTTP 5XX return codes are used for internal errors; the issue is on Binance’s side. It is important to NOT treat this as a failure operation; the execution status is UNKNOWN and could have been a success.

• Any endpoint can return an ERROR

Sample Payload below:

• Specific error codes and messages are defined in Errors Codes.

General Information on Endpoints

• For GET endpoints, parameters must be sent as a query string .
• For POST , PUT , and DELETE endpoints, the parameters may be sent as a query string or in the request body with content type application/x-www-form-urlencoded . You may mix parameters between both the query string and request body if you wish to do so.
• Parameters may be sent in any order.
• If a parameter sent in both the query string and request body , the query string parameter will be used.

General Info on Limits

• The following intervalLetter values for headers:
  • SECOND => S
  • MINUTE => M
  • HOUR => H
  • DAY => D
• intervalNum describes the amount of the interval. For example, intervalNum 5 with intervalLetter M means «Every 5 minutes».
• The /api/v3/exchangeInfo rateLimits array contains objects related to the exchange’s RAW_REQUEST , REQUEST_WEIGHT , and ORDER rate limits. These are further defined in the ENUM definitions section under Rate limiters (rateLimitType) .
• A 429 will be returned when either rate limit is violated.

• Every request will contain X-MBX-USED-WEIGHT-(intervalNum)(intervalLetter) in the response headers which has the current used weight for the IP for all request rate limiters defined.
• Each route has a weight which determines for the number of requests each endpoint counts for. Heavier endpoints and endpoints that do operations on multiple symbols will have a heavier weight .
• When a 429 is received, it’s your obligation as an API to back off and not spam the API.
• Repeatedly violating rate limits and/or failing to back off after receiving 429s will result in an automated IP ban (HTTP status 418).
• IP bans are tracked and scale in duration for repeat offenders, from 2 minutes to 3 days.
• A Retry-After header is sent with a 418 or 429 responses and will give the number of seconds required to wait, in the case of a 429, to prevent a ban, or, in the case of a 418, until the ban is over.
• The limits on the API are based on the IPs, not the API keys.

Order Rate Limits

• Every successful order response will contain a X-MBX-ORDER-COUNT-(intervalNum)(intervalLetter) header which has the current order count for the account for all order rate limiters defined.
• Rejected/unsuccessful orders are not guaranteed to have X-MBX-ORDER-COUNT-** headers in the response.
• The order rate limit is counted against each account.

• The API system is asynchronous, so some delay in the response is normal and expected.
• Each endpoint has a data source indicating where the data is being retrieved, and thus which endpoints have the most up-to-date response.

These are the three sources, ordered by which is has the most up-to-date response to the one with potential delays in updates.

• Matching Engine — the data is from the matching Engine
• Memory — the data is from a server’s local or external memory
• Database — the data is taken directly from a database

Some endpoints can have more than 1 data source. (e.g. Memory => Database) This means that the endpoint will check the first Data Source, and if it cannot find the value it’s looking for it will check the next one.

Endpoint security type

• Each endpoint has a security type that determines how you will interact with it. This is stated next to the NAME of the endpoint.
  • If no security type is stated, assume the security type is NONE.
• API-keys are passed into the Rest API via the X-MBX-APIKEY header.
• API-keys and secret-keys are case sensitive.
• API-keys can be configured to only access certain types of secure endpoints. For example, one API-key could be used for TRADE only, while another API-key can access everything except for TRADE routes.
• By default, API-keys can access all secure routes.

Security Type	Description
NONE	Endpoint can be accessed freely.
TRADE	Endpoint requires sending a valid API-Key and signature.
USER_DATA	Endpoint requires sending a valid API-Key and signature.
USER_STREAM	Endpoint requires sending a valid API-Key.
MARKET_DATA	Endpoint requires sending a valid API-Key.

• TRADE and USER_DATA endpoints are SIGNED endpoints.

SIGNED (TRADE and USER_DATA) Endpoint security

• SIGNED endpoints require an additional parameter, signature , to be sent in the query string or request body .
• Endpoints use HMAC SHA256 signatures. The HMAC SHA256 signature is a keyed HMAC SHA256 operation. Use your secretKey as the key and totalParams as the value for the HMAC operation.
• The signature is not case sensitive.
• totalParams is defined as the query string concatenated with the request body .

• A SIGNED endpoint also requires a parameter, timestamp , to be sent which should be the millisecond timestamp of when the request was created and sent.
• An additional parameter, recvWindow , may be sent to specify the number of milliseconds after timestamp the request is valid for. If recvWindow is not sent, it defaults to 5000.
• The logic is as follows:

Serious trading is about timing. Networks can be unstable and unreliable, which can lead to requests taking varying amounts of time to reach the servers. With recvWindow , you can specify that the request must be processed within a certain number of milliseconds or be rejected by the server.

It is recommended to use a small recvWindow of 5000 or less! The max cannot go beyond 60,000!

SIGNED Endpoint Examples for POST /api/v3/order

Here is a step-by-step example of how to send a valid signed payload from the Linux command line using echo , openssl , and curl .

Key	Value
apiKey	vmPUZE6mv9SD5VNHk4HlWFsOr6aKE2zvsw0MuIgwCIPy6utIco14y7Ju91duEh8A
secretKey	NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j

Parameter	Value
symbol	LTCBTC
side	BUY
type	LIMIT
timeInForce	GTC
quantity	1
price	0.1
recvWindow	5000
timestamp	1499827319559

Example 1: As a request body

requestBody: symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559

HMAC SHA256 signature:

curl command:

Example 2: As a query string

queryString: symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559

HMAC SHA256 signature:

curl command:

Example 3: Mixed query string and request body

queryString: symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC

requestBody: quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559

HMAC SHA256 signature:

curl command:

Note that the signature is different in example 3. There is no & between «GTC» and «quantity=1».

Public API Endpoints

These terms will be used throughout the documentation, so it is recommended especially for new users to read to help their understanding of the API.

• base asset refers to the asset that is the quantity of a symbol. For the symbol BTCUSDT, BTC would be the base asset .
• quote asset refers to the asset that is the price of a symbol. For the symbol BTCUSDT, USDT would be the quote asset .

Symbol status (status):

• PRE_TRADING
• TRADING
• POST_TRADING
• END_OF_DAY
• HALT
• AUCTION_MATCH
• BREAK

Symbol type:

Order status (status):

Status	Description
NEW	The order has been accepted by the engine.
PARTIALLY_FILLED	A part of the order has been filled.
FILLED	The order has been completed.
CANCELED	The order has been canceled by the user.
PENDING_CANCEL	Currently unused
REJECTED	The order was not accepted by the engine and not processed.
EXPIRED	The order was canceled according to the order type’s rules (e.g. LIMIT FOK orders with no fill, LIMIT IOC or MARKET orders that partially fill) or by the exchange, (e.g. orders canceled during liquidation, orders canceled during maintenance)

OCO Status (listStatusType):

Status	Description
RESPONSE	This is used when the ListStatus is responding to a failed action. (E.g. Orderlist placement or cancellation)
EXEC_STARTED	The order list has been placed or there is an update to the order list status.
ALL_DONE	The order list has finished executing and thus no longer active.

OCO Order Status (listOrderStatus):

Status	Description
EXECUTING	Either an order list has been placed or there is an update to the status of the list.
ALL_DONE	An order list has completed execution and thus no longer active.
REJECT	The List Status is responding to a failed action either during order placement or order canceled

ContingencyType

Order types (orderTypes, type):

More information on how the order types definitions can be found here: Types of Orders

• LIMIT
• MARKET
• STOP_LOSS
• STOP_LOSS_LIMIT
• TAKE_PROFIT
• TAKE_PROFIT_LIMIT
• LIMIT_MAKER

Order Response Type (newOrderRespType):

Order side (side):

Time in force (timeInForce):

This sets how long an order will be active before expiration.

Status	Description
GTC	Good Til Canceled An order will be on the book unless the order is canceled.
IOC	Immediate Or Cancel An order will try to fill the order as much as it can before the order expires.
FOK	Fill or Kill An order will expire if the full order cannot be filled upon execution.

Kline/Candlestick chart intervals:

m -> minutes; h -> hours; d -> days; w -> weeks; M -> months

Rate limiters (rateLimitType)

Rate limit intervals (interval)

Test connectivity to the Rest API.

Weight: 1

Parameters: NONE

Data Source: Memory

Response:

Check server time

Test connectivity to the Rest API and get the current server time.

Weight: 1

Parameters: NONE

Data Source: Memory

Response:

Current exchange trading rules and symbol information

Weight: 10

Parameters:

There are 3 possible options:

Options	Example
No parameter	curl -X GET «https://api.binance.com/api/v3/exchangeInfo»
symbol	curl -X GET «https://api.binance.com/api/v3/exchangeInfo?symbol=BNBBTC»
symbols	curl -X GET «https://api.binance.com/api/v3/exchangeInfo?symbols=%5B%22BNBBTC%22,%22BTCUSDT%22%5D» or curl -g GET ‘https://api.binance.com/api/v3/exchangeInfo?symbols=[«BTCUSDT»,»BNBBTC»]’

If any symbol provided in either symbol or symbols do not exist, the endpoint will throw an error.

Data Source: Memory

Response:

Market Data endpoints

Weight: Adjusted based on the limit:

Limit	Weight
5, 10, 20, 50, 100	1
500	5
1000	10
5000	50

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
limit	INT	NO	Default 100; max 5000. Valid limits:[5, 10, 20, 50, 100, 500, 1000, 5000]

Data Source: Memory

Response:

Recent trades list

Get recent trades.

Weight: 1

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
limit	INT	NO	Default 500; max 1000.

Data Source: Memory

Response:

Old trade lookup (MARKET_DATA)

Get older trades.

Weight: 5

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
limit	INT	NO	Default 500; max 1000.
fromId	LONG	NO	TradeId to fetch from. Default gets most recent trades.

Data Source: Database

Response:

Compressed/Aggregate trades list

Get compressed, aggregate trades. Trades that fill at the time, from the same taker order, with the same price will have the quantity aggregated.

Weight: 1

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
fromId	LONG	NO	ID to get aggregate trades from INCLUSIVE.
startTime	LONG	NO	Timestamp in ms to get aggregate trades from INCLUSIVE.
endTime	LONG	NO	Timestamp in ms to get aggregate trades until INCLUSIVE.
limit	INT	NO	Default 500; max 1000.

• If both startTime and endTime are sent, time between startTime and endTime must be less than 1 hour.
• If fromId, startTime, and endTime are not sent, the most recent aggregate trades will be returned.

Data Source: Database

Response:

Kline/candlestick bars for a symbol. Klines are uniquely identified by their open time.

Weight: 1

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
interval	ENUM	YES
startTime	LONG	NO
endTime	LONG	NO
limit	INT	NO	Default 500; max 1000.

• If startTime and endTime are not sent, the most recent klines are returned.

Data Source: Database

Response:

Current average price

Current average price for a symbol.

Weight: 1

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES

Data Source: Memory

Response:

24hr ticker price change statistics

24 hour rolling window price change statistics. Careful when accessing this with no symbol.

Weight: 1 for a single symbol; 40 when the symbol parameter is omitted

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	NO

• If the symbol is not sent, tickers for all symbols will be returned in an array.

Data Source: Memory

Response:

Symbol price ticker

Latest price for a symbol or symbols.

Weight: 1 for a single symbol; 2 when the symbol parameter is omitted

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	NO

• If the symbol is not sent, prices for all symbols will be returned in an array.

Data Source: Memory

Response:

Symbol order book ticker

Best price/qty on the order book for a symbol or symbols.

Weight: 1 for a single symbol; 2 when the symbol parameter is omitted

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	NO

• If the symbol is not sent, bookTickers for all symbols will be returned in an array.

Data Source: Memory

Response:

New order (TRADE)

Send in a new order.

Weight: 1

Parameters:

Name	Type	Mandatory	Description
symbol	STRING	YES
side	ENUM	YES
type	ENUM	YES
timeInForce	ENUM	NO
quantity	DECIMAL	NO
quoteOrderQty	DECIMAL	NO
price	DECIMAL	NO
newClientOrderId	STRING	NO	A unique id among open orders. Automatically generated if not sent. Orders with the same newClientOrderID can be accepted only when the previous one is filled, otherwise the order will be rejected.
stopPrice	DECIMAL	NO	Used with STOP_LOSS , STOP_LOSS_LIMIT , TAKE_PROFIT , and TAKE_PROFIT_LIMIT orders.
icebergQty	DECIMAL	NO	Used with LIMIT , STOP_LOSS_LIMIT , and TAKE_PROFIT_LIMIT to create an iceberg order.
newOrderRespType	ENUM	NO	Set the response JSON. ACK , RESULT , or FULL ; MARKET and LIMIT order types default to FULL , all other orders default to ACK .
recvWindow	LONG	NO	The value cannot be greater than 60000
timestamp	LONG	YES

Some additional mandatory parameters based on order type :

Type	Additional mandatory parameters	Additional Information
LIMIT	timeInForce , quantity , price
MARKET	quantity or quoteOrderQty	MARKET orders using the quantity field specifies the amount of the base asset the user wants to buy or sell at the market price. E.g. MARKET order on BTCUSDT will specify how much BTC the user is buying or selling. MARKET orders using quoteOrderQty specifies the amount the user wants to spend (when buying) or receive (when selling) the quote asset; the correct quantity will be determined based on the market liquidity and quoteOrderQty . E.g. Using the symbol BTCUSDT: BUY side, the order will buy as many BTC as quoteOrderQty USDT can. SELL side, the order will sell as much BTC needed to receive quoteOrderQty USDT.
STOP_LOSS	quantity , stopPrice	This will execute a MARKET order when the stopPrice is reached.
STOP_LOSS_LIMIT	timeInForce , quantity , price , stopPrice
TAKE_PROFIT	quantity , stopPrice	This will execute a MARKET order when the stopPrice is reached.
TAKE_PROFIT_LIMIT	timeInForce , quantity , price , stopPrice
LIMIT_MAKER	quantity , price	This is a LIMIT order that will be rejected if the order immediately matches and trades as a taker. This is also known as a POST-ONLY order.

Any LIMIT or LIMIT_MAKER type order can be made an iceberg order by sending an icebergQty .

Any order with an icebergQty MUST have timeInForce set to GTC .

MARKET orders using quoteOrderQty will not break LOT_SIZE filter rules; the order will execute a quantity that will have the notional value as close as possible to quoteOrderQty . Trigger order price rules against market price for both MARKET and LIMIT versions:

Price above market price: STOP_LOSS BUY , TAKE_PROFIT SELL

Price below market price: STOP_LOSS SELL , TAKE_PROFIT BUY

Data Source: Matching Engine

Источник