AI Card Bao AICARDBAO.COM

Industry News

OpenAI backend 8-hour slot, Batch API, Claude batch processing, Gemini Batch: four stalls that are 40% to 50% cheaper, and no one has written down the time limit as a guarantee

"Most batches can be run in less than an hour." Almost all tutorials on Claude's batch processing will quote this sentence, and it does come from Anthropic's Message Batches documentation. The problem is that there are two other sentences written a few lines down on the same page: the processing speed "may be slowed down due to current demand", and when it is slowed down, "more requests may be seen to expire." One hour is a description of common situations, and 24 hours is the line written into the rules. Requests that have passed the line but have not been processed will be invalidated directly.

Using this standard to check other companies, the conclusion is similar.

On November 17, OpenAI added two completion time limits of 2 hours and 8 hours to the background mode of Responses API, which are 30% and 40% lower than the standard price respectively. Together with the original Batch API, Claude's Message Batches and Gemini's Batch, there are now four mainstream "cheap if you are willing to wait" options on the market. In the four documents, the words used in the time limit column are target, completes within, most, and target. There is no guarantee.

What will each store leave for you after it expires?

Since there are no guarantees, the difference lies in the timeout.

OpenAI's Batch only has a 24h window. The original words of the document are "can only be set to 24h." The status of the timed-out batch becomes expired, and the unfinished requests are entered into the error file and marked as batch_expired; the completed requests are billed as usual, the results can be retrieved, and the output file is retained for 30 days by default.

On Claude's side, there is no charge for expired requests, and the running results can be downloaded for 29 days.

Gemini has the hardest writing. The target is 24 hours, and tasks expire if they are run or queued for more than 48 hours, and no results are available for expired tasks. The first two companies will at least return the completed work to you, but this one may be empty-handed.

This kind of thing really happens. From September 3 to September 5 this year, someone on the OpenAI developer forum reported that almost all batches submitted in those three days had expired. Official customer service verified three samples and the reply was that they "expired on September 5 after completing some requests." Earlier in March, some developers discovered that the batch processing task of gemini-3.1-pro-preview ran for more than 35 hours, and then expired quietly without an error. The old model in the same period ran out in a few minutes.

What about the new 8-hour slot? The documentation says that after the API accepts the request, it "targets completion within the selected window." After accepting the request, it will automatically retry if it encounters a capacity error. How to charge for more than 8 hours and how to deal with the results is not written.

So leave a margin when scheduling.

For a result that will be used at 9 o'clock tomorrow morning, the 8h file is submitted at 1 o'clock in the morning, which means there is no buffering. There is another layer of background mode: if the model needs to adjust the tool midway, your program must get the response, run the tool, and then send a subsequent request. The time limit only applies to a single response, regardless of when the whole thing is completed.

Three switches for 8 hours

The discount in background mode does not only apply if you turn on the background, but requires three conditions to be met at the same time.

First, background_options.completion_window must be explicitly written as 2h or 8h. If you don't write it, the default value is soonest, the request will still be executed asynchronously, and a response ID will still be returned for you to poll, but the bill will be the original price. This pitfall is very hidden, because there is no difference at all in the calling method.

Second, service_tier should be set to default. When set to auto, it only counts if the API happens to have Standard processing selected.

Third, the flow cannot be opened.

There are three additional rules after receiving the discount, each of which may change your structure.

The current limiting pool is shared. Requests of 2h and 8h do not have a regular per-minute quota, but share the same queuing token upper limit as the Batch API tasks, and the input token is counted at the moment of submission. Teams that are already running large batches of batches will squeeze each other.

The cache is isolated. The documentation states that Standard requests outside the window cannot reuse the prompt word cache created by the 2h or 8h request, and the prompt word must be word-for-word. If the online traffic during the day and the background traffic at night share a long system prompt word, you will build two caches for it.

The data will be downloaded. The background mode will temporarily store the response data. It is useless if the store is set to false, and it is useless if the account is set to zero data retention. The official added a sentence of static encryption, but the promise of "zero retention" is indeed compromised on this path.

Currently, only four models are available: GPT-6.2 Astra, GPT-6.2 Sol, GPT-6.1 Terra, and GPT-6 Luna, and the status is open beta.

Fourth gear side by side

OpenAI backend 2h / 8h OpenAI Batch Claude Message Batches Gemini Batch
Discount 30% / 40% 50% 50% 50%
time limit 2 or 8 hours, target 24 hours 24 hours, most within 1 hour Target 24 hours, expire in 48 hours
after timeout Documentation does not explain Billing completed and available for retrieval Expired portion will not be charged May be fruitless
Adjust tools midway Support, follow-up requests will share the discount Ability to return tool calls and arrange subsequent rounds by yourself Support tools to arrange subsequent rounds by yourself Arrange subsequent rounds yourself
Single batch limit Submit by request 50,000 items / 200 MB 100,000 items / 256 MB File 2 GB, inline 20 MB
Limiting online traffic Pool with Batch Pool with the backend Does not affect Messages API Flex file counts towards universal current limit
Zero data retention will still be temporarily stored Output files are retained for 30 days Not compliant with ZDR, stay 29 days Subject to official terms

Convert the price into a specific number. The standard price of GPT-6.2 Sol is $2 for input and $10 for output per million tokens, $1.40 and $7 for 2h, $1.20 and $6 for 8h, and $1 and $5 for Batch. The flagship Astra starts at $8 and $40, the 8h range is $4.80 and $24, and the Batch is $4 and $20. OpenAI does not separately announce the US dollar price of the backend, these are multiplied by the published percentage.

The Claude Sonnet 5.5 has the exact same price tag as the Sol, $2 and $10, and batches of $1 and $5. The Gemini 4 Hydrogen has standard prices of $1.50 and $7.50, and the Batch is $0.75 and $3.75, the lowest unit price of the three models.

the kind of tasks that are caught in the middle

Just looking at the unit price, the 8-hour model is 20% more expensive than Batch, so it can easily be classified as useless. This conclusion only holds true in one case: your requests are independent of each other and can be packaged and submitted at once.

The task of a tool-tuning agent is not like this. It only knows whether to read the second document after reading the first document. The input of each step depends on the output of the previous step. If you put it into Batch, the model returns a tool call, and this round is over. You have to wait for the batch to complete, get the results, execute the tool, and then form a new batch to submit. In the worst case scenario, each round has to wait 24 hours, and five rounds of tasks can take a week.

In background mode, subsequent requests will continue to enjoy discounts and be eligible for higher queuing priority as long as they use the same model, service level, completion window, continuation_id, and API project. In the OpenAI system, this is the only discount currently available for multi-step agent tasks. Its opponent is not Batch, but Original Price.

Looking at Anthropic in turn, the shortcomings are clear.

It only has the Batch level. OpenAI and Google both have another synchronization tier called Flex. The price is the same as Batch. Requests are sent simultaneously as usual, but may be rejected when busy. Google says that a single request usually returns in 1 to 15 minutes. Anthropic has no equivalent. Its service tier page also says that the capacity commitment of the Priority Tier has been discontinued, and Opus 5.5, Sonnet 5.5 and Haiku 5.5 are not supported; the Fast mode of Opus 5.5 is sold at the output speed of $8 and $40, which is twice the standard price, and cannot be used with Batch. By the way, OpenAI’s Priority changed its name to Fast mode on July 30 this year, and both companies now use the same word to refer to the same type of price-added services.

Therefore, a document pipeline that requires "results within 30 minutes after uploading" can be half-priced by changing a parameter in OpenAI and Google. There are only two options on Claude: pay full price, or rewrite it into an asynchronous task and accept a tail of up to 24 hours.

Claude's batch processing is also cleaner than others: its usage does not account for the current limit of the Messages API. No matter how large a batch is run at night, it will not affect online services during the day. OpenAI's backend and Batch squeeze each other, and Gemini's Flex is included in the general current limit, but it doesn't do this.

Gemini's own problem is maturity. Both Flex and Priority are still marked with preview; Flex is defined as "best effort, discardable", and returns 503 or 429 during peak periods, and the server will not automatically fall back to the standard level.

After the cache is stacked

Batch discounts and cache discounts are multiplied in both OpenAI and Anthropic. Sol’s standard entry price is $0.20 for cache and $0.10 for Batch. Sonnet 5.5's cache hit price is 0.05 times the base input price, $0.10 standard and $0.05 in batch.

Claude is cheaper on paper.

However, Anthropic added a limitation in the document: cache hits in batch processing are "best effort", and the actual hit rate seen by users is between 30% and 98%. A reasonable explanation is that the requests in the batch are processed concurrently. The previous one has not yet written the cache, but the latter one has already started running. The official recommendation is to use a 1-hour cache for batch processing. The default 5-minute cache may not be able to handle requests.

There is a third-party calculation using an evaluation task of 1 billion input tokens (of which 800 million are shared prefixes) and 100 million output tokens per month: Sol batch processing without caching costs 1,500 US dollars, and all prefix hits cost 780 US dollars; Sonnet 5.5 also starts from 1,500 US dollars, and when the hit rate is 98%, it is about 755 US dollars, and it is about 1,270 US dollars when it drops to 30%; Hydrogen is $1125 and $555. There is a difference of more than 500 US dollars between Claude's two numbers. You will have to run the first real batch and look at cache_read_input_tokens to know which end it falls on.

The 8 hour slot leaves a gap in the cache. The documentation does not list separate prices for cached input and cached writes within the window, nor does it say whether the 40% discount also applies to cached tokens. For tasks with a long prompt word prefix, run a small-scale reconciliation before increasing the amount.

There is another thing that is easy to miss: if the input of a single request exceeds 272,000 tokens, the input and cache of these models are charged twice, and the output is charged 1.5 times, and the entire request is translated together. If the input is doubled first and then discounted 40%, it is 20% more expensive than the original price of short context.

An account of nightly contract review

Set a specific task: review 5,000 contracts every night, each with 20,000 input tokens and 2,000 output tokens. No caching is required. The results are required before going to work the next day. A total of 100 million inputs and 10 million outputs in one night.

With Sol: Standard rate is $300 per night, 2h is $210, 8h is $180, Batch is $150. On a 30-day basis, that’s $9,000, $6,300, $5,400, and $4,500 respectively.

Using Sonnet 5.5 for batch processing is also $150 a night. Go Batch with Hydrogen, $112.50 a night, $3,375 a month.

If these contracts are not related to each other and are submitted once after get off work, 8h will cost $900 more per month than Batch, in exchange for not having to prepare task documents. Not a good deal.

If the model needs to adjust the clause library and check related contracts during the review process, the situation will change. Batch's multiple round trips couldn't catch up with the next morning, and comparable objects became standard prices, saving $3,600 per month in the 8h slot.

For contracts where data cannot be retained, the batch processing of the three companies and OpenAI's backend files must first pass compliance. This step takes precedence over price.

Also a reminder to those making annual budgets: All prices for Gemini 4 Carbon will double on December 1, and Gemini 3.8 Flash will double on January 1, 2027, with batch prices doubling accordingly. If you take November’s bill and push it forward to next year, the difference will be double.

The price ranges of the four grades are all clearly written down. What should I do if I don’t finish the run before the expiration date? There are only three companies that have written it down clearly.

Related comparisons:Gemini 4 Neon and Claude Opus 5.5: Programming agents are tied, $6.86 vs. $12.61 per task|GPT-6.2 Astra and Claude Fable 5.2: The list prices are both $8 and $40, the bill per question is 6.6 times different|Gemini 4 Hydrogen and DeepSeek V4.1 Flash 1017: The difference is five times or fifty times, depending on whether the cache is hit.

Related