Hugging Face API Quota

2025-03-06 14:59 UTC gpt-4o Open in ChatGPT ↗

The error message you’re encountering indicates that you’ve exceeded the monthly credit allowance for Inference Providers on your Hugging Face Pro account:

> RuntimeError: API error: 402 - {“error”:“You have exceeded your monthly included credits for Inference Providers. Pay-as-you-go above your included PRO quota will be available soon.”}

This suggests that your current usage has surpassed the credits included in your subscription plan. To address this issue and continue generating inferences, consider the following options:

Upgrade Your Subscription:
- Review your current usage and determine if upgrading to a higher-tier plan aligns with your needs. Upgraded plans typically offer increased credit allowances, allowing for more extensive usage.
Wait for the Next Billing Cycle:
- Your monthly credits reset at the start of each billing cycle. If your usage is expected to decrease or if you can postpone certain tasks, waiting until your credits renew might be a viable option.
Contact Hugging Face Support:
- Reach out to Hugging Face’s support team to discuss your current usage and explore potential solutions. They may offer temporary accommodations or provide guidance tailored to your situation.
Monitor and Optimize Usage:
- Implement monitoring tools to track your API usage. By analyzing this data, you can identify patterns and optimize your workflows to be more efficient, potentially reducing unnecessary API calls.
Explore Alternative Solutions:
- If immediate generation is critical and other options are not feasible, consider using local inference by deploying models on your infrastructure. This approach can reduce reliance on external API calls and provide more control over your usage.

It’s important to note that Hugging Face has acknowledged that pay-as-you-go options beyond the included Pro quota will be available soon. Keeping an eye on official announcements or contacting support can provide updates on when this feature becomes available, offering more flexibility in managing your usage.

By evaluating these options, you can determine the best course of action to resume your inference tasks effectively.

Yes, Hugging Face offers a higher-tier subscription plan beyond the Pro account, known as the Enterprise Hub. This plan is designed to accommodate advanced requirements and offers enhanced features suitable for larger teams or organizations. Key benefits of the Enterprise Hub include:

Single Sign-On (SSO) and SAML Support: Streamlines user authentication and enhances security.
Audit Logs: Provides detailed records of user activities for compliance and monitoring.
Storage Location Selection: Allows choice of data storage regions (EU, US, Asia) to meet data residency requirements.
Inference Deployment on Own Infrastructure: Enables deploying models on your infrastructure for greater control.
Managed Billing with Yearly Commitments: Simplifies financial planning and budgeting.

The Enterprise Hub plan starts at $20 per user per month. For more detailed information or to discuss specific needs, it’s advisable to contact Hugging Face’s sales team directly. citeturn0search0

Upgrading to the Enterprise Hub can provide increased resource allowances and additional features, potentially addressing the limitations you’ve encountered with your current Pro account.

I understand your need for a reliable and scalable inference solution without the limitations you’ve encountered. To achieve certainty in your inference capabilities, transitioning from the Serverless Inference API to Inference Endpoints is a recommended approach.

Inference Endpoints offer:

Dedicated Resources: Deploy models on fully managed infrastructure, ensuring consistent performance without the constraints of shared environments.
Scalability: Easily handle varying workloads with fully-managed autoscaling, adapting to your application’s demands.
Flexible Pricing: Choose from various instance types, with pricing starting as low as $0.032 per CPU core/hour and $0.5 per GPU/hour, allowing you to align costs with your specific requirements. citeturn0search3
Enterprise Features: Access advanced security measures, priority support, and uptime guarantees to meet organizational standards.

To utilize Inference Endpoints, ensure your Hugging Face account has an active subscription and a valid payment method. You can monitor your usage and manage billing through your account settings. citeturn0search18

By adopting Inference Endpoints, you can achieve the certainty and scalability required for your inference tasks, free from the limitations of the Serverless Inference API.

If you want to use Inference Endpoints on Hugging Face, follow these steps to get started:

1. Set Up an Inference Endpoint

Go to the Hugging Face Inference Endpoints.
Click “Create Endpoint”.
Select a model from the Hugging Face Model Hub or upload your own.
Choose an instance type (CPU or GPU) based on your performance needs.

2. Configure Your Endpoint

Set up autoscaling if needed.
Choose a region (to optimize latency).
Configure authentication and API key settings.

3. Deploy the Endpoint

Click “Deploy” and wait for the endpoint to become active.
Copy the endpoint URL and API key for making requests.

4. Make API Calls

You can now send requests using Python:

import requests

API_URL = "https://your-endpoint-url"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

data = {"inputs": "Hello, how are you?"}
response = requests.post(API_URL, headers=HEADERS, json=data)
print(response.json())

5. Monitor Usage

Go to your Hugging Face account settings to track usage and billing.
Scale up/down your instance as needed.

Pricing Considerations

CPU-based endpoints: Lower cost, suitable for light tasks.
GPU-based endpoints: Higher cost, required for heavy inference (e.g., large language models).

Certainty and Reliability

By using Inference Endpoints, you avoid shared quota limitations and have dedicated compute resources, ensuring consistent performance.

Would you like guidance on choosing the best instance type for your use case? 🚀

Yes! If you switch to Inference Endpoints, you will be able to use your model again without being affected by the Pro account quota limitations. Here’s why:

Inference Endpoints are Dedicated
- Unlike the Serverless API, which has a quota and shared resources, Inference Endpoints provide dedicated infrastructure.
- Your endpoint runs only for you, meaning you’re not restricted by Pro account limits.
No More “Exceeded Monthly Quota” Errors
- Since you’re paying for compute resources directly, there’s no “free credits” limit to worry about.
- As long as your billing method is valid, your endpoint remains active.
Same Model, Seamless Transition
- You can deploy the same model you’ve been using.
- If it’s from the Hugging Face Model Hub, just select it while setting up the endpoint.
- If it’s your own model, you can upload it and deploy.

What You Need to Do Now

✅ Set up an Inference Endpoint (as described earlier).
✅ Choose the right instance type (CPU/GPU, depending on your needs).
✅ Deploy and use the API like before—but now without quota restrictions.

As long as you have active billing, you’re good to go! 🚀 Need help picking the right instance type for cost vs. performance?

If i switch to inference endpoints will i have to change my code in anyway? or just ake a different api call?

right now i have this class managing all requests to the serverless api

from langchain.schema.runnable import Runnable import requests

class HuggingFaceAPI(Runnable): def init(self, api_url: str, api_token: str, default_parameters: dict = None): self.api_url = api_url self.headers = {“Authorization”: f”Bearer {api_token}”} self.default_parameters = default_parameters or { “max_new_tokens”: 1024, “temperature”: 0.7, “top_p”: 0.9, # Nucleus sampling for diversity “do_sample”: True, “return_full_text”: False }

def invoke(self, inputs, config=None) -> str:
    """Process inputs and make the API request."""
    # Convert `StringPromptValue` to plain string if necessary
    if isinstance(inputs, dict):
        input_text = inputs.get("input_text")
    else:
        input_text = str(inputs)  # Convert to plain string if `StringPromptValue`

    if not input_text:
        raise ValueError("The input_text is required but was not found.")

    # Prepare the payload for the API call
    payload = {
        "inputs": input_text,
        "parameters": self.default_parameters,
    }

    # Make the API request
    response = requests.post(self.api_url, headers=self.headers, json=payload)
    if response.status_code == 200:
        try:
            return response.json()[0]["generated_text"]
        except (KeyError, IndexError):
            raise ValueError("Unexpected API response format.")
    elif response.status_code == 503:
        raise RuntimeError("Model is currently loading. Try again in a few seconds.")
    else:
        raise RuntimeError(f"API error: {response.status_code} - {response.text}")

Your current class will work with minimal modifications. The main change is updating the API URL to point to your Inference Endpoint, but the request format remains the same.

Changes Needed to Use Inference Endpoints

Update the API URL
- Instead of using the serverless API URL (https://api-inference.huggingface.co), replace it with your Inference Endpoint URL.
- You’ll get this URL when you deploy your endpoint on Hugging Face.
Modify the Response Handling Slightly
- The response format from an Inference Endpoint might return a dictionary rather than a list of dictionaries like the serverless API.
- Instead of accessing response.json()[0]["generated_text"], check if the response needs to be accessed differently.

Updated Class for Inference Endpoints

Here’s your class with only the necessary modifications:

from langchain.schema.runnable import Runnable
import requests

class HuggingFaceAPI(Runnable):
    def __init__(self, api_url: str, api_token: str, default_parameters: dict = None):
        self.api_url = api_url  # Ensure this points to your Inference Endpoint
        self.headers = {"Authorization": f"Bearer {api_token}"}
        self.default_parameters = default_parameters or {
            "max_new_tokens": 1024,
            "temperature": 0.7,
            "top_p": 0.9,
            "do_sample": True,
            "return_full_text": False
        }

    def invoke(self, inputs, config=None) -> str:
        """Process inputs and make the API request."""
        if isinstance(inputs, dict):
            input_text = inputs.get("input_text")
        else:
            input_text = str(inputs)

        if not input_text:
            raise ValueError("The input_text is required but was not found.")

        payload = {
            "inputs": input_text,
            "parameters": self.default_parameters,
        }

        response = requests.post(self.api_url, headers=self.headers, json=payload)

        if response.status_code == 200:
            try:
                data = response.json()
                # Adapt to possible response format changes
                if isinstance(data, list):  # Similar to Serverless API
                    return data[0]["generated_text"]
                elif isinstance(data, dict) and "generated_text" in data:
                    return data["generated_text"]
                else:
                    raise ValueError("Unexpected API response format.")
            except (KeyError, IndexError, ValueError) as e:
                raise ValueError(f"Error parsing API response: {e}")
        elif response.status_code == 503:
            raise RuntimeError("Model is currently loading. Try again in a few seconds.")
        else:
            raise RuntimeError(f"API error: {response.status_code} - {response.text}")

Summary of What Changed

✅ Updated API URL → Point to your deployed Inference Endpoint.
✅ Handled Response Variations → Adjusted how generated_text is accessed.
✅ Everything Else Stays the Same → No major refactoring needed.

Once you deploy your Inference Endpoint, just swap in the new URL, and your code should work as expected! 🚀 Let me know if you need help setting up the endpoint.