Free Image Segmentation API Complete Tutorial: Zero-Cost Pixel-Level Object Extraction (2026-10-24 Verified)
Looking for more zero-cost developer resources? Browse our complete free API directory and create a free account to bookmark tutorials, track API quotas, and receive update notifications when vendors change their free tiers.
Image segmentation is one of the most precise tasks in computer vision. Instead of drawing bounding boxes like object detection, segmentation assigns a class label to every single pixel in an image. This tutorial walks you through five free image segmentation APIs with complete code examples and selection guidance.
Understanding the Three Levels of Image Segmentation
Image segmentation spans three granularity levels:
- Semantic Segmentation: Labels every pixel by category. All objects of the same class share one mask.
- Instance Segmentation: Distinguishes individual instances of the same class. Each object gets its own mask.
- Panoptic Segmentation: Combines semantic and instance segmentation, ensuring every pixel is assigned to either a class or a specific instance.
Solution 1: Hugging Face Inference API (Segment Anything Model)
Meta's Segment Anything Model (SAM) is a landmark in image segmentation. Trained on 1.1 billion images, it supports zero-shot segmentation with point and box prompts. Hugging Face provides a free inference endpoint.
Free Quota
- No hard rate limit, but individual requests time out after 30 seconds
- Best for intermittent calls; high-frequency use should self-deploy
Code Example
import requests
API_URL = "https://api-inference.huggingface.co/models/facebook/sam-vit-base"
headers = {"Authorization": "Bearer YOUR_HF_TOKEN"}
def segment_image(image_path, points=None):
with open(image_path, "rb") as f:
data = f.read()
payload = {"inputs": data}
if points:
payload["parameters"] = {"input_points": points}
response = requests.post(API_URL, headers=headers, json=payload)
if response.status_code == 200:
result = response.json()
masks = result[0].get("mask", [])
scores = result[0].get("score", 0)
print(f"Segmentation complete! Confidence: {scores:.4f}")
return masks
else:
print(f"Request failed: {response.status_code}")
return None
masks = segment_image("example.jpg", points=[[500, 375]])
Pros and Cons
- Zero-shot segmentation, no training required
- Supports point and box prompts
- Free endpoint may queue during peak hours
- Large images need pre-resizing
Solution 2: Google Cloud Vision API
Google Cloud Vision API offers multiple segmentation modes including image properties analysis and crop hints. The free tier provides 1,000 calls per month.
Free Quota
- 1,000 free calls per month for image annotation
- Excess charged at $3.50 per 1,000 calls
- Credit card required but auto-billing disabled by default
Code Example
from google.cloud import vision
import io
def google_segment(image_path):
client = vision.ImageAnnotatorClient()
with io.open(image_path, 'rb') as f:
content = f.read()
image = vision.Image(content=content)
response = client.image_properties(image=image)
if response.image_properties_annotation:
dominant_colors = response.image_properties_annotation.dominant_colors
print(f"Detected {len(dominant_colors.colors)} dominant color regions")
return response
def google_object_segment(image_path):
client = vision.ImageAnnotatorClient()
with io.open(image_path, 'rb') as f:
content = f.read()
image = vision.Image(content=content)
objects = client.object_localization(image=image).localized_object_annotations
for obj in objects:
print(f"Object: {obj.name} (confidence: {obj.score:.3f})")
vertices = obj.bounding_poly.normalized_vertices
for v in vertices:
print(f" Coordinate: ({v.x:.2f}, {v.y:.2f})")
return objects
Pros and Cons
- 1,000 true free calls per month
- Multiple segmentation modes available
- Google infrastructure ensures reliability
- Credit card binding required
- Pixel-level masks less precise than SAM
Solution 3: Remove.bg API (Background Removal)
Remove.bg specializes in background removal using deep learning models that automatically detect foreground subjects and produce precise cutouts.
Free Quota
- 50 free preview calls per month (low resolution)
- High-resolution output requires payment
- Additional quota available through apishare.cc registration
Code Example
import requests
def remove_background(image_path, output_path):
response = requests.post(
'https://api.remove.bg/v1.0/removebg',
data={'size': 'auto'},
files={'image_file': open(image_path, 'rb')},
headers={'X-Api-Key': 'YOUR_API_KEY'}
)
if response.status_code == 200:
with open(output_path, 'wb') as f:
f.write(response.content)
print(f"Background removed! Saved to {output_path}")
else:
print(f"Error: {response.status_code}")
remove_background("photo.jpg", "photo_nobg.png")
Pros and Cons
- One-line code for background removal
- Excellent quality for portraits and products
- Returns transparent PNG
- Limited free quota (50 calls per month)
- Only supports background removal, not multi-class segmentation
Solution 4: Roboflow Inference API
Roboflow provides end-to-end computer vision workflows supporting custom-trained segmentation models and pretrained model inference. The free tier offers 1,000 inferences per month.
Free Quota
- 1,000 free inferences per month
- Custom model deployment supported
- 10 GB workspace storage
Code Example
from roboflow import Roboflow
def roboflow_segment(image_path):
rf = Roboflow(api_key="YOUR_API_KEY")
project = rf.workspace().project("your-segmentation-project")
model = project.version(1).model
result = model.predict(image_path).json()
for pred in result.get("predictions", []):
print(f"Class: {pred['class']}")
print(f"Confidence: {pred['confidence']:.3f}")
points = pred.get("points", [])
print(f"Contour points: {len(points)}")
return result
roboflow_segment("street_scene.jpg")
Pros and Cons
- Supports custom-trained segmentation models
- Comprehensive visualization tools
- 1,000 free calls per month
- Requires self-training or community models
- Complex scenes need extensive labeled data
Solution 5: Self-Hosted SAM2 with OpenCV
For unlimited calls and full control, local deployment of SAM2 (Segment Anything Model 2) is the best choice. Combined with OpenCV, it enables a complete segmentation workflow.
Free Quota
- Completely free, no call limits
- Requires GPU (8 GB+ VRAM recommended) or high-performance CPU
Code Example
import numpy as np
import cv2
from segment_anything import sam_model_registry, SamPredictor
def sam_segment(image_path, point_coords=None):
sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h.pth")
sam.to(device="cuda")
predictor = SamPredictor(sam)
image = cv2.imread(image_path)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
predictor.set_image(image_rgb)
if point_coords is None:
h, w = image.shape[:2]
point_coords = np.array([[w//2, h//2]])
point_labels = np.array([1])
masks, scores, logits = predictor.predict(
point_coords=point_coords,
point_labels=point_labels,
multimask_output=True
)
best_mask = masks[np.argmax(scores)]
print(f"Generated {len(masks)} candidate masks")
print(f"Best confidence: {scores.max():.4f}")
print(f"Segmented pixels: {best_mask.sum()}")
overlay = image.copy()
overlay[best_mask] = [0, 255, 0]
result = cv2.addWeighted(image, 0.7, overlay, 0.3, 0)
cv2.imwrite("segment_result.jpg", result)
return best_mask
Pros and Cons
- Unlimited calls
- Highest pixel-level precision
- Data stays local, ensuring privacy
- Requires GPU hardware
- Initial setup is complex
Five-Solution Comparison
Decision Tree
Evaluation Metrics Reference
| Metric | Meaning | Formula | Use Case |
|---|---|---|---|
| mIoU | Mean Intersection over Union | Sum(IoU_i)/N | Semantic segmentation |
| mAP | Mean Average Precision | Integral of P dot dR | Instance segmentation |
| Pixel Accuracy | Pixel-level accuracy | Correct pixels / Total | Quick evaluation |
| Dice | Dice coefficient | 2 | A intersect B |
| Boundary F1 | Boundary F-score | Precision-Recall harmonic | Contour quality |
Advanced: E-commerce Product Cutout Pipeline
For real e-commerce scenarios processing thousands of product photos daily, here is a complete batch cutout workflow combining object detection with SAM segmentation:
import os
import time
import requests
import numpy as np
from PIL import Image
class ProductSegmentationPipeline:
def __init__(self, hf_token):
self.hf_token = hf_token
self.det_url = "https://api-inference.huggingface.co/models/facebook/detr-resnet-50"
self.seg_url = "https://api-inference.huggingface.co/models/facebook/sam-vit-base"
def detect_products(self, image_path):
with open(image_path, "rb") as f:
data = f.read()
response = requests.post(
self.det_url,
headers={"Authorization": f"Bearer {self.hf_token}"},
json={"inputs": data}
)
if response.status_code == 200:
results = response.json()
boxes = []
for item in results:
if item.get("score", 0) > 0.7:
box = item["box"]
boxes.append({
"label": item["label"],
"box": [box["xmin"], box["ymin"], box["xmax"], box["ymax"]],
"score": item["score"]
})
return boxes
return []
def segment_product(self, image_path, box):
with open(image_path, "rb") as f:
data = f.read()
response = requests.post(
self.seg_url,
headers={"Authorization": f"Bearer {self.hf_token}"},
json={"inputs": data, "parameters": {"input_boxes": [box]}}
)
if response.status_code == 200:
masks = response.json()
if masks and len(masks) > 0:
return masks[0].get("mask")
return None
def remove_background(self, image_path, mask):
image = Image.open(image_path).convert("RGBA")
mask_array = np.array(mask)
result = np.array(image)
result[:, :, 3] = (mask_array * 255).astype(np.uint8)
return Image.fromarray(result)
def process_batch(self, input_dir, output_dir):
os.makedirs(output_dir, exist_ok=True)
stats = {"total": 0, "success": 0, "failed": 0}
for fname in os.listdir(input_dir):
if not fname.endswith(('.jpg', '.png', '.jpeg')):
continue
stats["total"] += 1
image_path = os.path.join(input_dir, fname)
try:
boxes = self.detect_products(image_path)
if not boxes:
stats["failed"] += 1
continue
best_box = max(boxes, key=lambda x: x["score"])["box"]
mask = self.segment_product(image_path, best_box)
if mask:
result = self.remove_background(image_path, mask)
output_path = os.path.join(output_dir, f"nobg_{fname}")
result.save(output_path)
stats["success"] += 1
else:
stats["failed"] += 1
except Exception as e:
stats["failed"] += 1
time.sleep(1)
print(f"Batch complete: {stats}")
return stats
Rate Limit Management
Free APIs universally have rate limits. Proper quota planning maximizes utilization:
| Platform | Free Quota | Rate Limit | Excess Strategy |
|---|---|---|---|
| Hugging Face | No hard limit | 30s timeout | Queue wait |
| Google Vision | 1,000/month | 10 QPS | 429 throttle |
| Remove.bg | 50/month | 1 QPS | 402 payment |
| Roboflow | 1,000/month | No hard limit | Quota exhausted |
Multi-platform load balancing example:
import itertools
class SegmentationLoadBalancer:
def __init__(self):
self.providers = itertools.cycle([
("huggingface", self._hf_segment),
("google", self._google_segment),
("roboflow", self._roboflow_segment),
])
self.usage = {"huggingface": 0, "google": 0, "roboflow": 0}
self.limits = {"huggingface": float('inf'), "google": 1000, "roboflow": 1000}
def segment(self, image_path):
for _ in range(len(self.providers)):
name, func = next(self.providers)
if self.usage[name] < self.limits[name]:
try:
result = func(image_path)
if result:
self.usage[name] += 1
return {"provider": name, "result": result}
except Exception as e:
print(f"Provider {name} failed: {e}")
continue
raise Exception("All free quotas exhausted!")
Security and Privacy
When using cloud segmentation APIs, pay attention to data privacy:
- Do not upload sensitive images (medical scans, ID documents) to cloud APIs; use local SAM instead
- Ensure HTTPS encryption for all API requests
- Segmentation results may contain image features; encrypt when storing
- Verify GDPR compliance for EU user data processing
Summary and Best Practices
Image segmentation is transitioning from lab research to large-scale commercial use. Based on the five free solutions in this tutorial, we recommend:
- Prototype stage (0-1,000 calls/month): Start with Hugging Face SAM, zero configuration
- Small-scale production (1,000-3,000 calls/month): Google Vision plus Roboflow quota rotation
- Large-scale production (over 3,000 calls/month): Local SAM2 deployment, unlimited calls
- Vertical scenarios: Remove.bg for e-commerce cutouts, SAM2 with Dice evaluation for medical imaging
For more free API resources, visit our free API directory. New users can register for a dedicated developer key.