Batteries

File Storage

Grit includes an S3-compatible file storage service that works with AWS S3, Cloudflare R2, Backblaze B2, and MinIO (for local development). Upload files, generate thumbnails, and serve signed URLs -- all out of the box.

Supported Providers

The storage service uses the AWS SDK v2 under the hood, which means it works with any S3-compatible object storage provider. Switch providers by changing environment variables -- no code changes required.

ProviderUse CaseEndpoint
AWS S3Production (AWS)https://s3.amazonaws.com
Cloudflare R2Production (no egress fees)https://<id>.r2.cloudflarestorage.com
Backblaze B2Production (low cost)https://s3.us-west-002.backblazeb2.com
MinIOLocal developmenthttp://localhost:9000

Configuration

Storage is configured via environment variables. MinIO is included in the Docker Compose setup, so local development works out of the box.

.env
# Storage Configuration
STORAGE_ENDPOINT=http://localhost:9000   # MinIO for local dev
STORAGE_BUCKET=myapp-uploads
STORAGE_REGION=us-east-1
STORAGE_ACCESS_KEY=minioadmin
STORAGE_SECRET_KEY=minioadmin

Storage Service API

The storage service lives at internal/storage/storage.go and provides five core methods.

internal/storage/storage.go
// New creates a Storage instance from config.
// Automatically creates the bucket if it doesn't exist.
func New(cfg config.StorageConfig) (*Storage, error)

// Upload stores a file in the bucket at the given key.
func (s *Storage) Upload(ctx context.Context, key string, reader io.Reader, contentType string) error

// Download retrieves a file from the bucket.
func (s *Storage) Download(ctx context.Context, key string) (io.ReadCloser, error)

// Delete removes a file from the bucket.
func (s *Storage) Delete(ctx context.Context, key string) error

// GetURL returns the public URL for a stored file.
func (s *Storage) GetURL(key string) string

// GetSignedURL returns a pre-signed URL valid for the given duration.
func (s *Storage) GetSignedURL(ctx context.Context, key string, duration time.Duration) (string, error)

Usage Example

example.go
// Upload a file
err := store.Upload(ctx, "uploads/2026/01/avatar.jpg", fileReader, "image/jpeg")

// Get the public URL
url := store.GetURL("uploads/2026/01/avatar.jpg")
// -> http://localhost:9000/myapp-uploads/uploads/2026/01/avatar.jpg

// Generate a signed URL (expires in 1 hour)
signedURL, err := store.GetSignedURL(ctx, "uploads/2026/01/avatar.jpg", time.Hour)

// Download a file
reader, err := store.Download(ctx, "uploads/2026/01/avatar.jpg")
defer reader.Close()

// Delete a file
err = store.Delete(ctx, "uploads/2026/01/avatar.jpg")

Image Processing

The internal/storage/image.go file provides image processing utilities. Images are automatically resized if they exceed the maximum width, and thumbnails can be generated via background jobs.

internal/storage/image.go
const MaxImageWidth = 1920   // Maximum width for processed images
const ThumbnailSize = 300    // Size of generated thumbnails

// ProcessImage resizes an image if it exceeds MaxImageWidth.
// Preserves aspect ratio. Returns processed bytes.
func ProcessImage(reader io.Reader, mimeType string) ([]byte, error)

// GenerateThumbnail creates a square thumbnail (300x300).
// Uses center-crop with Lanczos resampling.
func GenerateThumbnail(reader io.Reader, mimeType string) ([]byte, error)

// IsImageMimeType returns true for supported image formats.
// Supports: image/jpeg, image/png, image/gif
func IsImageMimeType(mimeType string) bool

Upload Handler

The upload handler at POST /api/uploads handles multipart file uploads with MIME type validation, file size limits, and automatic thumbnail generation for images (via background jobs).

EndpointMethodDescription
/api/uploadsPOSTUpload a file (multipart/form-data)
/api/uploadsGETList uploads (paginated)
/api/uploads/:idGETGet upload details
/api/uploads/:idDELETEDelete upload and stored file

Allowed MIME Types

By default, the following file types are allowed, including images, videos, and documents. You can customize the AllowedMimeTypes map in internal/handlers/upload.go.

internal/handlers/upload.go
// MaxUploadSize is the maximum file size (50 MB).
const MaxUploadSize = 50 << 20

var AllowedMimeTypes = map[string]bool{
    "image/jpeg":      true,
    "image/png":       true,
    "image/gif":       true,
    "image/webp":      true,
    "video/mp4":       true,
    "video/webm":      true,
    "video/quicktime": true,
    "application/pdf": true,
    "text/plain":      true,
    "text/csv":        true,
    "application/json": true,
    "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": true,
    "application/vnd.openxmlformats-officedocument.wordprocessingml.document": true,
}

Upload Model

Every uploaded file is tracked in the database via the Upload GORM model. When a file is deleted, both the database record and the stored file are removed.

internal/models/upload.go
type Upload struct {
    ID            uint           `gorm:"primarykey" json:"id"`
    Filename      string         `gorm:"size:255;not null" json:"filename"`
    OriginalName  string         `gorm:"size:255;not null" json:"original_name"`
    MimeType      string         `gorm:"size:100;not null" json:"mime_type"`
    Size          int64          `gorm:"not null" json:"size"`
    Path          string         `gorm:"size:500;not null" json:"path"`
    URL           string         `gorm:"size:500;not null" json:"url"`
    ThumbnailURL  string         `gorm:"size:500" json:"thumbnail_url"`
    UserID        uint           `gorm:"not null;index" json:"user_id"`
    CreatedAt     time.Time      `json:"created_at"`
    UpdatedAt     time.Time      `json:"updated_at"`
    DeletedAt     gorm.DeletedAt `gorm:"index" json:"-"`
}

Upload Flow

When a file is uploaded, the handler validates it, uploads to S3, saves the record, and (for images) enqueues a background job to generate a thumbnail.

upload-flow.txt
POST /api/uploads (multipart/form-data)
  1. Validate file size (max 50 MB)
  2. Validate MIME type (AllowedMimeTypes)
  3. Generate unique filename: {timestamp}-{original_name}.{ext}
  4. Upload to S3 at key: uploads/{year}/{month}/{filename}
  5. Save Upload record to database
  6. If image -> enqueue ProcessImage background job
  7. Return { data: upload, message: "File uploaded successfully" }

cURL Example

terminal
$ curl -X POST http://localhost:8080/api/uploads \
-H "Authorization: Bearer $TOKEN" \
-F "file=@photo.jpg"

Admin File Browser

The admin panel includes a file browser page where administrators can view all uploaded files in a grid layout, filter by MIME type, view file details, and delete files. Image thumbnails are displayed inline when available.

Thumbnail generation: When an image is uploaded, a background job (via asynq) generates a 300x300 thumbnail and stores it at the thumbnails/ prefix. The Upload record is updated with the thumbnail URL once processing is complete. See the Background Jobs page for details.