Auditlog Implementation Guide

Overview

This project uses django-auditlog to track changes to models. By default, auditlog only captures changes made through Django Admin because it relies on middleware to get the current user from the request context.

This guide explains how to ensure auditlog works across all parts of the application:

• ✅ Django Admin (works by default)
• ✅ REST API (requires mixin)
• ✅ Management Commands (requires context manager)
• ✅ Celery Tasks (requires context manager)
• ✅ Scrapers (requires context manager)
• ✅ ORM operations (requires context manager)

Configuration

1. Middleware (Already Configured)

The auditlog middleware is already enabled in hotspots/settings.py:

MIDDLEWARE = [
    # ... other middleware
    "auditlog.middleware.AuditlogMiddleware",
    # ... other middleware
]

2. Model Registration

Models are registered for auditing using the auditlog.register() decorator at the bottom of the model file:

# In api/models/places.py
from auditlog.registry import auditlog

class HotspotsPlace(Thing):
    # ... model definition

auditlog.register(HotspotsPlace)

Usage by Context

REST API Views (DRF ViewSets)

Problem: DRF ViewSets don't automatically set the actor for auditlog.

Solution: Use the AuditlogMixin on your ViewSets:

from api.mixins import AuditlogMixin

class PlaceViewSet(AuditlogMixin, viewsets.ModelViewSet):
    queryset = HotspotsPlace.objects.all()
    serializer_class = PlaceSerializer

This mixin automatically sets the authenticated user as the actor for create, update, partial_update, and delete operations.

How it works:

• The mixin overrides the action methods (create, update, partial_update, destroy)
• It sets the auditlog actor at the start of each action
• The actor remains set for the entire duration of the request
• This means all .save() calls within custom ViewSet methods are tracked
• The actor is cleared after the action completes

Example with custom partial_update:

class EventViewSet(AuditlogMixin, viewsets.ModelViewSet):

    def partial_update(self, request, *args, **kwargs):
        # Actor is set at the START by AuditlogMixin
        instance = self.get_object()
        serializer = self.get_serializer(instance, data=request.data, partial=True)

        if serializer.is_valid():
            serializer.save()  # ✅ Logged with request.user

            # Custom logic with additional saves
            instance.some_field = "new value"
            instance.save()  # ✅ ALSO logged with request.user (because actor is still set)

        # Actor is cleared at the END by AuditlogMixin
        return Response(serializer.data)

Already Applied To:

• ✅ PlaceViewSet
• ✅ EventViewSet

TODO: Apply to other ViewSets that modify data:

• UserVibeView
• UserVibeTimeView
• UserTipView
• PlaceImageViewSet
• EventImageViewSet

Management Commands

Problem: Management commands have no request context, so no user is set.

Solution: Use the set_system_actor() context manager:

from django.core.management.base import BaseCommand
from api.utils.auditlog_utils import set_system_actor
from api.models import HotspotsPlace

class Command(BaseCommand):
    def handle(self, *args, **options):
        with set_system_actor():
            # All changes in this block will be tracked as system user
            place = HotspotsPlace.objects.get(id=123)
            place.name = "Updated Name"
            place.save()  # This will be logged with system@vibemap.com as actor

System User:

• Email: system@vibemap.com
• Username: system
• Cannot log in (is_active=False)
• Auto-created on first use

Scrapers and Data Import Scripts

Problem: Scrapers run without user context.

Solution: Use the set_scraper_actor() context manager:

from api.utils.auditlog_utils import set_scraper_actor
from api.models import HotspotsPlace

class MySpider(scrapy.Spider):
    def parse(self, response):
        with set_scraper_actor():
            place = HotspotsPlace.objects.create(
                name=response.css('h1::text').get(),
                address=response.css('.address::text').get()
            )
            # This creation will be logged with scraper@vibemap.com as actor

Scraper User:

• Email: scraper@vibemap.com
• Username: scraper
• Cannot log in (is_active=False)
• Auto-created on first use

Celery Tasks

Problem: Background tasks have no request context.

Solution: Use the set_system_actor() context manager:

from celery import shared_task
from api.utils.auditlog_utils import set_system_actor
from api.models import HotspotsPlace

@shared_task
def update_place_ratings():
    with set_system_actor():
        for place in HotspotsPlace.objects.filter(needs_rating_update=True):
            place.aggregate_rating = calculate_rating(place)
            place.save()  # Logged as system user

ORM Operations in Code

Problem: Direct ORM calls (outside views/admin) don't have user context.

Solution: Use the appropriate context manager:

from api.utils.auditlog_utils import set_user_actor, set_system_actor

# If you have a user object:
def my_function(user, place_id):
    with set_user_actor(user):
        place = HotspotsPlace.objects.get(id=place_id)
        place.name = "New Name"
        place.save()  # Logged as the specific user

# If it's a system operation:
def cleanup_duplicates():
    with set_system_actor():
        HotspotsPlace.objects.filter(is_duplicate=True).update(is_closed=True)
        # Logged as system user

Bulk Operations

Warning: Bulk operations like queryset.update() and queryset.bulk_create() DO NOT trigger Django signals, so they won't be logged by auditlog.

Workaround: Use individual saves when auditing is critical:

# ❌ Not logged
with set_system_actor():
    HotspotsPlace.objects.filter(city="SF").update(timezone="America/Los_Angeles")

# ✅ Logged (but slower for large datasets)
with set_system_actor():
    for place in HotspotsPlace.objects.filter(city="SF"):
        place.timezone = "America/Los_Angeles"
        place.save()

Viewing Audit Logs

In Django Admin

1. Go to Django Admin
2. Navigate to "Auditlog" → "Log entries"
3. Filter by:
   • Content type (e.g., "HotspotsPlace")
   • Actor (user who made the change)
   • Action (create, update, delete)
   • Timestamp

Programmatically

from auditlog.models import LogEntry
from django.contrib.contenttypes.models import ContentType
from api.models import HotspotsPlace

# Get all changes for a specific place
place = HotspotsPlace.objects.get(id=123)
content_type = ContentType.objects.get_for_model(HotspotsPlace)
logs = LogEntry.objects.filter(
    content_type=content_type,
    object_id=place.id
).order_by('-timestamp')

for log in logs:
    print(f"{log.timestamp}: {log.actor} - {log.action}")
    print(f"Changes: {log.changes}")

# Get all changes by system user
system_logs = LogEntry.objects.filter(actor__email='system@vibemap.com')

# Get all changes by scrapers
scraper_logs = LogEntry.objects.filter(actor__email='scraper@vibemap.com')

# Get recent changes
from django.utils import timezone
from datetime import timedelta

recent = timezone.now() - timedelta(days=7)
recent_logs = LogEntry.objects.filter(
    timestamp__gte=recent,
    content_type=content_type
)

Example: Viewing What Changed

from auditlog.models import LogEntry

log = LogEntry.objects.filter(object_id=123).latest('timestamp')

print(f"Actor: {log.actor}")
print(f"Action: {log.action}")  # 0=create, 1=update, 2=delete
print(f"Timestamp: {log.timestamp}")

# View specific field changes
if log.changes:
    import json
    changes = json.loads(log.changes)
    for field, (old_value, new_value) in changes.items():
        print(f"{field}: {old_value} → {new_value}")

Migration Guide

For Existing Management Commands

Find all management commands that modify data:

grep -r "\.save()" api/management/commands/
grep -r "\.update(" api/management/commands/
grep -r "\.create(" api/management/commands/

Then wrap the modification code with the context manager:

# Before
def handle(self, *args, **options):
    place.name = "New Name"
    place.save()

# After
from api.utils.auditlog_utils import set_system_actor

def handle(self, *args, **options):
    with set_system_actor():
        place.name = "New Name"
        place.save()

For Existing Scrapers

Wrap the item pipeline save logic:

# In scrapers/pipelines/places.py
from api.utils.auditlog_utils import set_scraper_actor

class PlacePipeline:
    def process_item(self, item, spider):
        with set_scraper_actor():
            place, created = HotspotsPlace.objects.update_or_create(...)
        return item

Performance Considerations

1. Auditlog adds overhead: Each save creates an additional database record in auditlog_logentry
2. Storage: Audit logs can grow large over time
3. Consider cleanup: Implement periodic cleanup of old audit logs:

# Example: Delete audit logs older than 2 years
from auditlog.models import LogEntry
from django.utils import timezone
from datetime import timedelta

cutoff = timezone.now() - timedelta(days=730)
LogEntry.objects.filter(timestamp__lt=cutoff).delete()

Troubleshooting

Changes Not Being Logged

Check:

1. Is the model registered? auditlog.register(YourModel) at bottom of models file
2. Is middleware enabled? Check MIDDLEWARE in settings.py
3. Are you using bulk operations? Use .save() instead of .update()
4. Is the actor set? Use context managers for non-request operations
5. For API requests: Is AuditlogMixin applied to the ViewSet?

Custom partial_update Not Logging

Problem: You have a custom partial_update method with manual .save() calls that aren't being logged.

Example of the problem:

class MyViewSet(viewsets.ModelViewSet):
    def partial_update(self, request, *args, **kwargs):
        instance = self.get_object()
        serializer = self.get_serializer(instance, data=request.data, partial=True)
        serializer.save()  # This gets logged

        # Custom logic
        instance.some_field = "updated"
        instance.save()  # ❌ This does NOT get logged (no actor set)
        return Response(serializer.data)

Solution: Add AuditlogMixin to the ViewSet:

from api.mixins import AuditlogMixin

class MyViewSet(AuditlogMixin, viewsets.ModelViewSet):
    def partial_update(self, request, *args, **kwargs):
        instance = self.get_object()
        serializer = self.get_serializer(instance, data=request.data, partial=True)
        serializer.save()  # ✅ Logged

        # Custom logic
        instance.some_field = "updated"
        instance.save()  # ✅ NOW logged (actor set for entire method)
        return Response(serializer.data)

The mixin sets the actor at the beginning of partial_update and clears it at the end, so all .save() calls in between are tracked.

"AnonymousUser" Showing as Actor

Problem: API requests without authentication show as AnonymousUser.

Solution: Either:

• Require authentication on the ViewSet: permission_classes = [IsAuthenticated]
• Or use set_system_actor() context manager in the view method

System/Scraper User Not Found

Problem: The system or scraper user doesn't exist.

Solution: The users are auto-created on first use of the context managers. If you want to pre-create them:

python manage.py shell

from api.utils.auditlog_utils import get_system_user, get_scraper_user
get_system_user()
get_scraper_user()

Best Practices

1. ✅ Always use context managers in management commands and scrapers
2. ✅ Use the mixin on all ViewSets that modify data
3. ✅ Be specific with actors: Use set_user_actor(user) when you know the user
4. ✅ Avoid bulk operations when auditing is critical
5. ✅ Clean up old logs periodically to manage database size
6. ✅ Monitor audit logs for suspicious activity or debugging

Summary Table

Context	Actor Source	How to Enable
Django Admin	Request user	✅ Automatic (middleware)
DRF API	Request user	Add AuditlogMixin to ViewSet
Management Command	System user	Use set_system_actor()
Celery Task	System user	Use set_system_actor()
Scraper	Scraper user	Use set_scraper_actor()
ORM (with user)	Specific user	Use set_user_actor(user)
ORM (without user)	System user	Use set_system_actor()