manage.py generate_fake_data

Generate Fake Data Command

The generate_fake_data management command populates the database with realistic KCIDB data for local development. It creates the full model cascade – checkouts, builds, tests, and test results – along with users, reference data, and optionally issues. This allows developers to work locally without needing a database dump.

Usage

usage: manage.py generate_fake_data [-h] [--checkouts N] [--builds-per-checkout N]
                                     [--tests-per-build N] [--results-per-test N]
                                     [--public-ratio RATIO] [--users N]
                                     [--with-issues] [--flush] [--force]

Generate fake KCIDB data (checkouts, builds, tests, results) for local development

options:
  -h, --help            show this help message and exit
  --checkouts N         Number of KCIDBCheckout objects to create (default: 5)
  --builds-per-checkout N
                        Number of KCIDBBuild objects per checkout (default: 2)
  --tests-per-build N   Number of KCIDBTest objects per build (default: 3)
  --results-per-test N  Number of KCIDBTestResult objects per test (default: 2)
  --public-ratio RATIO  Fraction of checkouts that are public (0.0-1.0, default: 0.7)
  --users N             Number of fake User objects to create (default: 5)
  --with-issues         Also generate Issues, IssueKinds, and IssueOccurrences
  --flush               Delete ALL existing KCIDB checkouts (with builds, tests and
                        results), ALL Issues and ALL non-superuser Users before
                        generating new data, not only generated ones
  --force               Skip the confirmation prompt for --flush

Example: default run

$ python manage.py generate_fake_data
Generating fake data: 5 checkouts, 10 builds, up to 30 tests, up to 60 results
Public ratio: 70%
Users: 5
Total KCIDB objects: ~155
Creating reference data...
  Policies: public, internal, retrigger
  Origin: redhat
  Trees: 5
  Compilers: 3
  Tests: 8
  Beaker hosts: 4
  Maintainers: 3
Creating 5 users...
  Created 5 users (1 admin, 3 internal, 2 public-only)
Creating 5 checkouts...
  Created 5 checkouts (4 public, 0 retrigger, 1 internal)

Successfully generated fake data: 5 users, 5 checkouts, 10 builds, 27 tests, 54 results

Example: large dataset with issues

$ python manage.py generate_fake_data --checkouts 50 --users 10 --with-issues
Generating fake data: 50 checkouts, 100 builds, up to 300 tests, up to 600 results
Public ratio: 70%
Users: 10
Total KCIDB objects: ~1550
...

Successfully generated fake data: 10 users, 50 checkouts, 100 builds, 276 tests, 552 results

Example: flush and regenerate

$ python manage.py generate_fake_data --flush --checkouts 10 --with-issues
...
WARNING: --flush will permanently delete ALL KCIDB checkouts, ALL Issues and ALL non-superuser Users in the database, not only generated data.
Are you sure you want to continue? [y/N]: y
Flushing existing data...
Deleted 155 KCIDB objects (cascade: {...})
Deleted 19 issue objects (cascade: {...})
Deleted 4 non-superuser User objects
...

What Gets Created

1. Users (auth.User)

• Creates N users (default 5) with realistic usernames and emails
• First user (admin) is set as is_staff=True and is_superuser=True
• Approximately 60% of remaining users get internal group access; others get public-only
• All users are assigned to the appropriate policy groups: policy_internal_read, policy_internal_write, policy_public_write, policy_retrigger_rw
• Users are wired into created_by/last_edited_by stamps on Issues and IssueOccurrences
• If a user with the same username already exists (e.g. from an imported dump), it is reused as-is: its password, flags, and group memberships are left untouched

2. Reference Data (idempotent)

Created with get_or_create – safe to run multiple times:

• Policies: public, internal, retrigger (with matching auth groups)
• KCIDBOrigin: redhat
• GitTrees: rhel-9, rhel-10, rhel-10.1, centos-stream-9, centos-stream-10
• Compilers: 3 cross-compiler strings matching production
• Tests: 8 test definitions (Boot, LTP, Sanity, Networking, etc.)
• BeakerResources: 4 Beaker hostnames
• Maintainers: 3 contacts used for checkout submitters, contacts M2M, and test maintainers

3. KCIDB Pipeline Cascade

The core data follows the production KCIDB model hierarchy. All generated checkout, build, and test IDs start with redhat:fake- (test results append .N to their test ID), so they never collide with real records from an imported dump and are easy to spot. Start times are spread over the last 30 days and never lie in the future.

KCIDBCheckout – with enriched metadata:

• Realistic git data (repo URL, branch, commit hash, kernel version)
• Policy distribution: --public-ratio for public, ~33% of remainder as retrigger, rest internal
• log_excerpt populated with realistic kernel log snippets
• scratch, retrigger, ready_to_report flags set probabilistically
• M2M: contacts, patches (~30%), provenance (~50%)
• Log Artifact created for each checkout

KCIDBBuild – per checkout:

• Varied architectures (aarch64, x86_64, s390x, ppc64le)
• ~8% of builds are invalid (failed): they always have a log_excerpt, no output files, and no tests
• log_excerpt on ~60% of builds
• M2M: input_files, output_files, provenance (~50%)

KCIDBTest – per valid build:

• Weighted status distribution (60% PASS, 10% FAIL, 8% ERROR, etc.)
• maintainers M2M assigned (1-2 maintainers per test)
• log_excerpt on ~50% of tests
• M2M: output_files, provenance (~30%)
• waived set on ~15% of failing tests

KCIDBTestResult – per test:

• Subtest names drawn from realistic per-test pools
• Status biased (60% chance) toward parent test’s status
• output_files M2M on ~50% of results

4. Issues (with --with-issues)

• IssueKinds: 4 kinds (Kernel Bug, Infra, Unstable Test, Pipeline Bug)
• Issues: 7 issues with realistic descriptions, mixed public/internal policies
  • tags assigned on ~20% of issues
  • created_by/last_edited_by set to random generated users
• IssueRegex: text matchers on ~60% of new issues
  • created_by/last_edited_by set to random generated users
• IssueOccurrences: linked to failing tests, test results, checkouts, and builds
  • is_regression set on ~20% of test/result occurrences, ~15% of checkout/build occurrences
  • related_checkout populated for proper checkout context
  • created_by set to random generated users

5. Artifacts

Created automatically as dependencies of other objects:

• Checkout logs, build logs, test logs
• Build input/output files (source tarballs, RPMs)
• Test output files
• Test result output files

Default Scale

With no arguments, the command creates:

Recommendations

1. Quick start for new developers:

   python manage.py generate_fake_data --with-issues
   

   This creates a small but complete dataset with all model types populated.

2. Realistic dataset for UI testing:

   python manage.py generate_fake_data --checkouts 50 --users 10 --with-issues
   

3. Clean slate:

   python manage.py generate_fake_data --flush --with-issues
   

   The --flush flag deletes all KCIDB data (via checkout cascade), all Issues, and all non-superuser Users before generating new data – including real records from an imported dump, not just generated ones. It asks for confirmation unless --force is given. The flush and the generation run in a single transaction, so if generation fails the existing data is left untouched.

4. Testing authorization:

   The admin user (admin / fakepw123) has access to all policies. Non-admin users with internal access can see internal and public data. Public-only users can only see public checkouts and issues.

5. Adjusting data mix:

   python manage.py generate_fake_data --public-ratio 0.3
   

   Creates 30% public, ~23% retrigger, ~47% internal checkouts.

Related Commands

• python manage.py delete_non_public - Delete all non-public data and sanitize users
• python manage.py createsuperuser - Create an admin user interactively
• python manage.py flush - Remove all data from database
• python manage.py loaddata - Load fixture data