Stage Tools Database API

A REST API service for managing database credentials in Stage Tools CI/CD pipelines. This service provides:

• 🔌 REST API for CI pipelines to create/retrieve database credentials


• 📊 Web UI for managing databases, users, and developer access


• 🗄️ Persistent storage of all credentials and user management


• 🔐 Developer access control to project databases




🎯 Purpose



This API replaces the inline database management logic in GitLab CI pipelines with a centralized service that:


• Stores credentials persistently in SQLite database


• Provides simple API endpoints for CI to call


• Manages developer database access through web interface


• Uses same naming conventions as existing Stage Tools solutions




🚀 Quick Start





Development Setup



1. Install dependencies:


   npm install

   ``



2. Set environment variables:

   `bash

   export MYSQL_HOST=your-mariadb-host

   export MYSQL_ROOT_PASSWORD=your-password

   export POSTGRES_HOST=your-postgres-host

   export POSTGRES_ROOT_PASSWORD=your-password

   `



3. Start the server:

   `bash

   npm start

   # or for development with auto-reload:

   npm run dev

   `



4. Access the web UI:

   `

   http://localhost:3000

   `



Docker Deployment




1. Build the image:

   `bash

   docker build -t stage-tools/db-api .

   `



2. Run with Docker Compose:

   `bash

   docker-compose up -d

   `



3. Deploy to Swarmpit:

   - Upload docker-compose.swarm.yml to Swarmpit

   - Set environment variables in Swarmpit UI

   - Deploy the stack



📡 API Endpoints




For CI Pipelines




| Method | Endpoint | Description |

|--------|----------|-------------|

| POST | /api/credentials/{project} | Create or get project credentials |

| GET  | /api/credentials/{project} | Get existing credentials |

| GET  | /api/credentials | List all project credentials |



For User Management




| Method | Endpoint | Description |

|--------|----------|-------------|

| POST | /api/developers | Create developer account |

| GET  | /api/developers | List developer accounts |

| POST | /api/developers/{id}/access | Grant database access |

| GET  | /api/developers/{id}/access | Get developer's access |



🔌 CI Integration




Before (Old GitLab CI with inline SQL):

yaml
setup_database:
stage: database
script:
- mariadb -h "$MYSQL_HOST" -u root -p"$MYSQL_ROOT_PASSWORD" -e "CREATE DATABASE IF NOT EXISTS..."
- mariadb -h "$MYSQL_HOST" -u root -p"$MYSQL_ROOT_PASSWORD" -e "CREATE USER..."
- mariadb -h "$MYSQL_HOST" -u root -p"$MYSQL_ROOT_PASSWORD" -e "GRANT ALL PRIVILEGES..."
# ... lots of complex SQL logic


After (New GitLab CI with API call):

yaml
setup_database:
stage: database
script:
- |
RESPONSE=$(curl -s -X POST -H "Content-Type: application/json" \
-d '{"environment":"stage","db_type":"mariadb"}' \
http://db-api:3000/api/credentials/$CI_PROJECT_NAME)

echo "DB_HOST=$(echo $RESPONSE | jq -r '.db_host')" > database_credentials.env
echo "DB_PORT=$(echo $RESPONSE | jq -r '.db_port')" >> database_credentials.env
echo "DB_NAME=$(echo $RESPONSE | jq -r '.db_name')" >> database_credentials.env
echo "DB_USER=$(echo $RESPONSE | jq -r '.db_user')" >> database_credentials.env
echo "DB_PASS=$(echo $RESPONSE | jq -r '.db_password')" >> database_credentials.env
artifacts:
reports:
dotenv: database_credentials.env


🌐 Web Interface




Features:


📊 Database Overview - View all project databases and credentials


👨‍💻 Developer Management - Create developer accounts


🔐 Access Control - Grant developers access to specific project databases


🔌 API Documentation - Built-in API usage examples




Default Admin Account:


Username: admin


Password: admin123


⚠️ Change this password immediately after first login!




🗄️ Database Schema




The service uses SQLite with these tables:



project_credentials


Stores database credentials for each project:

project_name - Project identifier


environment - stage/prod


db_type - mariadb/postgresql


db_name, db_user, db_password - Generated credentials


db_host, db_port - Database server details




developer_users


Developer accounts for web interface access:

username, email, password_hash


is_admin - Admin privileges flag




database_access


Maps developer access to specific project databases:

developer_id → project_name + db_type


access_level - read/write permissions




🔧 Configuration




Environment Variables




#### Required for MariaDB:

bash
MYSQL_HOST=your-mariadb-server
MYSQL_ROOT_PASSWORD=your-root-password
MYSQL_PORT=3306 # optional, defaults to 3306


#### Required for PostgreSQL:

bash
POSTGRES_HOST=your-postgres-server
POSTGRES_USER=postgres # optional, defaults to postgres
POSTGRES_ROOT_PASSWORD=your-postgres-password
POSTGRES_PORT=5432 # optional, defaults to 5432


#### Application Settings:

bash
PORT=3000 # API server port
NODE_ENV=production # production/development
DB_PATH=/app/data/credentials.db # SQLite database path


🛡️ Security Features




Rate limiting on API endpoints


Input validation for project names and parameters


Bcrypt password hashing for developer accounts


Helmet.js security headers


Non-root Docker container execution




🎭 Database Naming Convention




Same as existing Stage Tools solutions:


Database: proj_{project}_{environment}
User: user_{project}
Password: SHA256 deterministic hash based on project + date


Examples:

Database: proj_ecommerce_stage


User: user_ecommerce  


Password: 16-character deterministic hash




🔄 Integration with Stage Tools




Update Install Scripts




Modify existing Stage Tools install scripts to generate GitLab CI that calls this API:

bash


In drupal/nextjs/nette install scripts, replace database setup with:


setup_database:
stage: database
image: alpine:latest
before_script:
- apk add --no-cache curl jq
script:
- |
RESPONSE=\$(curl -s -X POST -H "Content-Type: application/json" \\
-d '{"environment":"${DEPLOY_ENV}","db_type":"mariadb"}' \\
http://db-api:3000/api/credentials/\$CI_PROJECT_NAME)

if [ \$(echo \$RESPONSE | jq -r '.error // empty') ]; then
echo "Database API Error: \$(echo \$RESPONSE | jq -r '.error')"
exit 1
fi

echo "DB_HOST=\$(echo \$RESPONSE | jq -r '.db_host')" > database_credentials.env
echo "DB_PORT=\$(echo \$RESPONSE | jq -r '.db_port')" >> database_credentials.env
echo "DB_NAME=\$(echo \$RESPONSE | jq -r '.db_name')" >> database_credentials.env
echo "DB_USER=\$(echo \$RESPONSE | jq -r '.db_user')" >> database_credentials.env
echo "DB_PASS=\$(echo \$RESPONSE | jq -r '.db_password')" >> database_credentials.env


📋 Example API Usage




Create Database for Project:

bash
curl -X POST -H "Content-Type: application/json" \
-d '{"environment":"stage","db_type":"mariadb"}' \
http://db-api:3000/api/credentials/myproject


Response:

json
{
"project": "myproject",
"environment": "stage",
"db_type": "mariadb",
"db_host": "mariadb.example.com",
"db_port": 3306,
"db_name": "proj_myproject_stage",
"db_user": "user_myproject",
"db_password": "a1b2c3d4e5f6g7h8",
"created": true
}


Create Developer Account:

bash
curl -X POST -H "Content-Type: application/json" \
-d '{"username":"developer","email":"dev@example.com","password":"securepass123"}' \
http://db-api:3000/api/developers


Grant Database Access:

bash
curl -X POST -H "Content-Type: application/json" \
-d '{"project_name":"myproject","db_type":"mariadb"}' \
http://db-api:3000/api/developers/2/access


🚀 Deployment




Swarmpit Deployment Steps:




1. Build and push image:

   `bash

   docker build -t your-registry/stage-tools-db-api .

   docker push your-registry/stage-tools-db-api

   `



2. Deploy via Swarmpit:

   - Upload docker-compose.swarm.yml

   - Set environment variables (database hosts, passwords)

   - Update Traefik labels with your domain

   - Deploy the stack



3. Configure Traefik routing:

   - Web UI: https://db-api.stage.yourdomain.com

   - Internal API: http://db-api.internal.yourdomain.com (for CI)



🧪 Testing




Run the test suite:

bash
npm test


Manual API testing:

bash


Health check


curl http://localhost:3000/health



Test database creation (requires valid database credentials)


curl -X POST -H "Content-Type: application/json" \
-d '{"environment":"stage","db_type":"mariadb"}' \
http://localhost:3000/api/credentials/testproject
``



🤝 Contributing



This API integrates with the existing Stage Tools ecosystem:

1. Keep compatibility with existing GitLab CI patterns
2. Maintain same database naming conventions
3. Follow Stage Tools documentation style
4. Test with both MariaDB and PostgreSQL



📚 Related Documentation




• [Stage Tools Main README](../../README.md)


• [Drupal Solution](../../drupal/default/v1.0/README.md)


• [NextJS Solution](../../nextjs/default/v1.0/README.md)


• [Nette Solution](../../nette/default/v1.0/README.md)

---

Part of Stage Tools - Production deployment solutions for modern web applications