diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..45eaa69 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,21 @@ +root = true + +[*] +charset = utf-8 +end_of_line = lf +indent_style = space +indent_size = 4 +insert_final_newline = true +trim_trailing_whitespace = true + +[*.{yml,yaml}] +indent_size = 2 + +[*.{json,js,ts}] +indent_size = 2 + +[*.md] +trim_trailing_whitespace = false + +[Makefile] +indent_style = tab \ No newline at end of file diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..51a8090 --- /dev/null +++ b/.env.example @@ -0,0 +1,38 @@ +# Environment Configuration Template +# Copy this file to .env and update with your specific values + +# Application Settings +APP_NAME=YourProjectName +DEBUG=false +LOG_LEVEL=INFO +SECRET_KEY=your-secret-key-here + +# Database Configuration +DATABASE_URL=sqlite:///app.db +# For PostgreSQL: postgresql://user:password@localhost:5432/dbname +# For MySQL: mysql://user:password@localhost:3306/dbname + +# External Services +API_KEY=your-api-key-here +EXTERNAL_SERVICE_URL=https://api.example.com + +# Email Configuration (if applicable) +EMAIL_HOST=smtp.gmail.com +EMAIL_PORT=587 +EMAIL_USER=your-email@example.com +EMAIL_PASSWORD=your-email-password + +# Redis Configuration (if applicable) +REDIS_URL=redis://localhost:6379/0 + +# File Storage (if applicable) +MEDIA_ROOT=/path/to/media/files +STATIC_ROOT=/path/to/static/files + +# Security Settings +ALLOWED_HOSTS=localhost,127.0.0.1,your-domain.com +CORS_ALLOWED_ORIGINS=http://localhost:3000,https://your-frontend.com + +# Feature Flags +ENABLE_FEATURE_X=false +ENABLE_DEBUG_TOOLBAR=false \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..44b31a6 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,35 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '[BUG] ' +labels: bug +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Environment (please complete the following information):** + - OS: [e.g. Ubuntu 20.04, Windows 10, macOS 12.0] + - Version: [e.g. 1.2.3] + - Python/Node.js version: [e.g. Python 3.9, Node.js 18.0] + +**Additional context** +Add any other context about the problem here. + +**Possible Solution** +If you have ideas on how to fix the issue, please describe them here. \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..c06a2de --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,26 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '[FEATURE] ' +labels: enhancement +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Use Cases** +Describe specific use cases where this feature would be beneficial. + +**Implementation Ideas** +If you have ideas on how this could be implemented, please describe them here. + +**Additional context** +Add any other context, mockups, or screenshots about the feature request here. \ No newline at end of file diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md new file mode 100644 index 0000000..c8d3d7c --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md @@ -0,0 +1,41 @@ +## Description + +Please include a summary of the changes and the related issue. Please also include relevant motivation and context. + +Fixes # (issue) + +## Type of change + +Please delete options that are not relevant. + +- [ ] Bug fix (non-breaking change which fixes an issue) +- [ ] New feature (non-breaking change which adds functionality) +- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) +- [ ] This change requires a documentation update + +## How Has This Been Tested? + +Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. + +- [ ] Test A +- [ ] Test B + +**Test Configuration**: +* Operating System: +* Python/Node.js version: +* Dependencies version: + +## Checklist: + +- [ ] My code follows the style guidelines of this project +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have made corresponding changes to the documentation +- [ ] My changes generate no new warnings +- [ ] I have added tests that prove my fix is effective or that my feature works +- [ ] New and existing unit tests pass locally with my changes +- [ ] Any dependent changes have been merged and published in downstream modules + +## Additional Notes + +Add any additional notes, concerns, or considerations here. \ No newline at end of file diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..29f678b --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,65 @@ +name: CI + +on: + push: + branches: [ main, develop ] + pull_request: + branches: [ main, develop ] + +jobs: + test: + runs-on: ubuntu-latest + strategy: + matrix: + # Customize based on your project's needs + python-version: [3.8, 3.9, "3.10", "3.11"] + # node-version: [16, 18, 20] + + steps: + - uses: actions/checkout@v4 + + # Python setup example - uncomment and customize as needed + # - name: Set up Python ${{ matrix.python-version }} + # uses: actions/setup-python@v4 + # with: + # python-version: ${{ matrix.python-version }} + + # - name: Install dependencies + # run: | + # python -m pip install --upgrade pip + # pip install -r requirements.txt + # pip install -r requirements-dev.txt + + # - name: Lint with flake8 + # run: | + # # stop the build if there are Python syntax errors or undefined names + # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics + # # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide + # flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics + + # - name: Test with pytest + # run: | + # pytest + + # Node.js setup example - uncomment and customize as needed + # - name: Use Node.js ${{ matrix.node-version }} + # uses: actions/setup-node@v4 + # with: + # node-version: ${{ matrix.node-version }} + + # - name: Install dependencies + # run: npm ci + + # - name: Run linter + # run: npm run lint + + # - name: Run tests + # run: npm test + + # Placeholder step - replace with actual CI steps + - name: Placeholder CI step + run: | + echo "Replace this with your actual CI steps" + echo "Examples: linting, testing, building" + echo "Current directory contents:" + ls -la \ No newline at end of file diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..907d0a7 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,128 @@ +name: Release + +on: + push: + tags: + - 'v*' # Trigger on version tags like v1.0.0 + +jobs: + create-release: + runs-on: ubuntu-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Fetch full history for changelog + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: '3.9' + + # Python build example + - name: Install build dependencies + run: | + python -m pip install --upgrade pip + pip install build twine + + - name: Build package + run: python -m build + + - name: Check package + run: twine check dist/* + + # Node.js build example (uncomment if using Node.js) + # - name: Set up Node.js + # uses: actions/setup-node@v4 + # with: + # node-version: '18' + # registry-url: 'https://registry.npmjs.org' + + # - name: Install dependencies + # run: npm ci + + # - name: Build package + # run: npm run build + + # - name: Run tests + # run: npm test + + - name: Extract release notes + id: extract-release-notes + run: | + # Extract release notes from CHANGELOG.md + version=${GITHUB_REF#refs/tags/} + awk -v version="$version" ' + /^## \[/ { + if ($0 ~ version) { + found=1; next + } else if (found) { + exit + } + } + found && /^## \[/ { exit } + found { print } + ' CHANGELOG.md > release_notes.txt + + - name: Create Release + uses: actions/create-release@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + tag_name: ${{ github.ref }} + release_name: Release ${{ github.ref }} + body_path: release_notes.txt + draft: false + prerelease: false + + # Python package publishing (uncomment if publishing to PyPI) + # - name: Publish to PyPI + # env: + # TWINE_USERNAME: __token__ + # TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + # run: twine upload dist/* + + # Node.js package publishing (uncomment if publishing to npm) + # - name: Publish to npm + # run: npm publish + # env: + # NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + + # Docker image publishing (uncomment if publishing Docker images) + # - name: Set up Docker Buildx + # uses: docker/setup-buildx-action@v2 + + # - name: Login to Docker Hub + # uses: docker/login-action@v2 + # with: + # username: ${{ secrets.DOCKERHUB_USERNAME }} + # password: ${{ secrets.DOCKERHUB_TOKEN }} + + # - name: Build and push Docker image + # uses: docker/build-push-action@v4 + # with: + # context: . + # push: true + # tags: | + # your-username/your-project:latest + # your-username/your-project:${{ github.ref_name }} + + # Notify about release + notify: + needs: create-release + runs-on: ubuntu-latest + if: always() + + steps: + - name: Notify success + if: needs.create-release.result == 'success' + run: | + echo "โœ… Release ${{ github.ref }} created successfully!" + # Add notification logic here (Slack, Discord, email, etc.) + + - name: Notify failure + if: needs.create-release.result == 'failure' + run: | + echo "โŒ Release ${{ github.ref }} failed!" + # Add failure notification logic here \ No newline at end of file diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..3b8b1d3 --- /dev/null +++ b/.gitignore @@ -0,0 +1,273 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +pip-wheel-metadata/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +.python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow +__pypackages__/ + +# Celery stuff +celerybeat-schedule +celerybeat.pid + +# SageMath parsed files +*.sage.py + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +# Node.js +node_modules/ +npm-debug.log* +yarn-debug.log* +yarn-error.log* +lerna-debug.log* + +# Runtime data +pids +*.pid +*.seed +*.pid.lock + +# Coverage directory used by tools like istanbul +coverage/ +*.lcov + +# nyc test coverage +.nyc_output + +# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) +.grunt + +# Bower dependency directory (https://bower.io/) +bower_components + +# node-waf configuration +.lock-wscript + +# Compiled binary addons (https://nodejs.org/api/addons.html) +build/Release + +# Dependency directories +jspm_packages/ + +# TypeScript v1 declaration files +typings/ + +# TypeScript cache +*.tsbuildinfo + +# Optional npm cache directory +.npm + +# Optional eslint cache +.eslintcache + +# Microbundle cache +.rpt2_cache/ +.rts2_cache_cjs/ +.rts2_cache_es/ +.rts2_cache_umd/ + +# Optional REPL history +.node_repl_history + +# Output of 'npm pack' +*.tgz + +# Yarn Integrity file +.yarn-integrity + +# dotenv environment variables file +.env.test + +# parcel-bundler cache (https://parceljs.org/) +.parcel-cache + +# Next.js build output +.next + +# Nuxt.js build / generate output +.nuxt +dist + +# Gatsby files +.cache/ +public + +# Storybook build outputs +.out +.storybook-out + +# Temporary folders +tmp/ +temp/ + +# Logs +logs +*.log + +# Runtime data +pids +*.pid +*.seed +*.pid.lock + +# Optional npm cache directory +.npm + +# Optional REPL history +.node_repl_history + +# Output of 'npm pack' +*.tgz + +# Yarn Integrity file +.yarn-integrity + +# parcel-bundler cache (https://parceljs.org/) +.parcel-cache + +# IDE files +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# OS generated files +.DS_Store +.DS_Store? +._* +.Spotlight-V100 +.Trashes +ehthumbs.db +Thumbs.db + +# Backup files +*.bak +*.backup +*.tmp + +# Local configuration files +local_config.py +local_settings.py +.local + +# Data files (uncomment if you want to ignore data files) +# *.csv +# *.json +# *.xml +# *.xlsx \ No newline at end of file diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..b639c46 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,76 @@ +# Pre-commit hooks configuration +# See https://pre-commit.com for more information + +repos: + # General hooks + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.4.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-added-large-files + - id: check-case-conflict + - id: check-merge-conflict + - id: check-toml + - id: debug-statements + - id: mixed-line-ending + + # Python-specific hooks (uncomment if using Python) + # - repo: https://github.com/psf/black + # rev: 23.3.0 + # hooks: + # - id: black + # language_version: python3 + + # - repo: https://github.com/pycqa/isort + # rev: 5.12.0 + # hooks: + # - id: isort + # args: ["--profile", "black"] + + # - repo: https://github.com/pycqa/flake8 + # rev: 6.0.0 + # hooks: + # - id: flake8 + # additional_dependencies: [flake8-docstrings] + + # - repo: https://github.com/pre-commit/mirrors-mypy + # rev: v1.3.0 + # hooks: + # - id: mypy + # additional_dependencies: [types-all] + + # JavaScript/TypeScript hooks (uncomment if using Node.js) + # - repo: https://github.com/pre-commit/mirrors-eslint + # rev: v8.42.0 + # hooks: + # - id: eslint + # files: \.(js|ts|jsx|tsx)$ + # types: [file] + + # - repo: https://github.com/pre-commit/mirrors-prettier + # rev: v3.0.0 + # hooks: + # - id: prettier + # files: \.(js|ts|jsx|tsx|json|css|md)$ + + # Security scanning + - repo: https://github.com/Yelp/detect-secrets + rev: v1.4.0 + hooks: + - id: detect-secrets + args: ['--baseline', '.secrets.baseline'] + exclude: package.lock.json + + # Commit message formatting + - repo: https://github.com/commitizen-tools/commitizen + rev: v3.2.2 + hooks: + - id: commitizen + stages: [commit-msg] + +# Configuration for individual hooks +default_language_version: + python: python3.9 + node: 16.14.0 \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..64670ac --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,66 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- Initial project template structure +- Basic documentation templates +- GitHub workflow templates +- Contributing guidelines + +### Changed +- Nothing yet + +### Deprecated +- Nothing yet + +### Removed +- Nothing yet + +### Fixed +- Nothing yet + +### Security +- Nothing yet + +## [1.0.0] - 2024-01-01 + +### Added +- Initial release of the BioNanomics project template +- Basic project structure +- Documentation templates +- Development workflow setup + +--- + +## Template Usage Notes + +When using this template for a new project: + +1. Update the `[Unreleased]` section with your initial changes +2. Replace the template version history with your project's actual releases +3. Follow the format for each release: + - Use semantic versioning (MAJOR.MINOR.PATCH) + - Include the release date + - Categorize changes as Added/Changed/Deprecated/Removed/Fixed/Security +4. Link to compare views and release tags when available + +### Example Entry Format: + +```markdown +## [1.2.3] - 2024-MM-DD + +### Added +- New feature description + +### Changed +- Modified feature description + +### Fixed +- Bug fix description +``` \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..9d8fa57 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,96 @@ +# Contributing to [Project Name] + +We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's: + +- Reporting a bug +- Discussing the current state of the code +- Submitting a fix +- Proposing new features +- Becoming a maintainer + +## Development Process + +We use GitHub to host code, to track issues and feature requests, as well as accept pull requests. + +### Pull Requests + +1. Fork the repo and create your branch from `main`. +2. If you've added code that should be tested, add tests. +3. If you've changed APIs, update the documentation. +4. Ensure the test suite passes. +5. Make sure your code lints. +6. Issue that pull request! + +### Coding Standards + +- Use consistent indentation (spaces, not tabs) +- Follow the existing code style +- Write meaningful commit messages +- Include tests for new functionality +- Update documentation as needed + +## Code of Conduct + +### Our Pledge + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to making participation in our project and +our community a harassment-free experience for everyone. + +### Our Standards + +Examples of behavior that contributes to creating a positive environment include: + +- Using welcoming and inclusive language +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery and unwelcome sexual attention or advances +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information without explicit permission + +## Reporting Issues + +We use GitHub issues to track public bugs. Report a bug by [opening a new issue](../../issues/new). + +**Great Bug Reports** tend to have: + +- A quick summary and/or background +- Steps to reproduce + - Be specific! + - Give sample code if you can +- What you expected would happen +- What actually happens +- Notes (possibly including why you think this might be happening, or stuff you tried that didn't work) + +## Feature Requests + +We welcome feature requests! Please provide: + +- A clear and detailed explanation of the feature +- Any relevant examples or use cases +- How it would benefit the project and users + +## Development Setup + +1. Clone your fork of the repository +2. Install dependencies (see README.md) +3. Create a new branch: `git checkout -b feature-name` +4. Make your changes +5. Run tests: `[testing command]` +6. Commit your changes: `git commit -am 'Add some feature'` +7. Push to the branch: `git push origin feature-name` +8. Create a pull request + +## License + +By contributing, you agree that your contributions will be licensed under the same license as the project (see LICENSE file). + +## Questions? + +Feel free to contact the maintainers if you have any questions. We're here to help! \ No newline at end of file diff --git a/Dockerfile.template b/Dockerfile.template new file mode 100644 index 0000000..e1533c8 --- /dev/null +++ b/Dockerfile.template @@ -0,0 +1,95 @@ +# Multi-stage Docker build for Python applications +# Customize based on your specific needs + +# Build stage +FROM python:3.9-slim as builder + +# Set build arguments +ARG BUILD_DATE +ARG VCS_REF +ARG VERSION + +# Set metadata +LABEL org.opencontainers.image.title="Your Project Name" +LABEL org.opencontainers.image.description="Your project description" +LABEL org.opencontainers.image.created=$BUILD_DATE +LABEL org.opencontainers.image.revision=$VCS_REF +LABEL org.opencontainers.image.version=$VERSION +LABEL org.opencontainers.image.vendor="BioNanomics" + +# Install system dependencies +RUN apt-get update && apt-get install -y \ + build-essential \ + && rm -rf /var/lib/apt/lists/* + +# Set work directory +WORKDIR /build + +# Copy requirements first for better caching +COPY requirements.txt requirements-dev.txt ./ + +# Install Python dependencies +RUN pip install --no-cache-dir --upgrade pip && \ + pip install --no-cache-dir -r requirements.txt + +# Production stage +FROM python:3.9-slim as production + +# Create non-root user +RUN useradd --create-home --shell /bin/bash app + +# Install runtime dependencies +RUN apt-get update && apt-get install -y \ + && rm -rf /var/lib/apt/lists/* + +# Copy installed packages from builder +COPY --from=builder /usr/local/lib/python3.9/site-packages /usr/local/lib/python3.9/site-packages +COPY --from=builder /usr/local/bin /usr/local/bin + +# Set work directory +WORKDIR /app + +# Copy application code +COPY --chown=app:app src/ ./src/ +COPY --chown=app:app config/ ./config/ +COPY --chown=app:app *.py ./ + +# Switch to non-root user +USER app + +# Set environment variables +ENV PYTHONPATH=/app +ENV PYTHONUNBUFFERED=1 +ENV PYTHONDONTWRITEBYTECODE=1 + +# Expose port +EXPOSE 8000 + +# Health check +HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \ + CMD curl -f http://localhost:8000/health || exit 1 + +# Run application +CMD ["python", "-m", "src.main"] + +# Development stage (optional) +FROM production as development + +# Switch back to root to install dev dependencies +USER root + +# Install development dependencies +COPY requirements-dev.txt ./ +RUN pip install --no-cache-dir -r requirements-dev.txt + +# Install development tools +RUN apt-get update && apt-get install -y \ + curl \ + git \ + && rm -rf /var/lib/apt/lists/* + +# Switch back to app user +USER app + +# Override command for development +CMD ["python", "-m", "src.main", "--debug"] \ No newline at end of file diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..67cad7c --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2024 BioNanomics + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..0835a72 --- /dev/null +++ b/Makefile @@ -0,0 +1,129 @@ +# Makefile for project automation +# Customize based on your specific needs + +.PHONY: help install install-dev test test-cov lint format clean build run docker-build docker-run + +# Default target +help: ## Show this help message + @echo "Available commands:" + @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' + +# Python project commands +install: ## Install production dependencies + pip install -r requirements.txt + +install-dev: ## Install development dependencies + pip install -r requirements.txt -r requirements-dev.txt + pre-commit install + +test: ## Run tests + pytest + +test-cov: ## Run tests with coverage + pytest --cov=src --cov-report=html --cov-report=term + +lint: ## Run linting checks + flake8 src/ tests/ + pylint src/ + mypy src/ + +format: ## Format code + black src/ tests/ + isort src/ tests/ + +# Node.js project commands (uncomment if using Node.js) +# install: ## Install dependencies +# npm install + +# install-dev: ## Install development dependencies +# npm install --include=dev + +# test: ## Run tests +# npm test + +# test-cov: ## Run tests with coverage +# npm run test:coverage + +# lint: ## Run linting +# npm run lint + +# format: ## Format code +# npm run format + +# Build commands +build: ## Build the project + python -m build + # npm run build # For Node.js projects + +clean: ## Clean build artifacts + rm -rf build/ + rm -rf dist/ + rm -rf *.egg-info/ + rm -rf .pytest_cache/ + rm -rf .coverage + rm -rf htmlcov/ + find . -type d -name __pycache__ -delete + find . -type f -name "*.pyc" -delete + +# Runtime commands +run: ## Run the application + python -m src.main + # npm start # For Node.js projects + +dev: ## Run in development mode + python -m src.main --debug + # npm run dev # For Node.js projects + +# Docker commands +docker-build: ## Build Docker image + docker build -t your-project-name . + +docker-run: ## Run Docker container + docker run -p 8000:8000 your-project-name + +docker-compose-up: ## Start services with docker-compose + docker-compose up -d + +docker-compose-down: ## Stop services + docker-compose down + +# Database commands (customize based on your needs) +db-migrate: ## Run database migrations + python manage.py migrate + # npm run migrate # For Node.js projects + +db-reset: ## Reset database + python manage.py reset_db + # npm run db:reset # For Node.js projects + +# Documentation +docs: ## Generate documentation + cd docs && make html + # npm run docs # For Node.js projects + +docs-serve: ## Serve documentation locally + cd docs/_build/html && python -m http.server 8080 + +# Security +security-check: ## Run security checks + pip-audit + bandit -r src/ + # npm audit # For Node.js projects + +# Release +release: clean test lint ## Prepare for release + @echo "Ready for release!" + +# Environment setup +setup: ## Set up development environment + python -m venv venv + @echo "Now run: source venv/bin/activate && make install-dev" + +# Git hooks +pre-commit: ## Run pre-commit hooks + pre-commit run --all-files + +# Performance +profile: ## Profile the application + python -m cProfile -o profile.stats src/main.py + python -c "import pstats; pstats.Stats('profile.stats').sort_stats('cumulative').print_stats(20)" \ No newline at end of file diff --git a/README.md b/README.md index 1298dd9..5334135 100644 --- a/README.md +++ b/README.md @@ -1 +1,100 @@ -# template \ No newline at end of file +# BioNanomics Project Template + +A comprehensive template for new BioNanomics projects, providing a standardized structure and development workflow. + +## ๐Ÿš€ Quick Start + +1. **Use this template** - Click "Use this template" button on GitHub or clone this repository +2. **Customize your project** - Replace placeholder content with your project details +3. **Set up development environment** - Follow the setup instructions below +4. **Start coding** - Begin development using the provided structure + +## ๐Ÿ“ Project Structure + +``` +โ”œโ”€โ”€ .github/ # GitHub workflows and templates +โ”œโ”€โ”€ docs/ # Documentation files +โ”œโ”€โ”€ src/ # Source code +โ”œโ”€โ”€ tests/ # Test files +โ”œโ”€โ”€ scripts/ # Utility scripts +โ”œโ”€โ”€ config/ # Configuration files +โ”œโ”€โ”€ data/ # Data files (if applicable) +โ”œโ”€โ”€ .gitignore # Git ignore rules +โ”œโ”€โ”€ .editorconfig # Editor configuration +โ”œโ”€โ”€ README.md # This file +โ”œโ”€โ”€ LICENSE # Project license +โ”œโ”€โ”€ CONTRIBUTING.md # Contribution guidelines +โ””โ”€โ”€ CHANGELOG.md # Project changelog +``` + +## ๐Ÿ› ๏ธ Setup Instructions + +### Prerequisites +- Git +- Your preferred development environment +- Language-specific requirements (see project-specific docs) + +### Installation +1. Clone the repository: + ```bash + git clone https://github.com/BioNanomics/[your-project-name] + cd [your-project-name] + ``` + +2. Follow language-specific setup instructions in the `docs/` directory + +## ๐Ÿงช Development Workflow + +### Testing +- Place tests in the `tests/` directory +- Follow testing conventions outlined in `docs/testing.md` + +### Documentation +- Keep documentation in the `docs/` directory +- Update `CHANGELOG.md` for notable changes + +### Contributing +- Follow guidelines in `CONTRIBUTING.md` +- Use feature branches for development +- Submit pull requests for review + +## ๐Ÿ“‹ Customization Checklist + +When using this template, update the following: + +- [ ] Project name in README.md +- [ ] Project description and purpose +- [ ] Setup instructions specific to your technology stack +- [ ] License file (if different from MIT) +- [ ] Contributing guidelines +- [ ] GitHub repository settings +- [ ] Language-specific configuration files +- [ ] Remove unused directories/files +- [ ] Update .gitignore for your specific needs + +## ๐Ÿ“š Documentation + +- [Setup Guide](docs/setup.md) - Detailed setup instructions +- [Development Guide](docs/development.md) - Development best practices +- [Testing Guide](docs/testing.md) - Testing strategies and tools +- [Deployment Guide](docs/deployment.md) - Deployment procedures + +## ๐Ÿค Contributing + +We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on: +- Code of conduct +- Development process +- Pull request procedure +- Issue reporting + +## ๐Ÿ“„ License + +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. + +## ๐Ÿข About BioNanomics + +This template is maintained by BioNanomics. For more information about our projects and mission, visit [our organization page](https://github.com/BioNanomics). + +--- + +**Note**: This is a template repository. Replace this content with your actual project information when using the template. \ No newline at end of file diff --git a/config/README.md b/config/README.md new file mode 100644 index 0000000..99560f9 --- /dev/null +++ b/config/README.md @@ -0,0 +1,40 @@ +# Configuration Directory + +This directory contains configuration files for the project. + +## Files + +Place environment-specific and application configuration files here: + +- `settings.py` - Application settings +- `database.py` - Database configuration +- `logging.conf` - Logging configuration +- `nginx.conf` - Web server configuration +- `docker-compose.yml` - Docker composition +- `.env.example` - Environment variables template + +## Environment Variables + +Create a `.env.example` file in the project root with template variables: + +```bash +# Database +DATABASE_URL=sqlite:///app.db +DATABASE_PASSWORD=change_me + +# API Keys +API_KEY=your_api_key_here +SECRET_KEY=your_secret_key_here + +# Application +DEBUG=false +LOG_LEVEL=INFO +``` + +## Guidelines + +- Never commit sensitive information +- Use environment variables for secrets +- Provide example/template configuration files +- Document all configuration options +- Use different configurations for different environments \ No newline at end of file diff --git a/data/README.md b/data/README.md new file mode 100644 index 0000000..079725e --- /dev/null +++ b/data/README.md @@ -0,0 +1,124 @@ +# Data Directory + +This directory is for storing data files used by your project. + +## Structure + +Organize your data files according to your project's needs: + +``` +data/ +โ”œโ”€โ”€ raw/ # Raw, unprocessed data +โ”œโ”€โ”€ processed/ # Cleaned and processed data +โ”œโ”€โ”€ external/ # Data from external sources +โ”œโ”€โ”€ interim/ # Intermediate data during processing +โ”œโ”€โ”€ fixtures/ # Test data and fixtures +โ””โ”€โ”€ README.md # This file +``` + +## Guidelines + +### Data Management +- Keep raw data immutable +- Version control small data files only +- Use `.gitignore` for large data files +- Document data sources and formats +- Include data validation scripts + +### Security +- Never commit sensitive data +- Use environment variables for data paths +- Encrypt sensitive data at rest +- Follow data privacy regulations + +### Large Files +For large data files, consider: +- Git LFS (Large File Storage) +- External storage services (AWS S3, Google Cloud) +- Data version control tools (DVC) +- Database storage for structured data + +## Data Types + +### Sample Data (fixtures/) +Small datasets for testing and examples: +- JSON configuration files +- CSV sample data +- Test images or documents +- Mock API responses + +### External Data (external/) +Data from external sources: +- API downloads +- Third-party datasets +- Partner data feeds +- Reference data + +### Processed Data (processed/) +Cleaned and processed datasets: +- Normalized data +- Feature-engineered datasets +- Aggregated summaries +- Model-ready data + +## Example .gitignore Patterns + +Add to your `.gitignore`: +``` +# Large data files +data/raw/*.csv +data/raw/*.xlsx +data/raw/*.json +data/processed/*.parquet +data/external/*.zip + +# Keep structure but ignore contents +data/*/ +!data/*/README.md +!data/fixtures/ +``` + +## Data Access Patterns + +### Python Example +```python +import os +from pathlib import Path + +# Data directory paths +DATA_DIR = Path(__file__).parent / "data" +RAW_DATA_DIR = DATA_DIR / "raw" +PROCESSED_DATA_DIR = DATA_DIR / "processed" + +def load_data(filename): + """Load data from the appropriate directory.""" + return pd.read_csv(RAW_DATA_DIR / filename) +``` + +### Environment Variables +```bash +# .env +DATA_ROOT=/path/to/data +RAW_DATA_PATH=${DATA_ROOT}/raw +PROCESSED_DATA_PATH=${DATA_ROOT}/processed +``` + +## Data Validation + +Create validation scripts: +```python +def validate_data(df): + """Validate data quality.""" + assert not df.empty, "Data is empty" + assert df.isnull().sum().sum() == 0, "Data contains null values" + return True +``` + +## Documentation + +Document your data: +- Source and collection method +- Data dictionary/schema +- Known issues or limitations +- Processing steps applied +- Update frequency \ No newline at end of file diff --git a/docker-compose.yml.template b/docker-compose.yml.template new file mode 100644 index 0000000..54e94bd --- /dev/null +++ b/docker-compose.yml.template @@ -0,0 +1,127 @@ +version: '3.8' + +services: + # Main application service + app: + build: + context: . + dockerfile: Dockerfile + target: development # Use production for prod + ports: + - "8000:8000" + environment: + - DATABASE_URL=postgresql://user:password@db:5432/appdb + - REDIS_URL=redis://redis:6379/0 + - DEBUG=true + env_file: + - .env + volumes: + - ./src:/app/src # Mount source for development + - ./config:/app/config + depends_on: + - db + - redis + networks: + - app-network + restart: unless-stopped + + # Database service + db: + image: postgres:13-alpine + environment: + POSTGRES_DB: appdb + POSTGRES_USER: user + POSTGRES_PASSWORD: password + volumes: + - postgres_data:/var/lib/postgresql/data + - ./scripts/init.sql:/docker-entrypoint-initdb.d/init.sql + ports: + - "5432:5432" # Remove in production + networks: + - app-network + restart: unless-stopped + + # Redis service (for caching/sessions) + redis: + image: redis:7-alpine + ports: + - "6379:6379" # Remove in production + volumes: + - redis_data:/data + networks: + - app-network + restart: unless-stopped + command: redis-server --appendonly yes + + # Nginx reverse proxy (production) + nginx: + image: nginx:alpine + ports: + - "80:80" + - "443:443" + volumes: + - ./config/nginx.conf:/etc/nginx/nginx.conf + - ./config/ssl:/etc/ssl/certs + - static_files:/app/static + depends_on: + - app + networks: + - app-network + restart: unless-stopped + profiles: + - production + + # Worker service (for background tasks) + worker: + build: + context: . + dockerfile: Dockerfile + environment: + - DATABASE_URL=postgresql://user:password@db:5432/appdb + - REDIS_URL=redis://redis:6379/0 + env_file: + - .env + depends_on: + - db + - redis + networks: + - app-network + restart: unless-stopped + command: python -m src.worker + profiles: + - production + + # Monitoring (optional) + prometheus: + image: prom/prometheus + ports: + - "9090:9090" + volumes: + - ./config/prometheus.yml:/etc/prometheus/prometheus.yml + networks: + - app-network + profiles: + - monitoring + + grafana: + image: grafana/grafana + ports: + - "3000:3000" + environment: + - GF_SECURITY_ADMIN_PASSWORD=admin + volumes: + - grafana_data:/var/lib/grafana + networks: + - app-network + profiles: + - monitoring + +volumes: + postgres_data: + redis_data: + static_files: + grafana_data: + +networks: + app-network: + driver: bridge \ No newline at end of file diff --git a/docs/TEMPLATE_USAGE.md b/docs/TEMPLATE_USAGE.md new file mode 100644 index 0000000..1ef0e38 --- /dev/null +++ b/docs/TEMPLATE_USAGE.md @@ -0,0 +1,276 @@ +# Template Usage Guide + +This guide explains how to use this BioNanomics project template to create a new project. + +## Getting Started + +### 1. Create New Repository from Template + +#### Method A: Using GitHub UI +1. Navigate to https://github.com/BioNanomics/template +2. Click "Use this template" button +3. Choose "Create a new repository" +4. Fill in your repository details: + - Repository name + - Description + - Public/Private setting +5. Click "Create repository from template" + +#### Method B: Using GitHub CLI +```bash +gh repo create your-project-name --template BioNanomics/template --public +cd your-project-name +``` + +### 2. Customize Your Project + +Work through this checklist to customize the template for your specific project: + +#### Basic Information +- [ ] Update `README.md` with your project details +- [ ] Replace `[your-project-name]` placeholders throughout files +- [ ] Update `CHANGELOG.md` with your initial version +- [ ] Modify `LICENSE` if using a different license + +#### Project Configuration +- [ ] Choose and configure your tech stack: + - [ ] Python: Use `pyproject.toml.template` โ†’ `pyproject.toml` + - [ ] Node.js: Use `package.json.template` โ†’ `package.json` + - [ ] Docker: Use `Dockerfile.template` โ†’ `Dockerfile` +- [ ] Update `requirements.txt` and `requirements-dev.txt` (Python) +- [ ] Customize `.gitignore` for your specific needs +- [ ] Configure `.env.example` with your environment variables + +#### Development Setup +- [ ] Update GitHub Actions workflows in `.github/workflows/` +- [ ] Customize pre-commit hooks in `.pre-commit-config.yaml` +- [ ] Modify `Makefile` commands for your project +- [ ] Update setup script `scripts/setup.sh` + +#### Documentation +- [ ] Customize documentation in `docs/` directory +- [ ] Update setup instructions in `docs/setup.md` +- [ ] Modify development guide in `docs/development.md` +- [ ] Adjust testing guide in `docs/testing.md` +- [ ] Update deployment guide in `docs/deployment.md` + +#### GitHub Configuration +- [ ] Customize issue templates in `.github/ISSUE_TEMPLATE/` +- [ ] Update pull request template in `.github/PULL_REQUEST_TEMPLATE/` +- [ ] Configure repository settings (branch protection, etc.) + +#### Clean Up +- [ ] Remove unused template files (`.template` extensions) +- [ ] Delete directories you don't need +- [ ] Remove this usage guide (`docs/TEMPLATE_USAGE.md`) + +### 3. Project-Specific Setup + +#### For Python Projects +```bash +# Rename template files +mv pyproject.toml.template pyproject.toml +mv Dockerfile.template Dockerfile +mv docker-compose.yml.template docker-compose.yml + +# Create virtual environment +python -m venv venv +source venv/bin/activate # On Windows: venv\Scripts\activate + +# Install dependencies +pip install -r requirements-dev.txt + +# Set up pre-commit hooks +pre-commit install + +# Initialize your source code +mkdir -p src/your_package +touch src/your_package/__init__.py +``` + +#### For Node.js Projects +```bash +# Rename template files +mv package.json.template package.json +mv Dockerfile.template Dockerfile +mv docker-compose.yml.template docker-compose.yml + +# Install dependencies +npm install + +# Set up pre-commit hooks (if using) +npx husky install +``` + +#### For Docker Projects +```bash +# Rename template files +mv Dockerfile.template Dockerfile +mv docker-compose.yml.template docker-compose.yml + +# Build and test +docker build -t your-project-name . +docker-compose up -d +``` + +## Template Structure Explained + +### Core Files +- `README.md`: Main project documentation +- `LICENSE`: MIT license (customize if needed) +- `CHANGELOG.md`: Version history tracking +- `CONTRIBUTING.md`: Contribution guidelines +- `.gitignore`: Comprehensive ignore patterns +- `.editorconfig`: Editor configuration +- `.env.example`: Environment variables template + +### Development Configuration +- `.pre-commit-config.yaml`: Pre-commit hooks +- `Makefile`: Common development commands +- `requirements*.txt`: Python dependencies +- `package.json.template`: Node.js configuration +- `pyproject.toml.template`: Modern Python project config + +### Directory Structure +- `src/`: Source code +- `tests/`: Test files +- `docs/`: Documentation +- `scripts/`: Utility scripts +- `config/`: Configuration files +- `data/`: Data files (if applicable) + +### GitHub Integration +- `.github/workflows/`: CI/CD workflows +- `.github/ISSUE_TEMPLATE/`: Issue templates +- `.github/PULL_REQUEST_TEMPLATE/`: PR templates + +### Docker Support +- `Dockerfile.template`: Multi-stage Docker build +- `docker-compose.yml.template`: Development environment + +## Best Practices + +### Version Control +- Use meaningful commit messages +- Follow conventional commits format +- Create feature branches for development +- Use pull requests for code review + +### Code Quality +- Set up automated linting and formatting +- Write comprehensive tests +- Use type hints (Python) or TypeScript +- Document your code + +### Security +- Never commit secrets or credentials +- Use environment variables for configuration +- Keep dependencies updated +- Run security scans regularly + +### Documentation +- Keep README up to date +- Document API endpoints +- Write clear setup instructions +- Maintain changelog + +## Customization Examples + +### Adding New Dependencies + +#### Python +```bash +# Add to requirements.txt +echo "fastapi>=0.68.0" >> requirements.txt + +# For development dependencies +echo "pytest-asyncio>=0.18.0" >> requirements-dev.txt +``` + +#### Node.js +```bash +# Production dependency +npm install express + +# Development dependency +npm install --save-dev jest +``` + +### Adding New Scripts + +#### Makefile +```makefile +migrate: ## Run database migrations + python manage.py migrate + +deploy: ## Deploy to production + ./scripts/deploy.sh +``` + +#### package.json +```json +{ + "scripts": { + "migrate": "npx sequelize-cli db:migrate", + "seed": "npx sequelize-cli db:seed:all" + } +} +``` + +### Custom GitHub Actions + +Create `.github/workflows/custom.yml`: +```yaml +name: Custom Workflow + +on: + push: + branches: [main] + +jobs: + custom-job: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Custom step + run: echo "Add your custom logic here" +``` + +## Troubleshooting + +### Common Issues + +1. **Template files not renamed**: Remember to rename `.template` files +2. **Dependencies not found**: Run installation commands after customization +3. **Tests failing**: Update test configuration for your specific setup +4. **CI/CD not working**: Check GitHub Actions and update for your tech stack + +### Getting Help + +- Check existing issues in the template repository +- Create a new issue if you find a problem +- Join BioNanomics community discussions +- Review documentation in the `docs/` directory + +## Contributing Back to Template + +If you make improvements that would benefit other projects: + +1. Fork the template repository +2. Create a feature branch +3. Make your improvements +4. Submit a pull request +5. Include examples and documentation + +## Next Steps + +After customizing your template: + +1. Set up your development environment +2. Write your first test +3. Implement your core functionality +4. Set up continuous integration +5. Write comprehensive documentation +6. Plan your first release + +Happy coding! ๐Ÿš€ \ No newline at end of file diff --git a/docs/deployment.md b/docs/deployment.md new file mode 100644 index 0000000..dbcfce8 --- /dev/null +++ b/docs/deployment.md @@ -0,0 +1,536 @@ +# Deployment Guide + +This guide covers deployment strategies and procedures for this project. + +## Deployment Overview + +This project supports multiple deployment environments: + +- **Development**: Local development environment +- **Staging**: Pre-production testing environment +- **Production**: Live production environment + +## Prerequisites + +Before deploying, ensure you have: + +- Proper access credentials +- Environment-specific configuration +- Required dependencies installed +- Database migrations ready (if applicable) + +## Environment Configuration + +### Environment Variables + +Create environment-specific configuration files: + +```bash +# .env.development +DEBUG=true +DATABASE_URL=sqlite:///dev.db +API_KEY=dev_key_here + +# .env.staging +DEBUG=false +DATABASE_URL=postgresql://user:pass@staging-db:5432/app +API_KEY=staging_key_here + +# .env.production +DEBUG=false +DATABASE_URL=postgresql://user:pass@prod-db:5432/app +API_KEY=prod_key_here +``` + +### Configuration Management + +```python +# config.py example +import os +from pathlib import Path + +class Config: + """Base configuration.""" + SECRET_KEY = os.environ.get('SECRET_KEY') + DATABASE_URL = os.environ.get('DATABASE_URL') + +class DevelopmentConfig(Config): + """Development configuration.""" + DEBUG = True + TESTING = False + +class StagingConfig(Config): + """Staging configuration.""" + DEBUG = False + TESTING = False + +class ProductionConfig(Config): + """Production configuration.""" + DEBUG = False + TESTING = False + +config = { + 'development': DevelopmentConfig, + 'staging': StagingConfig, + 'production': ProductionConfig, + 'default': DevelopmentConfig +} +``` + +## Deployment Methods + +### 1. Manual Deployment + +#### Local Development + +```bash +# Start development server +python manage.py runserver +# or +npm run dev +``` + +#### Staging/Production + +```bash +# 1. Pull latest code +git pull origin main + +# 2. Install dependencies +pip install -r requirements.txt +# or +npm install --production + +# 3. Run database migrations +python manage.py migrate +# or +npm run migrate + +# 4. Collect static files (if applicable) +python manage.py collectstatic --noinput + +# 5. Restart application server +sudo systemctl restart your-app +``` + +### 2. Docker Deployment + +#### Dockerfile + +```dockerfile +FROM python:3.9-slim + +WORKDIR /app + +# Install dependencies +COPY requirements.txt . +RUN pip install -r requirements.txt + +# Copy application +COPY src/ ./src/ +COPY manage.py . + +# Expose port +EXPOSE 8000 + +# Run application +CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"] +``` + +#### Docker Compose + +```yaml +version: '3.8' + +services: + web: + build: . + ports: + - "8000:8000" + environment: + - DATABASE_URL=postgresql://user:pass@db:5432/app + depends_on: + - db + volumes: + - ./src:/app/src + + db: + image: postgres:13 + environment: + POSTGRES_DB: app + POSTGRES_USER: user + POSTGRES_PASSWORD: pass + volumes: + - postgres_data:/var/lib/postgresql/data + +volumes: + postgres_data: +``` + +#### Deployment Commands + +```bash +# Build and start services +docker-compose up -d + +# View logs +docker-compose logs -f + +# Run migrations +docker-compose exec web python manage.py migrate + +# Scale services +docker-compose up -d --scale web=3 +``` + +### 3. Cloud Platform Deployment + +#### Heroku + +```bash +# Install Heroku CLI and login +heroku login + +# Create application +heroku create your-app-name + +# Set environment variables +heroku config:set SECRET_KEY=your-secret-key +heroku config:set DATABASE_URL=your-db-url + +# Deploy +git push heroku main + +# Run migrations +heroku run python manage.py migrate +``` + +#### AWS Elastic Beanstalk + +```bash +# Install EB CLI +pip install awsebcli + +# Initialize application +eb init + +# Create environment +eb create staging + +# Deploy +eb deploy + +# View logs +eb logs +``` + +#### Google Cloud Platform + +```yaml +# app.yaml for App Engine +runtime: python39 + +env_variables: + SECRET_KEY: "your-secret-key" + DATABASE_URL: "your-db-connection-string" + +automatic_scaling: + min_instances: 1 + max_instances: 10 +``` + +```bash +# Deploy to App Engine +gcloud app deploy +``` + +### 4. CI/CD Pipeline + +#### GitHub Actions + +```yaml +# .github/workflows/deploy.yml +name: Deploy + +on: + push: + branches: [main] + +jobs: + deploy: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: 3.9 + + - name: Install dependencies + run: | + pip install -r requirements.txt + + - name: Run tests + run: | + pytest + + - name: Deploy to staging + if: github.ref == 'refs/heads/main' + run: | + # Add deployment commands here + echo "Deploying to staging..." + + - name: Deploy to production + if: github.ref == 'refs/heads/main' && github.event_name == 'push' + run: | + # Add production deployment commands + echo "Deploying to production..." +``` + +## Database Migrations + +### Before Deployment + +```bash +# Create migration files +python manage.py makemigrations + +# Review migrations +python manage.py showmigrations + +# Test migrations on staging data +python manage.py migrate --dry-run +``` + +### During Deployment + +```bash +# Apply migrations +python manage.py migrate + +# If rollback needed +python manage.py migrate app_name 0001 # Rollback to specific migration +``` + +## Health Checks + +### Application Health Check + +```python +# health_check.py +def health_check(): + """Basic health check endpoint.""" + try: + # Check database connection + db.engine.execute('SELECT 1') + + # Check external services + external_service.ping() + + return {"status": "healthy", "timestamp": datetime.utcnow()} + except Exception as e: + return {"status": "unhealthy", "error": str(e)} +``` + +### Monitoring + +```bash +# Check application status +curl https://your-app.com/health + +# Monitor logs +tail -f /var/log/your-app/app.log + +# Check resource usage +htop +df -h +``` + +## Rollback Procedures + +### Quick Rollback + +```bash +# Rollback to previous version +git checkout HEAD~1 +# or use specific commit +git checkout abc123 + +# Redeploy +./deploy.sh +``` + +### Database Rollback + +```bash +# Rollback database migrations +python manage.py migrate app_name 0001 + +# Restore from backup if needed +pg_restore -d database_name backup_file.sql +``` + +## Security Considerations + +### Secrets Management + +- Never commit secrets to version control +- Use environment variables or secret management services +- Rotate secrets regularly +- Use different secrets for each environment + +### SSL/TLS + +```nginx +# Nginx configuration example +server { + listen 443 ssl; + server_name your-domain.com; + + ssl_certificate /path/to/certificate.crt; + ssl_certificate_key /path/to/private.key; + + location / { + proxy_pass http://localhost:8000; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + } +} +``` + +### Firewall Rules + +```bash +# Allow only necessary ports +ufw allow 22 # SSH +ufw allow 80 # HTTP +ufw allow 443 # HTTPS +ufw enable +``` + +## Performance Optimization + +### Static Files + +```bash +# Collect and compress static files +python manage.py collectstatic --noinput +python manage.py compress +``` + +### Caching + +```python +# Redis caching example +CACHES = { + 'default': { + 'BACKEND': 'django_redis.cache.RedisCache', + 'LOCATION': 'redis://127.0.0.1:6379/1', + 'OPTIONS': { + 'CLIENT_CLASS': 'django_redis.client.DefaultClient', + } + } +} +``` + +### Load Balancing + +```nginx +# Nginx load balancer +upstream app_servers { + server 127.0.0.1:8000; + server 127.0.0.1:8001; + server 127.0.0.1:8002; +} + +server { + location / { + proxy_pass http://app_servers; + } +} +``` + +## Monitoring and Logging + +### Application Metrics + +- Response times +- Error rates +- Resource usage +- Active users + +### Log Management + +```python +# Structured logging +import logging +import json + +logger = logging.getLogger(__name__) + +def log_request(request, response): + log_data = { + 'timestamp': datetime.utcnow().isoformat(), + 'method': request.method, + 'url': request.url, + 'status_code': response.status_code, + 'response_time': response.time + } + logger.info(json.dumps(log_data)) +``` + +## Troubleshooting + +### Common Issues + +1. **Application won't start**: Check environment variables and dependencies +2. **Database connection errors**: Verify database credentials and network connectivity +3. **Static files not loading**: Check static file configuration and permissions +4. **High memory usage**: Profile application and optimize queries + +### Debug Mode + +```bash +# Enable debug logging +export LOG_LEVEL=DEBUG + +# Run with verbose output +python manage.py runserver --verbosity=2 +``` + +## Backup and Recovery + +### Database Backups + +```bash +# Create backup +pg_dump database_name > backup_$(date +%Y%m%d).sql + +# Automated backup script +#!/bin/bash +BACKUP_DIR="/backups" +DATE=$(date +%Y%m%d_%H%M%S) +pg_dump database_name | gzip > "$BACKUP_DIR/backup_$DATE.sql.gz" + +# Keep only last 7 days of backups +find $BACKUP_DIR -name "backup_*.sql.gz" -mtime +7 -delete +``` + +### Application Data + +```bash +# Backup uploaded files +tar -czf files_backup_$(date +%Y%m%d).tar.gz /app/media/ + +# Sync to remote storage +aws s3 sync /app/media/ s3://your-backup-bucket/media/ +``` + +## Resources + +- [Deployment Best Practices](https://12factor.net/) +- [Docker Documentation](https://docs.docker.com/) +- [Cloud Platform Documentation](relevant-cloud-docs) +- [Monitoring Tools](monitoring-tools-docs) \ No newline at end of file diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..9711674 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,253 @@ +# Development Guide + +This guide covers best practices and conventions for developing this project. + +## Project Structure + +``` +src/ # Source code +โ”œโ”€โ”€ main/ # Main application code +โ”œโ”€โ”€ utils/ # Utility functions +โ”œโ”€โ”€ config/ # Configuration modules +โ””โ”€โ”€ __init__.py # Package initialization + +tests/ # Test files +โ”œโ”€โ”€ unit/ # Unit tests +โ”œโ”€โ”€ integration/ # Integration tests +โ”œโ”€โ”€ fixtures/ # Test fixtures +โ””โ”€โ”€ conftest.py # Pytest configuration + +docs/ # Documentation +scripts/ # Utility scripts +config/ # Configuration files +``` + +## Coding Standards + +### General Principles + +- Write clean, readable, and maintainable code +- Follow the DRY (Don't Repeat Yourself) principle +- Use meaningful variable and function names +- Write comprehensive tests +- Document your code appropriately + +### Code Style + +This project follows [specific style guide, e.g., PEP 8 for Python, Airbnb for JavaScript]. + +#### Naming Conventions + +- **Variables and functions**: `snake_case` (Python) or `camelCase` (JavaScript) +- **Classes**: `PascalCase` +- **Constants**: `UPPER_SNAKE_CASE` +- **Files and modules**: `snake_case.py` or `kebab-case.js` + +#### Comments and Documentation + +- Use docstrings for all public functions and classes +- Write comments for complex logic +- Keep comments up-to-date with code changes + +```python +def calculate_efficiency(input_data: dict) -> float: + """ + Calculate the efficiency based on input parameters. + + Args: + input_data (dict): Dictionary containing calculation parameters + + Returns: + float: Calculated efficiency value + + Raises: + ValueError: If input_data is missing required keys + """ + pass +``` + +## Development Workflow + +### Branching Strategy + +We use Git Flow branching model: + +- `main`: Production-ready code +- `develop`: Integration branch for features +- `feature/*`: Feature development branches +- `hotfix/*`: Critical bug fixes +- `release/*`: Release preparation branches + +### Feature Development + +1. Create a feature branch from `develop`: + ```bash + git checkout develop + git pull origin develop + git checkout -b feature/your-feature-name + ``` + +2. Develop your feature with frequent commits: + ```bash + git add . + git commit -m "Add: specific change description" + ``` + +3. Write tests for your feature +4. Update documentation if needed +5. Push your branch and create a pull request + +### Commit Messages + +Follow conventional commit format: + +``` +type(scope): description + +[optional body] + +[optional footer] +``` + +Types: +- `feat`: New feature +- `fix`: Bug fix +- `docs`: Documentation changes +- `style`: Code style changes +- `refactor`: Code refactoring +- `test`: Adding or updating tests +- `chore`: Maintenance tasks + +Examples: +``` +feat(auth): add user authentication system +fix(api): resolve data validation issue +docs(readme): update installation instructions +``` + +## Testing + +### Test Structure + +- **Unit tests**: Test individual functions/methods in isolation +- **Integration tests**: Test component interactions +- **End-to-end tests**: Test complete workflows + +### Writing Tests + +```python +import pytest +from src.main.calculator import calculate_efficiency + +class TestCalculateEfficiency: + def test_valid_input(self): + """Test calculation with valid input.""" + input_data = {"param1": 10, "param2": 20} + result = calculate_efficiency(input_data) + assert result == expected_value + + def test_invalid_input(self): + """Test calculation with invalid input.""" + with pytest.raises(ValueError): + calculate_efficiency({}) +``` + +### Running Tests + +```bash +# Run all tests +pytest + +# Run specific test file +pytest tests/test_calculator.py + +# Run with coverage +pytest --cov=src + +# Run specific test +pytest tests/test_calculator.py::TestCalculateEfficiency::test_valid_input +``` + +## Debugging + +### Logging + +Use structured logging throughout the application: + +```python +import logging + +logger = logging.getLogger(__name__) + +def process_data(data): + logger.info("Processing data", extra={"data_size": len(data)}) + try: + # Process data + logger.debug("Data processed successfully") + except Exception as e: + logger.error("Failed to process data", exc_info=True) + raise +``` + +### Debug Tools + +- Use debugger (`pdb` for Python, Chrome DevTools for JavaScript) +- Add strategic print statements for quick debugging +- Use IDE debugging features + +## Performance Considerations + +- Profile your code to identify bottlenecks +- Use appropriate data structures +- Consider memory usage for large datasets +- Implement caching where appropriate + +## Security Best Practices + +- Never commit secrets or credentials +- Validate and sanitize all inputs +- Use secure communication protocols +- Keep dependencies updated +- Follow security guidelines for your language/framework + +## Documentation + +### API Documentation + +- Document all public APIs +- Include examples and usage instructions +- Keep documentation synchronized with code + +### Code Documentation + +- Write clear docstrings +- Explain complex algorithms +- Document assumptions and limitations + +## Review Process + +### Before Submitting PR + +- [ ] Code follows project style guidelines +- [ ] All tests pass +- [ ] New code has appropriate test coverage +- [ ] Documentation is updated +- [ ] No sensitive information is committed + +### Code Review Guidelines + +- Be constructive and respectful +- Focus on code, not the person +- Explain the reasoning behind suggestions +- Approve when code meets standards + +## Deployment + +See [Deployment Guide](deployment.md) for detailed deployment instructions. + +## Resources + +- [Project Documentation](../README.md) +- [Contributing Guidelines](../CONTRIBUTING.md) +- [Testing Guide](testing.md) +- [Style Guide Reference](https://pep8.org/) (or relevant style guide) \ No newline at end of file diff --git a/docs/setup.md b/docs/setup.md new file mode 100644 index 0000000..14d731c --- /dev/null +++ b/docs/setup.md @@ -0,0 +1,150 @@ +# Setup Guide + +This guide will help you set up the development environment for this project. + +## Prerequisites + +Before you begin, ensure you have the following installed: + +- Git +- [Language/Runtime] (e.g., Python 3.8+, Node.js 16+) +- [Package Manager] (e.g., pip, npm, yarn) +- [Additional Tools] (if required) + +## Installation + +### 1. Clone the Repository + +```bash +git clone https://github.com/BioNanomics/[project-name].git +cd [project-name] +``` + +### 2. Set Up Virtual Environment (Python example) + +```bash +# Create virtual environment +python -m venv venv + +# Activate virtual environment +# On Windows: +venv\Scripts\activate +# On macOS/Linux: +source venv/bin/activate +``` + +### 3. Install Dependencies + +```bash +# Python example +pip install -r requirements.txt +pip install -r requirements-dev.txt # Development dependencies + +# Node.js example +npm install +# or +yarn install +``` + +### 4. Environment Configuration + +Copy the example environment file and configure it: + +```bash +cp .env.example .env +# Edit .env with your specific configuration +``` + +### 5. Initialize Database (if applicable) + +```bash +# Example database setup commands +python manage.py migrate +# or +npm run db:setup +``` + +### 6. Verify Installation + +Run the test suite to ensure everything is working: + +```bash +# Python example +pytest + +# Node.js example +npm test +``` + +## Development Tools + +### Code Formatting + +This project uses automated code formatting: + +```bash +# Python example +black . +isort . + +# Node.js example +npm run format +``` + +### Linting + +```bash +# Python example +flake8 . +pylint src/ + +# Node.js example +npm run lint +``` + +### Pre-commit Hooks (Optional) + +Set up pre-commit hooks to automatically format and lint code: + +```bash +pip install pre-commit +pre-commit install +``` + +## IDE Configuration + +### Visual Studio Code + +Recommended extensions: +- Python (if using Python) +- ESLint (if using JavaScript/TypeScript) +- Prettier +- GitLens + +### PyCharm/IntelliJ + +Configure the interpreter to use your virtual environment. + +## Troubleshooting + +### Common Issues + +1. **Dependency conflicts**: Try recreating your virtual environment +2. **Permission errors**: Ensure you have proper permissions for the project directory +3. **Port conflicts**: Check if required ports are available + +### Getting Help + +If you encounter issues: + +1. Check the [FAQ](faq.md) +2. Search existing [issues](https://github.com/BioNanomics/[project-name]/issues) +3. Create a new issue with detailed information + +## Next Steps + +After successful setup: + +1. Read the [Development Guide](development.md) +2. Check out the [Testing Guide](testing.md) +3. Review the [Contributing Guidelines](../CONTRIBUTING.md) \ No newline at end of file diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000..581fbc7 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,434 @@ +# Testing Guide + +This guide covers testing strategies, tools, and best practices for this project. + +## Testing Philosophy + +We follow a comprehensive testing approach that includes: + +- **Unit Tests**: Test individual components in isolation +- **Integration Tests**: Test component interactions +- **End-to-End Tests**: Test complete user workflows +- **Performance Tests**: Ensure acceptable performance under load + +## Testing Framework + +This project uses: +- **[Testing Framework]** (e.g., pytest for Python, Jest for JavaScript) +- **[Mocking Library]** (e.g., unittest.mock, jest.mock) +- **[Coverage Tool]** (e.g., coverage.py, jest coverage) + +## Test Structure + +``` +tests/ +โ”œโ”€โ”€ unit/ # Unit tests +โ”‚ โ”œโ”€โ”€ test_calculator.py +โ”‚ โ”œโ”€โ”€ test_validator.py +โ”‚ โ””โ”€โ”€ __init__.py +โ”œโ”€โ”€ integration/ # Integration tests +โ”‚ โ”œโ”€โ”€ test_api.py +โ”‚ โ”œโ”€โ”€ test_database.py +โ”‚ โ””โ”€โ”€ __init__.py +โ”œโ”€โ”€ e2e/ # End-to-end tests +โ”‚ โ”œโ”€โ”€ test_user_flows.py +โ”‚ โ””โ”€โ”€ __init__.py +โ”œโ”€โ”€ fixtures/ # Test data and fixtures +โ”‚ โ”œโ”€โ”€ sample_data.json +โ”‚ โ””โ”€โ”€ mock_responses.py +โ”œโ”€โ”€ conftest.py # Pytest configuration +โ””โ”€โ”€ __init__.py +``` + +## Writing Tests + +### Unit Tests + +Unit tests should test individual functions or methods in isolation: + +```python +import pytest +from unittest.mock import Mock, patch +from src.calculator import Calculator + +class TestCalculator: + def setup_method(self): + """Set up test fixtures before each test method.""" + self.calculator = Calculator() + + def test_add_positive_numbers(self): + """Test addition of positive numbers.""" + result = self.calculator.add(2, 3) + assert result == 5 + + def test_add_negative_numbers(self): + """Test addition of negative numbers.""" + result = self.calculator.add(-2, -3) + assert result == -5 + + def test_divide_by_zero(self): + """Test division by zero raises appropriate exception.""" + with pytest.raises(ZeroDivisionError): + self.calculator.divide(10, 0) + + @patch('src.calculator.external_service') + def test_with_external_dependency(self, mock_service): + """Test function that depends on external service.""" + mock_service.get_data.return_value = {"value": 10} + result = self.calculator.calculate_with_service() + assert result == 10 + mock_service.get_data.assert_called_once() +``` + +### Integration Tests + +Integration tests verify that components work together correctly: + +```python +import pytest +from src.api import create_app +from src.database import db + +@pytest.fixture +def client(): + """Create test client.""" + app = create_app(testing=True) + with app.test_client() as client: + with app.app_context(): + db.create_all() + yield client + db.drop_all() + +def test_user_creation_flow(client): + """Test complete user creation flow.""" + # Create user + response = client.post('/api/users', json={ + 'name': 'Test User', + 'email': 'test@example.com' + }) + assert response.status_code == 201 + + # Verify user exists + user_id = response.json['id'] + response = client.get(f'/api/users/{user_id}') + assert response.status_code == 200 + assert response.json['name'] == 'Test User' +``` + +### End-to-End Tests + +E2E tests simulate real user interactions: + +```python +import pytest +from selenium import webdriver +from selenium.webdriver.common.by import By + +@pytest.fixture +def browser(): + """Set up browser for E2E tests.""" + driver = webdriver.Chrome() + yield driver + driver.quit() + +def test_user_login_flow(browser): + """Test complete user login flow.""" + browser.get("http://localhost:3000/login") + + # Enter credentials + browser.find_element(By.ID, "email").send_keys("test@example.com") + browser.find_element(By.ID, "password").send_keys("password") + browser.find_element(By.ID, "login-button").click() + + # Verify successful login + assert "Dashboard" in browser.title + assert browser.find_element(By.CLASS_NAME, "welcome-message").is_displayed() +``` + +## Test Data and Fixtures + +### Using Fixtures + +Create reusable test data with fixtures: + +```python +import pytest +from src.models import User + +@pytest.fixture +def sample_user(): + """Create a sample user for testing.""" + return User( + name="Test User", + email="test@example.com", + age=25 + ) + +@pytest.fixture +def user_data(): + """Sample user data dictionary.""" + return { + "name": "Test User", + "email": "test@example.com", + "preferences": {"theme": "dark"} + } + +def test_user_creation(sample_user): + """Test using fixture.""" + assert sample_user.name == "Test User" + assert sample_user.is_valid() +``` + +### Parametrized Tests + +Test multiple scenarios efficiently: + +```python +import pytest + +@pytest.mark.parametrize("input_value,expected", [ + (0, 0), + (1, 1), + (2, 4), + (3, 9), + (-2, 4), +]) +def test_square_function(input_value, expected): + from src.math_utils import square + assert square(input_value) == expected +``` + +## Mocking + +### External Dependencies + +Mock external services and APIs: + +```python +from unittest.mock import Mock, patch +import requests + +@patch('requests.get') +def test_api_call(mock_get): + """Test function that makes HTTP request.""" + mock_response = Mock() + mock_response.json.return_value = {"status": "success"} + mock_response.status_code = 200 + mock_get.return_value = mock_response + + from src.api_client import fetch_data + result = fetch_data("http://api.example.com/data") + + assert result["status"] == "success" + mock_get.assert_called_once_with("http://api.example.com/data") +``` + +### Database Mocking + +```python +@patch('src.database.connection') +def test_database_query(mock_connection): + """Test database query function.""" + mock_cursor = Mock() + mock_cursor.fetchall.return_value = [("user1",), ("user2",)] + mock_connection.cursor.return_value = mock_cursor + + from src.user_service import get_all_users + users = get_all_users() + + assert len(users) == 2 + assert "user1" in users +``` + +## Running Tests + +### Basic Commands + +```bash +# Run all tests +pytest + +# Run specific test file +pytest tests/unit/test_calculator.py + +# Run specific test +pytest tests/unit/test_calculator.py::TestCalculator::test_add + +# Run tests with verbose output +pytest -v + +# Run tests and stop on first failure +pytest -x +``` + +### Coverage Reports + +```bash +# Run tests with coverage +pytest --cov=src + +# Generate HTML coverage report +pytest --cov=src --cov-report=html + +# Set minimum coverage threshold +pytest --cov=src --cov-fail-under=80 +``` + +### Parallel Execution + +```bash +# Install pytest-xdist +pip install pytest-xdist + +# Run tests in parallel +pytest -n auto +``` + +## Test Configuration + +### pytest.ini + +```ini +[tool:pytest] +testpaths = tests +python_files = test_*.py +python_classes = Test* +python_functions = test_* +addopts = + --strict-markers + --disable-warnings + --cov=src + --cov-report=term-missing + --cov-report=html:htmlcov + --cov-fail-under=80 +markers = + slow: marks tests as slow + integration: marks tests as integration tests + e2e: marks tests as end-to-end tests +``` + +### conftest.py + +```python +import pytest +from src.app import create_app +from src.database import db + +@pytest.fixture(scope="session") +def app(): + """Create application for testing.""" + app = create_app(testing=True) + return app + +@pytest.fixture(scope="function") +def client(app): + """Create test client.""" + return app.test_client() + +@pytest.fixture(scope="function") +def database(app): + """Create database for testing.""" + with app.app_context(): + db.create_all() + yield db + db.drop_all() +``` + +## Best Practices + +### Test Organization + +- Group related tests in classes +- Use descriptive test names +- Follow the AAA pattern (Arrange, Act, Assert) +- Keep tests independent and isolated + +### Test Quality + +- Test both happy path and edge cases +- Use appropriate assertions +- Avoid testing implementation details +- Keep tests simple and focused + +### Test Maintenance + +- Update tests when requirements change +- Remove obsolete tests +- Refactor tests to reduce duplication +- Review test coverage regularly + +## Continuous Integration + +Tests are automatically run on: +- Every push to feature branches +- Pull requests to main/develop +- Scheduled daily runs + +See `.github/workflows/ci.yml` for CI configuration. + +## Performance Testing + +### Load Testing + +```python +import time +import pytest + +def test_performance_requirement(): + """Test that function meets performance requirement.""" + start_time = time.time() + + # Call function under test + result = expensive_function() + + execution_time = time.time() - start_time + assert execution_time < 1.0 # Should complete within 1 second +``` + +### Memory Testing + +```python +import psutil +import os + +def test_memory_usage(): + """Test memory usage stays within bounds.""" + process = psutil.Process(os.getpid()) + initial_memory = process.memory_info().rss + + # Run memory-intensive operation + large_operation() + + final_memory = process.memory_info().rss + memory_increase = final_memory - initial_memory + + # Should not increase memory by more than 100MB + assert memory_increase < 100 * 1024 * 1024 +``` + +## Troubleshooting + +### Common Issues + +1. **Tests fail locally but pass in CI**: Check for environment differences +2. **Flaky tests**: Often caused by timing issues or external dependencies +3. **Slow test suite**: Profile tests and optimize or parallelize + +### Debugging Tests + +```bash +# Run with pdb on failures +pytest --pdb + +# Print output during tests +pytest -s + +# Run only failed tests from last run +pytest --lf +``` + +## Resources + +- [Pytest Documentation](https://docs.pytest.org/) +- [Testing Best Practices](https://docs.python-guide.org/writing/tests/) +- [Mock Documentation](https://docs.python.org/3/library/unittest.mock.html) \ No newline at end of file diff --git a/package.json.template b/package.json.template new file mode 100644 index 0000000..e526e7b --- /dev/null +++ b/package.json.template @@ -0,0 +1,46 @@ +{ + "name": "your-project-name", + "version": "1.0.0", + "description": "Your project description", + "main": "src/index.js", + "scripts": { + "start": "node src/index.js", + "dev": "nodemon src/index.js", + "test": "jest", + "test:watch": "jest --watch", + "test:coverage": "jest --coverage", + "lint": "eslint src/", + "lint:fix": "eslint src/ --fix", + "format": "prettier --write src/", + "format:check": "prettier --check src/", + "build": "webpack --mode production", + "build:dev": "webpack --mode development" + }, + "keywords": ["bionanomics", "template"], + "author": "BioNanomics", + "license": "MIT", + "dependencies": { + "express": "^4.18.0", + "dotenv": "^16.0.0" + }, + "devDependencies": { + "nodemon": "^2.0.0", + "jest": "^28.0.0", + "eslint": "^8.0.0", + "prettier": "^2.0.0", + "webpack": "^5.0.0", + "webpack-cli": "^4.0.0" + }, + "engines": { + "node": ">=16.0.0", + "npm": ">=8.0.0" + }, + "repository": { + "type": "git", + "url": "https://github.com/BioNanomics/your-project-name.git" + }, + "bugs": { + "url": "https://github.com/BioNanomics/your-project-name/issues" + }, + "homepage": "https://github.com/BioNanomics/your-project-name#readme" +} \ No newline at end of file diff --git a/pyproject.toml.template b/pyproject.toml.template new file mode 100644 index 0000000..4010a3e --- /dev/null +++ b/pyproject.toml.template @@ -0,0 +1,139 @@ +[build-system] +requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"] +build-backend = "setuptools.build_meta" + +[project] +name = "your-project-name" +description = "Your project description" +readme = "README.md" +license = {file = "LICENSE"} +authors = [ + {name = "BioNanomics", email = "contact@bionanomics.com"}, +] +maintainers = [ + {name = "BioNanomics", email = "contact@bionanomics.com"}, +] +keywords = ["bionanomics", "template"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", +] +requires-python = ">=3.8" +dependencies = [ + # Add your project dependencies here + # "requests>=2.25.0", + # "click>=8.0.0", +] +dynamic = ["version"] + +[project.optional-dependencies] +dev = [ + "pytest>=6.0.0", + "pytest-cov>=2.12.0", + "black>=21.0.0", + "isort>=5.9.0", + "flake8>=3.9.0", + "mypy>=0.910", + "pre-commit>=2.13.0", +] +docs = [ + "sphinx>=4.0.0", + "sphinx-rtd-theme>=0.5.0", +] + +[project.urls] +Homepage = "https://github.com/BioNanomics/your-project-name" +Repository = "https://github.com/BioNanomics/your-project-name.git" +Issues = "https://github.com/BioNanomics/your-project-name/issues" +Changelog = "https://github.com/BioNanomics/your-project-name/blob/main/CHANGELOG.md" + +[project.scripts] +# your-cli-command = "your_package.cli:main" + +[tool.setuptools] +package-dir = {"" = "src"} + +[tool.setuptools.packages.find] +where = ["src"] + +[tool.setuptools_scm] +write_to = "src/your_package/_version.py" + +[tool.black] +line-length = 88 +target-version = ['py38'] +include = '\.pyi?$' +extend-exclude = ''' +/( + # directories + \.eggs + | \.git + | \.hg + | \.mypy_cache + | \.tox + | \.venv + | build + | dist +)/ +''' + +[tool.isort] +profile = "black" +multi_line_output = 3 +line_length = 88 +known_first_party = ["your_package"] + +[tool.mypy] +python_version = "3.8" +warn_return_any = true +warn_unused_configs = true +disallow_untyped_defs = true +disallow_incomplete_defs = true +check_untyped_defs = true +disallow_untyped_decorators = true +no_implicit_optional = true +warn_redundant_casts = true +warn_unused_ignores = true +warn_no_return = true +warn_unreachable = true +strict_equality = true + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = [ + "--strict-markers", + "--strict-config", + "--cov=src", + "--cov-report=term-missing", + "--cov-report=html:htmlcov", + "--cov-fail-under=80", +] +markers = [ + "slow: marks tests as slow", + "integration: marks tests as integration tests", +] + +[tool.coverage.run] +source = ["src"] +omit = [ + "*/tests/*", + "*/test_*", + "*/__pycache__/*", + "*/venv/*", + "*/virtualenv/*", +] + +[tool.coverage.report] +exclude_lines = [ + "pragma: no cover", + "def __repr__", + "raise AssertionError", + "raise NotImplementedError", + "if __name__ == .__main__.:", +] \ No newline at end of file diff --git a/requirements-dev.txt b/requirements-dev.txt new file mode 100644 index 0000000..ceefb3c --- /dev/null +++ b/requirements-dev.txt @@ -0,0 +1,37 @@ +# Development dependencies +# These are only needed for development and testing + +# Testing +pytest>=6.0.0 +pytest-cov>=2.12.0 +pytest-mock>=3.6.0 +pytest-xdist>=2.3.0 # For parallel test execution + +# Code Quality +black>=21.0.0 # Code formatting +isort>=5.9.0 # Import sorting +flake8>=3.9.0 # Linting +pylint>=2.9.0 # Static analysis +mypy>=0.910 # Type checking + +# Pre-commit hooks +pre-commit>=2.13.0 + +# Documentation +sphinx>=4.0.0 # Documentation generation +sphinx-rtd-theme>=0.5.0 + +# Development tools +ipython>=7.25.0 # Enhanced Python shell +jupyter>=1.0.0 # Notebook support +python-dotenv>=0.19.0 # Environment variables + +# Debugging +pdbpp>=0.10.0 # Enhanced debugger +ipdb>=0.13.0 # IPython debugger + +# Database migrations (if using Alembic) +# alembic>=1.6.0 + +# NOTE: This is a template file. +# Customize based on your project's development needs. \ No newline at end of file diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..340e53c --- /dev/null +++ b/requirements.txt @@ -0,0 +1,32 @@ +# Core dependencies - customize based on your project needs +# Remove unused dependencies and add your specific requirements + +# Web Framework Examples (choose one): +# flask>=2.0.0 +# django>=4.0.0 +# fastapi>=0.68.0 + +# Database +# sqlalchemy>=1.4.0 +# psycopg2-binary>=2.8.0 # For PostgreSQL +# pymysql>=1.0.0 # For MySQL + +# HTTP Requests +# requests>=2.25.0 + +# Data Processing +# pandas>=1.3.0 +# numpy>=1.21.0 + +# Configuration +# python-dotenv>=0.19.0 + +# Utilities +# click>=8.0.0 +# pydantic>=1.8.0 + +# Production Server +# gunicorn>=20.0.0 + +# NOTE: This is a template file. +# Replace with your actual project dependencies. \ No newline at end of file diff --git a/scripts/deploy.sh b/scripts/deploy.sh new file mode 100755 index 0000000..f68c5bb --- /dev/null +++ b/scripts/deploy.sh @@ -0,0 +1,231 @@ +#!/bin/bash +# Deployment script template +# Customize this script based on your deployment needs + +set -e # Exit on any error + +# Configuration +PROJECT_NAME="your-project-name" +BRANCH="main" +REMOTE="origin" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +# Functions +log_info() { + echo -e "${GREEN}[INFO]${NC} $1" +} + +log_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +log_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +check_prerequisites() { + log_info "Checking prerequisites..." + + # Check if git is available + if ! command -v git &> /dev/null; then + log_error "Git is required but not installed." + exit 1 + fi + + # Check if we're in a git repository + if ! git rev-parse --git-dir > /dev/null 2>&1; then + log_error "Not in a git repository." + exit 1 + fi + + # Check for clean working directory + if ! git diff-index --quiet HEAD --; then + log_error "Working directory is not clean. Please commit or stash changes." + exit 1 + fi + + log_info "Prerequisites check passed." +} + +run_tests() { + log_info "Running tests..." + + # Python tests + if [ -f "requirements.txt" ]; then + if command -v pytest &> /dev/null; then + pytest + else + log_warn "pytest not found, skipping tests" + fi + fi + + # Node.js tests + if [ -f "package.json" ]; then + if command -v npm &> /dev/null; then + npm test + else + log_warn "npm not found, skipping tests" + fi + fi + + log_info "Tests completed." +} + +build_application() { + log_info "Building application..." + + # Python build + if [ -f "pyproject.toml" ]; then + python -m build + fi + + # Node.js build + if [ -f "package.json" ]; then + npm run build + fi + + # Docker build + if [ -f "Dockerfile" ]; then + log_info "Building Docker image..." + docker build -t "$PROJECT_NAME:latest" . + fi + + log_info "Build completed." +} + +deploy_to_staging() { + log_info "Deploying to staging..." + + # Add your staging deployment logic here + # Examples: + # - Deploy to staging server + # - Update staging database + # - Run staging tests + + # Docker deployment example + if [ -f "docker-compose.staging.yml" ]; then + docker-compose -f docker-compose.staging.yml up -d + fi + + # Heroku deployment example + # if command -v heroku &> /dev/null; then + # heroku git:remote -a your-staging-app + # git push heroku $BRANCH:main + # fi + + log_info "Staging deployment completed." +} + +deploy_to_production() { + log_info "Deploying to production..." + + # Add your production deployment logic here + # Examples: + # - Deploy to production server + # - Update production database + # - Run smoke tests + + # Docker deployment example + if [ -f "docker-compose.prod.yml" ]; then + docker-compose -f docker-compose.prod.yml up -d + fi + + # AWS deployment example + # if command -v aws &> /dev/null; then + # aws ecs update-service --cluster your-cluster --service your-service --force-new-deployment + # fi + + log_info "Production deployment completed." +} + +rollback() { + log_warn "Rolling back deployment..." + + # Add rollback logic here + # Examples: + # - Revert to previous Docker image + # - Restore database backup + # - Switch traffic back to previous version + + log_info "Rollback completed." +} + +cleanup() { + log_info "Cleaning up..." + + # Clean up build artifacts + # Remove temporary files + # Prune Docker images if needed + + log_info "Cleanup completed." +} + +# Main deployment function +main() { + local environment=${1:-staging} + + log_info "Starting deployment to $environment..." + + # Run checks + check_prerequisites + + # Pull latest changes + log_info "Pulling latest changes from $REMOTE/$BRANCH..." + git pull $REMOTE $BRANCH + + # Run tests + run_tests + + # Build application + build_application + + # Deploy based on environment + case $environment in + staging) + deploy_to_staging + ;; + production) + deploy_to_production + ;; + *) + log_error "Unknown environment: $environment" + log_info "Usage: $0 [staging|production]" + exit 1 + ;; + esac + + # Cleanup + cleanup + + log_info "Deployment to $environment completed successfully! ๐Ÿš€" +} + +# Script options +case "${1:-}" in + -h|--help) + echo "Usage: $0 [staging|production]" + echo "" + echo "Options:" + echo " staging Deploy to staging environment (default)" + echo " production Deploy to production environment" + echo " -h, --help Show this help message" + echo "" + echo "Examples:" + echo " $0 # Deploy to staging" + echo " $0 staging # Deploy to staging" + echo " $0 production # Deploy to production" + exit 0 + ;; + rollback) + rollback + exit 0 + ;; + *) + main "$@" + ;; +esac \ No newline at end of file diff --git a/scripts/setup.sh b/scripts/setup.sh new file mode 100755 index 0000000..bf17964 --- /dev/null +++ b/scripts/setup.sh @@ -0,0 +1,74 @@ +#!/bin/bash +# Setup script for the project + +set -e # Exit on any error + +echo "๐Ÿš€ Setting up project..." + +# Check if Python is installed +if ! command -v python3 &> /dev/null; then + echo "โŒ Python 3 is required but not installed." + exit 1 +fi + +# Check if pip is installed +if ! command -v pip3 &> /dev/null; then + echo "โŒ pip3 is required but not installed." + exit 1 +fi + +# Create virtual environment if it doesn't exist +if [ ! -d "venv" ]; then + echo "๐Ÿ“ฆ Creating virtual environment..." + python3 -m venv venv +fi + +# Activate virtual environment +echo "๐Ÿ”ง Activating virtual environment..." +source venv/bin/activate + +# Upgrade pip +echo "โฌ†๏ธ Upgrading pip..." +pip install --upgrade pip + +# Install requirements if requirements.txt exists +if [ -f "requirements.txt" ]; then + echo "๐Ÿ“‹ Installing requirements..." + pip install -r requirements.txt +fi + +# Install development requirements if they exist +if [ -f "requirements-dev.txt" ]; then + echo "๐Ÿ› ๏ธ Installing development requirements..." + pip install -r requirements-dev.txt +fi + +# Set up pre-commit hooks if .pre-commit-config.yaml exists +if [ -f ".pre-commit-config.yaml" ]; then + echo "๐ŸŽฃ Setting up pre-commit hooks..." + pip install pre-commit + pre-commit install +fi + +# Create .env file from template if it doesn't exist +if [ -f ".env.example" ] && [ ! -f ".env" ]; then + echo "โš™๏ธ Creating .env file from template..." + cp .env.example .env + echo "๐Ÿ“ Please edit .env file with your specific configuration" +fi + +# Run initial tests if they exist +if [ -d "tests" ] && command -v pytest &> /dev/null; then + echo "๐Ÿงช Running initial tests..." + pytest --version + # Uncomment the next line to run tests during setup + # pytest +fi + +echo "โœ… Setup completed successfully!" +echo "" +echo "Next steps:" +echo "1. Activate the virtual environment: source venv/bin/activate" +echo "2. Edit .env file if created" +echo "3. Review README.md for project-specific instructions" +echo "4. Start coding! ๐ŸŽ‰" \ No newline at end of file diff --git a/src/README.md b/src/README.md new file mode 100644 index 0000000..c01bd31 --- /dev/null +++ b/src/README.md @@ -0,0 +1,43 @@ +# Source Code Directory + +This directory contains the main source code for the project. + +## Structure + +Organize your source code according to your project's needs. Common patterns include: + +### Python Project Example +``` +src/ +โ”œโ”€โ”€ main/ # Main application module +โ”‚ โ”œโ”€โ”€ __init__.py +โ”‚ โ”œโ”€โ”€ app.py # Application entry point +โ”‚ โ”œโ”€โ”€ models.py # Data models +โ”‚ โ”œโ”€โ”€ views.py # View controllers +โ”‚ โ””โ”€โ”€ utils.py # Utility functions +โ”œโ”€โ”€ config/ # Configuration modules +โ”‚ โ”œโ”€โ”€ __init__.py +โ”‚ โ”œโ”€โ”€ settings.py # Application settings +โ”‚ โ””โ”€โ”€ database.py # Database configuration +โ””โ”€โ”€ __init__.py +``` + +### JavaScript/Node.js Project Example +``` +src/ +โ”œโ”€โ”€ components/ # React components +โ”œโ”€โ”€ services/ # API services +โ”œโ”€โ”€ utils/ # Utility functions +โ”œโ”€โ”€ styles/ # CSS/SCSS files +โ”œโ”€โ”€ config/ # Configuration files +โ”œโ”€โ”€ app.js # Main application file +โ””โ”€โ”€ index.js # Entry point +``` + +## Guidelines + +- Keep modules focused and cohesive +- Use descriptive names for files and directories +- Separate concerns (models, views, controllers, utilities) +- Follow your language's conventions +- Document complex modules and functions \ No newline at end of file diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..664b530 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,59 @@ +# Tests Directory + +This directory contains all test files for the project. + +## Structure + +``` +tests/ +โ”œโ”€โ”€ unit/ # Unit tests - test individual components +โ”œโ”€โ”€ integration/ # Integration tests - test component interactions +โ”œโ”€โ”€ e2e/ # End-to-end tests - test complete workflows +โ”œโ”€โ”€ fixtures/ # Test data and fixtures +โ”œโ”€โ”€ conftest.py # Test configuration (pytest) +โ””โ”€โ”€ README.md # This file +``` + +## Test Types + +### Unit Tests +- Fast execution +- Test individual functions/methods +- Use mocks for external dependencies +- High code coverage + +### Integration Tests +- Test component interactions +- Use test databases/services +- Verify data flow between modules + +### End-to-End Tests +- Test complete user workflows +- Use real or staging environments +- Validate business requirements + +## Running Tests + +```bash +# Run all tests +pytest + +# Run specific test type +pytest tests/unit/ +pytest tests/integration/ + +# Run with coverage +pytest --cov=src + +# Run specific test file +pytest tests/unit/test_example.py +``` + +## Guidelines + +- Write tests before or alongside code (TDD/BDD) +- Use descriptive test names +- Keep tests independent and isolated +- Mock external dependencies in unit tests +- Maintain good test coverage (aim for >80%) +- Update tests when changing functionality \ No newline at end of file