Added admin interface to manage channels and enabled plugins (#9)

* Added base admin login/logout flows

* Ignore local database

* Channel model

* Admin interface for channels and plugins

* Added database tests along with workflows

* Added some docstrings

* Ignore .coverage file

* Creating plugins docs WIP

* Documentation

* Black everything

* Some documentation

* Coverage for the plugins package as well

* DB Fixes

* Absolute FROM in Dockerfile

* Database and logging fixes

* Slack: Support private channels

* Added pre-commit

* black'd

* Fixed UserQuery.create

* Fixed ChannelPluginQuery.create exists call

* Added ChannelPlugin menu for debugging

* Ignore sqlite databases

* Updated contributing docs

.env-example

@@ -1,4 +1,4 @@
-# For information about this variables check config.py
+# For information about this variables check butterrobot/config.py
 
 SLACK_TOKEN=xxx
 TELEGRAM_TOKEN=xxx

.github/workflows/black.yaml

@@ -0,0 +1,27 @@
+name: Black
+
+on:
+  push:
+    branches: [ master, stable ]
+  pull_request:
+    branches: [ master, stable ]
+
+jobs:
+  black:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.8
+
+    - name: Install dependencies
+      run: |
+        pip install --upgrade pip
+        pip install black
+
+    - name: Black check
+      run: |
+        black --check butterrobot

.github/workflows/pytest.yaml

@@ -0,0 +1,32 @@
+name: Pytest
+
+on:
+  push:
+    branches: [ master, stable ]
+  pull_request:
+    branches: [ master, stable ]
+
+jobs:
+  pytest:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies
+      run: |
+        pip install --upgrade pip poetry
+        poetry install
+
+    - name: Test with pytest
+      run: |
+        ls
+        poetry run pytest --cov=butterrobot --cov=butterrobot_plugins_contrib

.gitignore

@@ -5,6 +5,7 @@ __pycache__
 *.cert
 .env-local
 test.py
+.coverage
 
 # Distribution
 dist
@@ -12,4 +13,7 @@ dist
 pip-wheel-metadata
 
 # Github Codespaces
-pythonenv3.8
+pythonenv3.8
+
+# Butterrobot
+*.sqlite*

.pre-commit-config.yaml

@@ -0,0 +1,22 @@
+repos:
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v2.2.3
+  hooks:
+  - id: trailing-whitespace
+  - id: end-of-file-fixer
+  - id: flake8
+
+- repo: https://github.com/asottile/seed-isort-config
+  rev: v1.9.2
+  hooks:
+  - id: seed-isort-config
+- repo: https://github.com/pre-commit/mirrors-isort
+  rev: v4.3.20
+  hooks:
+  - id: isort
+
+- repo: https://github.com/ambv/black
+  rev: stable
+  hooks:
+  - id: black
+    language_version: python3

Dockerfile.dev

@@ -1,20 +1,26 @@
-FROM alpine:3.11
+FROM docker.io/library/alpine:3.11
 
 ENV PYTHON_VERSION=3.8.2-r1
 ENV APP_PORT 8080
 ENV BUILD_DIR /tmp/build
+ENV APP_PATH /etc/butterrobot
 
 WORKDIR ${BUILD_DIR}
 COPY README.md ${BUILD_DIR}/README.md
 COPY poetry.lock ${BUILD_DIR}/poetry.lock
 COPY pyproject.toml ${BUILD_DIR}/pyproject.toml
-COPY ./butterrobot ${BUILD_DIR}/butterrobot
 COPY ./butterrobot_plugins_contrib ${BUILD_DIR}/butterrobot_plugins_contrib
+COPY ./butterrobot ${BUILD_DIR}/butterrobot
 RUN apk --update add curl python3-dev==${PYTHON_VERSION} gcc musl-dev libffi-dev openssl-dev && \
     pip3 install poetry && \
     poetry build && \
     pip3 install ${BUILD_DIR}/dist/butterrobot-*.tar.gz && \
-    rm -rf ${BUILD_DIR}
+    rm -rf ${BUILD_DIR} && \
+    mkdir ${APP_PATH} && \
+    chown -R 1000:1000 ${APP_PATH}
+
+USER 1000
+WORKDIR ${APP_PATH}
 COPY ./docker/bin/start-server.sh /usr/local/bin/start-server
 
 CMD ["/usr/local/bin/start-server"]

Makefile

@@ -19,6 +19,9 @@ podman@dev:
 	make podman@tag-dev
 	make podman@push-dev
 
+test:
+	poetry run pytest --cov=butterrobot --cov=butterrobot_plugins_contrib
+
 clean:
 	rm -rf dist
 	rm -rf butterrobot.egg-info

README.md

@@ -1,7 +1,9 @@
 # Butter Robot
 
-![Build stable tag docker image](https://github.com/fmartingr/butterrobot/workflows/Build%20stable%20tag%20docker%20image/badge.svg?branch=stable)
+| Stable | Master |
-![Build latest tag docker image](https://github.com/fmartingr/butterrobot/workflows/Build%20latest%20tag%20docker%20image/badge.svg?branch=master)
+| --- | --- |
+| ![Build stable tag docker image](https://github.com/fmartingr/butterrobot/workflows/Build%20stable%20tag%20docker%20image/badge.svg?branch=stable) | ![Build latest tag docker image](https://github.com/fmartingr/butterrobot/workflows/Build%20latest%20tag%20docker%20image/badge.svg?branch=master) |
+| ![Pytest](https://github.com/fmartingr/butterrobot/workflows/Pytest/badge.svg?branch=stable) | ![Pytest](https://github.com/fmartingr/butterrobot/workflows/Pytest/badge.svg?branch=master) |
 
 Python framework to create bots for several platforms.
 
@@ -9,25 +11,9 @@ Python framework to create bots for several platforms.
 
 > What is my purpose?
 
-## Supported platforms
+## Documentation
 
-| Name            | Receive messages | Send messages |
+[Go to documentation](./docs)
-| --------------- | ---------------- | ------------- |
-| Slack (app)     | Yes              | Yes           |
-| Telegram        | Yes              | Yes           |
-
-## Provided plugins
-
-
-### Development
-
-- `!ping`: Say `!ping` to get response with time elapsed.
-
-### Fun and entertainment
-
-
-- Lo quito: What happens when you say _"lo quito"_...? (Spanish pun)
-- Dice: Put `!dice` and wathever roll you want to perform.
 
 ## Installation
 
@@ -44,12 +30,11 @@ $ python -m butterrobot
 
 ### Containers
 
-The `fmartingr/butterrobot/butterrobot` container image is published on Github packages to
+The `fmartingr/butterrobot/butterrobot` container image is published on Github packages to use with your favourite tool:
-use with your favourite tool:
 
 ```
 docker pull docker.pkg.github.com/fmartingr/butterrobot/butterrobot:latest
-podman run -d --name fmartingr/butterrobot/butterrobot -p 8080:8080 
+podman run -d --name fmartingr/butterrobot/butterrobot -p 8080:8080
 ```
 
 ## Contributing
@@ -62,8 +47,7 @@ cd butterrobot
 poetry install
 ```
 
-Create a `.env-local` file with the required environment variables,
+Create a `.env-local` file with the required environment variables, you have [an example file](.env-example).
-you have [an example file](.env-example).
 
 ```
 SLACK_TOKEN=xxx

butterrobot/admin/blueprint.py

@@ -0,0 +1,156 @@
+import os.path
+from functools import wraps
+
+import structlog
+from flask import (
+    Blueprint,
+    g,
+    flash,
+    request,
+    session,
+    url_for,
+    redirect,
+    render_template,
+)
+
+from butterrobot.db import UserQuery, ChannelQuery, ChannelPluginQuery
+from butterrobot.plugins import get_available_plugins
+
+admin = Blueprint("admin", __name__, url_prefix="/admin")
+admin.template_folder = os.path.join(os.path.dirname(__name__), "templates")
+logger = structlog.get_logger(__name__)
+
+
+def login_required(f):
+    @wraps(f)
+    def decorated_function(*args, **kwargs):
+        if g.user is None:
+            return redirect(url_for("admin.login_view", next=request.path))
+        return f(*args, **kwargs)
+
+    return decorated_function
+
+
+@admin.before_app_request
+def load_logged_in_user():
+    user_id = session.get("user_id")
+
+    if user_id is None:
+        g.user = None
+    else:
+        try:
+            user = UserQuery.get(id=user_id)
+            g.user = user
+        except UserQuery.NotFound:
+            g.user = None
+
+
+@admin.route("/")
+@login_required
+def index_view():
+    if not session.get("logged_in", False):
+        return redirect(url_for("admin.login_view"))
+    return redirect(url_for("admin.channel_list_view"))
+
+
+@admin.route("/login", methods=["GET", "POST"])
+def login_view():
+    error = None
+    if request.method == "POST":
+        user = UserQuery.check_credentials(
+            request.form["username"], request.form["password"]
+        )
+        if not user:
+            flash("Incorrect credentials", category="danger")
+        else:
+            session["logged_in"] = True
+            session["user_id"] = user.id
+            flash("You were logged in", category="success")
+            _next = request.args.get("next", url_for("admin.index_view"))
+            return redirect(_next)
+    return render_template("login.j2", error=error)
+
+
+@admin.route("/logout")
+@login_required
+def logout_view():
+    session.clear()
+    flash("You were logged out", category="success")
+    return redirect(url_for("admin.index_view"))
+
+
+@admin.route("/plugins")
+@login_required
+def plugin_list_view():
+    return render_template("plugin_list.j2", plugins=get_available_plugins().values())
+
+
+@admin.route("/channels")
+@login_required
+def channel_list_view():
+    return render_template("channel_list.j2", channels=ChannelQuery.all())
+
+
+@admin.route("/channels/<channel_id>", methods=["GET", "POST"])
+@login_required
+def channel_detail_view(channel_id):
+    if request.method == "POST":
+        ChannelQuery.update(
+            channel_id,
+            enabled=request.form["enabled"] == "true",
+        )
+        flash("Channel updated", "success")
+
+    channel = ChannelQuery.get(channel_id)
+    return render_template(
+        "channel_detail.j2", channel=channel, plugins=get_available_plugins()
+    )
+
+
+@admin.route("/channel/<channel_id>/delete", methods=["POST"])
+@login_required
+def channel_delete_view(channel_id):
+    ChannelQuery.delete(channel_id)
+    flash("Channel removed", category="success")
+    return redirect(url_for("admin.channel_list_view"))
+
+
+@admin.route("/channelplugins", methods=["GET", "POST"])
+@login_required
+def channel_plugin_list_view():
+    if request.method == "POST":
+        data = request.form
+        try:
+            ChannelPluginQuery.create(
+                data["channel_id"], data["plugin_id"], enabled=data["enabled"] == "y"
+            )
+            flash(f"Plugin {data['plugin_id']} added to the channel", "success")
+        except ChannelPluginQuery.Duplicated:
+            flash(
+                f"Plugin {data['plugin_id']} is already present on the channel", "error"
+            )
+        return redirect(request.headers.get("Referer"))
+
+    channel_plugins = ChannelPluginQuery.all()
+    return render_template("channel_plugins_list.j2", channel_plugins=channel_plugins)
+
+
+@admin.route("/channelplugins/<channel_plugin_id>", methods=["GET", "POST"])
+@login_required
+def channel_plugin_detail_view(channel_plugin_id):
+    if request.method == "POST":
+        ChannelPluginQuery.update(
+            channel_plugin_id,
+            enabled=request.form["enabled"] == "true",
+        )
+        flash("Plugin updated", category="success")
+
+    return redirect(request.headers.get("Referer"))
+
+
+@admin.route("/channelplugins/<channel_plugin_id>/delete", methods=["POST"])
+@login_required
+def channel_plugin_delete_view(channel_plugin_id):
+    ChannelPluginQuery.delete(channel_plugin_id)
+    flash("Plugin removed", category="success")
+    return redirect(request.headers.get("Referer"))

butterrobot/admin/templates/_base.j2

@@ -0,0 +1,122 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>ButterRobot Admin</title>
+    <link rel="stylesheet" href="https://unpkg.com/@tabler/core@latest/dist/css/tabler.min.css">
+</head>
+
+<body>
+    <div class="page">
+        <div class="sticky-top">
+            <header class="navbar navbar-expand-md navbar-light sticky-top d-print-none">
+                <div class="container-xl">
+                    <button class="navbar-toggler" type="button" data-bs-toggle="collapse"
+                        data-bs-target="#navbar-menu">
+                        <span class="navbar-toggler-icon"></span>
+                    </button>
+                    <h1 class="navbar-brand navbar-brand-autodark d-none-navbar-horizontal pr-0 pr-md-3">
+                        <a href="/admin/">
+                            <h1>ButterRobot Admin</h1>
+                        </a>
+                    </h1>
+                    <div class="navbar-nav flex-row order-md-last">
+                        <div class="nav-item">
+                            {% if not session.logged_in %}
+                            <a href="{{ url_for('admin.login_view') }}">Log in</a>
+                            {% else %}
+                            <div class="d-none d-xl-block pl-2">
+                                <div>{{ g.user.username }} - <a class="mt-1 small"
+                                        href="{{ url_for('admin.logout_view') }}">Log out</a></div>
+                            </div>
+                            </a>
+                            {% endif %}
+                        </div>
+                    </div>
+                </div>
+            </header>
+            {% if session.logged_in %}
+            <div class="navbar-expand-md">
+                <div class="collapse navbar-collapse" id="navbar-menu">
+                    <div class="navbar navbar-light">
+                        <div class="container-xl">
+                            <ul class="navbar-nav">
+                                <li class="nav-item {% if '/channels' in request.url %}active{% endif %}">
+                                    <a class="nav-link" href="{{ url_for('admin.channel_list_view') }}">
+                                        <span class="nav-link-icon d-md-none d-lg-inline-block">
+                                            <svg xmlns="http://www.w3.org/2000/svg" class="icon" width="24" height="24"
+                                                viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none"
+                                                stroke-linecap="round" stroke-linejoin="round">
+                                                <path stroke="none" d="M0 0h24v24H0z" fill="none" />
+                                                <line x1="5" y1="9" x2="19" y2="9" />
+                                                <line x1="5" y1="15" x2="19" y2="15" />
+                                                <line x1="11" y1="4" x2="7" y2="20" />
+                                                <line x1="17" y1="4" x2="13" y2="20" /></svg>
+                                        </span>
+                                        <span class="nav-link-title">
+                                            Channels
+                                        </span>
+                                    </a>
+                                </li>
+                                <li class="nav-item {% if '/plugins' in request.url %}active{% endif %}">
+                                    <a class="nav-link" href="{{ url_for('admin.plugin_list_view') }}">
+                                        <span class="nav-link-icon d-md-none d-lg-inline-block">
+                                            <svg xmlns="http://www.w3.org/2000/svg" class="icon" width="24" height="24"
+                                                viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none"
+                                                stroke-linecap="round" stroke-linejoin="round">
+                                                <path stroke="none" d="M0 0h24v24H0z" fill="none" />
+                                                <path
+                                                    d="M4 7h3a1 1 0 0 0 1 -1v-1a2 2 0 0 1 4 0v1a1 1 0 0 0 1 1h3a1 1 0 0 1 1 1v3a1 1 0 0 0 1 1h1a2 2 0 0 1 0 4h-1a1 1 0 0 0 -1 1v3a1 1 0 0 1 -1 1h-3a1 1 0 0 1 -1 -1v-1a2 2 0 0 0 -4 0v1a1 1 0 0 1 -1 1h-3a1 1 0 0 1 -1 -1v-3a1 1 0 0 1 1 -1h1a2 2 0 0 0 0 -4h-1a1 1 0 0 1 -1 -1v-3a1 1 0 0 1 1 -1" />
+                                            </svg>
+                                        </span>
+                                        <span class="nav-link-title">
+                                            Plugins
+                                        </span>
+                                    </a>
+                                </li>
+                                <li class="nav-item {% if '/channelplugins' in request.url %}active{% endif %}">
+                                   <a class="nav-link" href="{{ url_for('admin.channel_plugin_list_view') }}">
+                                        <span class="nav-link-icon d-md-none d-lg-inline-block">
+                                            <svg xmlns="http://www.w3.org/2000/svg" class="icon" width="24" height="24"
+                                                viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none"
+                                                stroke-linecap="round" stroke-linejoin="round">
+                                                <path stroke="none" d="M0 0h24v24H0z" fill="none" />
+                                                <path
+                                                    d="M4 7h3a1 1 0 0 0 1 -1v-1a2 2 0 0 1 4 0v1a1 1 0 0 0 1 1h3a1 1 0 0 1 1 1v3a1 1 0 0 0 1 1h1a2 2 0 0 1 0 4h-1a1 1 0 0 0 -1 1v3a1 1 0 0 1 -1 1h-3a1 1 0 0 1 -1 -1v-1a2 2 0 0 0 -4 0v1a1 1 0 0 1 -1 1h-3a1 1 0 0 1 -1 -1v-3a1 1 0 0 1 1 -1h1a2 2 0 0 0 0 -4h-1a1 1 0 0 1 -1 -1v-3a1 1 0 0 1 1 -1" />
+                                            </svg>
+                                        </span>
+                                        <span class="nav-link-title">
+                                            Channel Plugins
+                                        </span>
+                                    </a>
+                                </li>
+                            </ul>
+                        </div>
+                    </div>
+                </div>
+            </div>
+            {% endif %}
+        </div>
+
+        {% for category, message in get_flashed_messages(with_categories=True) %}
+        <div class="card">
+            <div class="card-status-top bg-{{ category }}"></div>
+            <div class="card-body">
+                <p>{{ message }}</p>
+            </div>
+        </div>
+        {% endfor %}
+
+        <div class="content">
+            <div class="container-xl">
+                {% block content %}
+                {% endblock %}
+            </div>
+        </div>
+
+    </div>
+</body>
+
+</html>

butterrobot/admin/templates/channel_detail.j2

@@ -0,0 +1,140 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+<div class="page-header d-print-none">
+    <div class="row align-items-center">
+        <div class="col">
+            <h2 class="page-title">
+                Channel: {{ channel.channel_name }}
+            </h2>
+        </div>
+    </div>
+</div>
+<div class="row row-cards">
+    <div class="col-12">
+        <div class="card">
+            <div class="card-header">
+                <ul class="nav nav-pills card-header-pills">
+                    <li class="nav-item">
+                        <form
+                            action="{{ url_for('admin.channel_detail_view', channel_id=channel.id) }}"
+                            method="POST">
+                            <input type="hidden" name="enabled" value="{{ 'false' if channel.enabled else 'true' }}" />
+                            <input class="btn btn-{% if channel.enabled %}danger{% else %}success{% endif %}"
+                                type="submit" value="{{ "Enable" if not channel.enabled else "Disable" }}">
+                        </form>
+                    </li>
+                    <li class="nav-item">
+                        <form action="{{ url_for('admin.channel_delete_view', channel_id=channel.id) }}" method="POST">
+                            <input type="submit" value="Delete" class="btn btn-danger">
+                        </form>
+                    </li>
+                </ul>
+            </div>
+            <div class="card-body">
+                <table class="table table-vcenter card-table">
+                    <tbody>
+                        <tr>
+                            <th width="20%">ID</th>
+                            <td>{{ channel.id }}</td>
+                        </tr>
+                        <tr>
+                            <th>Platform</th>
+                            <td>{{ channel.platform }}</td>
+                        </tr>
+                        <tr>
+                            <th>Platform Channel ID</th>
+                            <td>{{ channel.platform_channel_id }}</td>
+                        </tr>
+                        <tr>
+                            <th>RAW</th>
+                            <td>
+                                <pre>{{ channel.channel_raw }}</pre>
+                            </td>
+                        </tr>
+                    </tbody>
+                </table>
+            </div>
+        </div>
+    </div>
+    <div class="col-12">
+        <div class="card">
+            <div class="card-body">
+                <h3 class="card-title">Plugins</h3>
+            </div>
+            <div class="card-body">
+                <form action="{{ url_for('admin.channel_plugin_list_view') }}" method="POST">
+                    <input type="hidden" name="channel_id" value="{{ channel.id }}" />
+                    <input type="hidden" name="enabled" value="y" />
+                    <p>
+                    <div class="row">
+                        <div class="col-4">
+                            Enable plugin
+                        </div>
+                        <div class="col-4">
+                            <select class="form-select" name="plugin_id">
+                                {% for plugin in plugins.values() %}
+                                <option value="{{ plugin.id }}">{{ plugin.name }}</option>
+                                {% endfor %}
+                            </select>
+                        </div>
+                        <div class="col-4">
+                            <input type="submit" value="Enable" class="btn">
+                        </div>
+                    </div>
+                    </p>
+                </form>
+                <div>
+                    <table class="table table-vcenter card-table">
+                        <thead>
+                            <tr>
+                                <th>Name</th>
+                                <th>Configuration</th>
+                                <th>Actions</th>
+                            </tr>
+                        </thead>
+
+                        <tbody>
+                            {% for channel_plugin in channel.plugins.values() %}
+                            <tr>
+                                <td width="20%">{{ plugins[channel_plugin.plugin_id].name }}</td>
+                                <td>
+                                    <pre>{{ channel_plugin.config }}</pre>
+                                </td>
+                                <td width="20%">
+                                    <div class="row">
+                                        <div class="col-6">
+                                            <form
+                                                action="{{ url_for('admin.channel_plugin_detail_view', channel_plugin_id=channel_plugin.id) }}"
+                                                method="POST">
+                                                <input type="hidden" name="enabled"
+                                                    value="{{ 'false' if channel_plugin.enabled else 'true' }}" />
+                                                <input
+                                                    class="btn btn-{% if channel_plugin.enabled %}danger{% else %}success{% endif %}"
+                                                    type="submit"
+                                                    value="{{ "Enable" if not channel_plugin.enabled else "Disable" }}">
+                                            </form>
+                                        </div>
+                                        <div class="col-6">
+                                            <form
+                                                action="{{ url_for('admin.channel_plugin_delete_view', channel_plugin_id=channel_plugin.id) }}"
+                                                method="POST">
+                                                <input type="submit" value="Delete" class="btn btn-danger">
+                                            </form>
+                                        </div>
+                                    </div>
+                                </td>
+                            </tr>
+                            {% else %}
+                            <tr>
+                                <td colspan="3" class="text-center">No plugin is enabled on this channel</td>
+                            </tr>
+                            {% endfor %}
+                        </tbody>
+                    </table>
+                </div>
+            </div>
+        </div>
+    </div>
+</div>
+{% endblock %}

butterrobot/admin/templates/channel_list.j2

@@ -0,0 +1,45 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+<div class="page-header d-print-none">
+    <div class="row align-items-center">
+        <div class="col">
+            <h2 class="page-title">
+                Channel list
+            </h2>
+        </div>
+    </div>
+</div>
+
+<div class="row">
+    <div class="table-responsive">
+        <table class="table table-vcenter card-table">
+            <thead>
+                <tr>
+                    <th>Platform</th>
+                    <th>Channel name</th>
+                    <th>Channel ID</th>
+                    <th>Enabled</th>
+                    <th class="w-1"></th>
+                </tr>
+            </thead>
+
+            <tbody>
+                {% for channel in channels %}
+                <tr>
+                    <td>{{ channel.platform }}</td>
+                    <td>{{ channel.channel_name }}</td>
+                    <td class="text-muted">
+                        {{ channel.platform_channel_id }}
+                    </td>
+                    <td class="text-muted">{{ channel.enabled }}</td>
+                    <td>
+                        <a href="{{ url_for("admin.channel_detail_view", channel_id=channel.id) }}">Edit</a>
+                    </td>
+                </tr>
+                {% endfor %}
+            </tbody>
+        </table>
+    </div>
+</div>
+{% endblock %}

butterrobot/admin/templates/channel_plugins_list.j2

@@ -0,0 +1,41 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+<div class="page-header d-print-none">
+    <div class="row align-items-center">
+        <div class="col">
+            <h2 class="page-title">
+                Channel list
+            </h2>
+        </div>
+    </div>
+</div>
+
+<div class="row">
+    <div class="table-responsive">
+        <table class="table table-vcenter card-table">
+            <thead>
+                <tr>
+                    <th>ID</th>
+                    <th>Channel ID</th>
+                    <th>Plugin ID</th>
+                    <th>Enabled</th>
+                </tr>
+            </thead>
+
+            <tbody>
+                {% for channel_plugin in channel_plugins %}
+                <tr>
+                    <td>{{ channel_plugin.id }}</td>
+                    <td>{{ channel_plugin.channel_id }}</td>
+                    <td class="text-muted">
+                        {{ channel_plugin.plugin_id }}
+                    </td>
+                    <td class="text-muted">{{ channel_plugin.enabled }}</td>
+                </tr>
+                {% endfor %}
+            </tbody>
+        </table>
+    </div>
+</div>
+{% endblock %}

butterrobot/admin/templates/index.j2

@@ -0,0 +1,5 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+
+{% endblock %}

butterrobot/admin/templates/login.j2

@@ -0,0 +1,32 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+<div class="row">
+
+    {% if error %}<p class=error><strong>Error:</strong> {{ error }}{% endif %}
+    <div class="card">
+        <div class="card-header">
+            <h3 class="card-title">Login</h3>
+        </div>
+        <div class="card-body">
+            <form action="" method="post">
+                <div class="form-group mb-3 ">
+                    <label class="form-label">Username</label>
+                    <div>
+                        <input type="text" name="username" class="form-control" placeholder="Username">
+                    </div>
+                </div>
+                <div class="form-group mb-3 ">
+                    <label class="form-label">Password</label>
+                    <div>
+                        <input type="password" class="form-control" placeholder="Password" name="password">
+                    </div>
+                </div>
+                <div class="form-footer">
+                    <button type="submit" class="btn btn-primary">Submit</button>
+                </div>
+            </form>
+        </div>
+    </div>
+</div>
+{% endblock %}

butterrobot/admin/templates/plugin_list.j2

@@ -0,0 +1,33 @@
+{% extends "_base.j2" %}
+
+{% block content %}
+<div class="page-header d-print-none">
+    <div class="row align-items-center">
+        <div class="col">
+            <h2 class="page-title">
+                Plugin list
+            </h2>
+        </div>
+    </div>
+</div>
+
+<div class="row">
+    <div class="table-responsive">
+        <table class="table table-vcenter card-table">
+            <thead>
+                <tr>
+                    <th>Name</th>
+                </tr>
+            </thead>
+
+            <tbody>
+                {% for plugin in plugins %}
+                <tr>
+                    <td>{{ plugin.name }}</td>
+                </tr>
+                {% endfor %}
+            </tbody>
+        </table>
+    </div>
+</div>
+{% endblock %}

butterrobot/app.py

@@ -1,69 +1,35 @@
-import traceback
+import asyncio
 
-from flask import Flask, request
 import structlog
+from flask import Flask, request
 
 import butterrobot.logging  # noqa
-from butterrobot.config import ENABLED_PLUGINS
+from butterrobot.http import ExternalProxyFix
-from butterrobot.objects import Message
+from butterrobot.queue import q
-from butterrobot.plugins import get_available_plugins
+from butterrobot.config import SECRET_KEY
-from butterrobot.platforms import PLATFORMS
+from butterrobot.platforms import get_available_platforms
-from butterrobot.platforms.base import Platform
+from butterrobot.admin.blueprint import admin as admin_bp
-
 
+loop = asyncio.get_event_loop()
 logger = structlog.get_logger(__name__)
 app = Flask(__name__)
-available_platforms = {}
+app.config.update(SECRET_KEY=SECRET_KEY)
-plugins = get_available_plugins()
+app.register_blueprint(admin_bp)
-enabled_plugins = [
+app.wsgi_app = ExternalProxyFix(app.wsgi_app)
-    plugin for plugin_name, plugin in plugins.items() if plugin_name in ENABLED_PLUGINS
-]
-
-
-def handle_message(platform: str, message: Message):
-    for plugin in enabled_plugins:
-        for response_message in plugin.on_message(message):
-            available_platforms[platform].methods.send_message(response_message)
-
-
-@app.before_first_request
-def init_platforms():
-    for platform in PLATFORMS.values():
-        logger.debug("Setting up", platform=platform.ID)
-        try:
-            platform.init(app=app)
-            available_platforms[platform.ID] = platform
-            logger.info("platform setup completed", platform=platform.ID)
-        except platform.PlatformInitError as error:
-            logger.error("Platform init error", error=error, platform=platform.ID)
 
 
 @app.route("/<platform>/incoming", methods=["POST"])
 @app.route("/<platform>/incoming/<path:path>", methods=["POST"])
 def incoming_platform_message_view(platform, path=None):
-    if platform not in available_platforms:
+    if platform not in get_available_platforms():
         return {"error": "Unknown platform"}, 400
 
-    try:
+    q.put(
-        message = available_platforms[platform].parse_incoming_message(
+        {
-            request=request
+            "platform": platform,
-        )
+            "request": {"path": request.path, "json": request.get_json()},
-    except Platform.PlatformAuthResponse as response:
+        }
-        return response.data, response.status_code
+    )
-    except Exception as error:
-        logger.error(
-            "Error parsing message",
-            platform=platform,
-            error=error,
-            traceback=traceback.format_exc(),
-        )
-        return {"error": str(error)}, 400
-
-    if not message or message.from_bot:
-        return {}
-
-    # TODO: make with rq/dramatiq
-    handle_message(platform, message)
 
     return {}
 

butterrobot/config.py

@@ -3,12 +3,16 @@ import os
 # --- Butter Robot -----------------------------------------------------------------
 DEBUG = os.environ.get("DEBUG", "n") == "y"
 
-HOSTNAME = os.environ.get("BUTTERROBOT_HOSTNAME", "butterrobot-dev.int.fmartingr.network")
+HOSTNAME = os.environ.get(
+    "BUTTERROBOT_HOSTNAME", "butterrobot-dev.int.fmartingr.network"
+)
 
 LOG_LEVEL = os.environ.get("LOG_LEVEL", "ERROR")
 
-ENABLED_PLUGINS = os.environ.get("ENABLED_PLUGINS", "contrib/dev/ping").split(",")
+SECRET_KEY = os.environ.get("SECRET_KEY", "1234")
 
+# --- DATABASE ---------------------------------------------------------------------
+DATABASE_PATH = os.environ.get("DATABASE_PATH", "sqlite:///butterrobot.sqlite")
 
 # --- PLATFORMS ---------------------------------------------------------------------
 # ---

butterrobot/db.py

@@ -0,0 +1,163 @@
+import hashlib
+from typing import Union
+
+import dataset
+
+from butterrobot.config import SECRET_KEY, DATABASE_PATH
+from butterrobot.objects import User, Channel, ChannelPlugin
+
+db = dataset.connect(DATABASE_PATH)
+
+
+class Query:
+    class NotFound(Exception):
+        pass
+
+    class Duplicated(Exception):
+        pass
+
+    @classmethod
+    def all(cls):
+        """
+        Iterate over all rows on a table.
+        """
+        for row in db[cls.tablename].all():
+            yield cls.obj(**row)
+
+    @classmethod
+    def get(cls, **kwargs):
+        """
+        Returns the object representation of an specific row in a table.
+        Allows retrieving object by multiple columns.
+        Raises `NotFound` error if query return no results.
+        """
+        row = db[cls.tablename].find_one(**kwargs)
+        if not row:
+            raise cls.NotFound
+        return cls.obj(**row)
+
+    @classmethod
+    def create(cls, **kwargs):
+        """
+        Creates a new row in the table with the provided arguments.
+        Returns the row_id
+        TODO: Return obj?
+        """
+        return db[cls.tablename].insert(kwargs)
+
+    @classmethod
+    def exists(cls, **kwargs) -> bool:
+        """
+        Check for the existence of a row with the provided columns.
+        """
+        try:
+            cls.get(**kwargs)
+        except cls.NotFound:
+            return False
+        return True
+
+    @classmethod
+    def update(cls, row_id, **fields):
+        fields.update({"id": row_id})
+        return db[cls.tablename].update(fields, ("id",))
+
+    @classmethod
+    def delete(cls, id):
+        return db[cls.tablename].delete(id=id)
+
+
+class UserQuery(Query):
+    tablename = "users"
+    obj = User
+
+    @classmethod
+    def _hash_password(cls, password):
+        return hashlib.pbkdf2_hmac(
+            "sha256", password.encode("utf-8"), str.encode(SECRET_KEY), 100000
+        ).hex()
+
+    @classmethod
+    def check_credentials(cls, username, password) -> Union[User, "False"]:
+        user = db[cls.tablename].find_one(username=username)
+        if user:
+            hash_password = cls._hash_password(password)
+            if user["password"] == hash_password:
+                return cls.obj(**user)
+        return False
+
+    @classmethod
+    def create(cls, **kwargs):
+        kwargs["password"] = cls._hash_password(kwargs["password"])
+        return super().create(**kwargs)
+
+
+class ChannelQuery(Query):
+    tablename = "channels"
+    obj = Channel
+
+    @classmethod
+    def create(cls, platform, platform_channel_id, enabled=False, channel_raw={}):
+        params = {
+            "platform": platform,
+            "platform_channel_id": platform_channel_id,
+            "enabled": enabled,
+            "channel_raw": channel_raw,
+        }
+        super().create(**params)
+        return cls.obj(**params)
+
+    @classmethod
+    def get(cls, _id):
+        channel = super().get(id=_id)
+        plugins = ChannelPluginQuery.get_from_channel_id(_id)
+        channel.plugins = {plugin.plugin_id: plugin for plugin in plugins}
+        return channel
+
+    @classmethod
+    def get_by_platform(cls, platform, platform_channel_id):
+        result = db[cls.tablename].find_one(
+            platform=platform, platform_channel_id=platform_channel_id
+        )
+        if not result:
+            raise cls.NotFound
+
+        plugins = ChannelPluginQuery.get_from_channel_id(result["id"])
+
+        return cls.obj(
+            plugins={plugin.plugin_id: plugin for plugin in plugins}, **result
+        )
+
+    @classmethod
+    def delete(cls, _id):
+        ChannelPluginQuery.delete_by_channel(channel_id=_id)
+        super().delete(_id)
+
+
+class ChannelPluginQuery(Query):
+    tablename = "channel_plugin"
+    obj = ChannelPlugin
+
+    @classmethod
+    def create(cls, channel_id, plugin_id, enabled=False, config={}):
+        if cls.exists(channel_id=channel_id, plugin_id=plugin_id):
+            raise cls.Duplicated
+
+        params = {
+            "channel_id": channel_id,
+            "plugin_id": plugin_id,
+            "enabled": enabled,
+            "config": config,
+        }
+        obj_id = super().create(**params)
+        return cls.obj(id=obj_id, **params)
+
+    @classmethod
+    def get_from_channel_id(cls, channel_id):
+        yield from [
+            cls.obj(**row) for row in db[cls.tablename].find(channel_id=channel_id)
+        ]
+
+    @classmethod
+    def delete_by_channel(cls, channel_id):
+        channel_plugins = cls.get_from_channel_id(channel_id)
+        [cls.delete(item.id) for item in channel_plugins]

butterrobot/http.py

@@ -0,0 +1,15 @@
+class ExternalProxyFix(object):
+    """
+    Custom proxy helper to get the external hostname from the `X-External-Host` header
+    used by one of the reverse proxies in front of this in production.
+    It does nothing if the header is not present.
+    """
+
+    def __init__(self, app):
+        self.app = app
+
+    def __call__(self, environ, start_response):
+        host = environ.get("HTTP_X_EXTERNAL_HOST", "")
+        if host:
+            environ["HTTP_HOST"] = host
+        return self.app(environ, start_response)

butterrobot/lib/slack.py

@@ -11,6 +11,7 @@ logger = structlog.get_logger()
 
 class SlackAPI:
     BASE_URL = "https://slack.com/api"
+    HEADERS = {"Authorization": f"Bearer {SLACK_BOT_OAUTH_ACCESS_TOKEN}"}
 
     class SlackError(Exception):
         pass
@@ -18,6 +19,30 @@ class SlackAPI:
     class SlackClientError(Exception):
         pass
 
+    @classmethod
+    def get_conversations_info(cls, chat_id) -> dict:
+        params = {"channel": chat_id}
+        response = requests.get(
+            f"{cls.BASE_URL}/conversations.info", params=params, headers=cls.HEADERS,
+        )
+        response_json = response.json()
+        if not response_json["ok"]:
+            raise cls.SlackClientError(response_json)
+
+        return response_json["channel"]
+
+    @classmethod
+    def get_user_info(cls, chat_id) -> dict:
+        params = {"user": chat_id}
+        response = requests.get(
+            f"{cls.BASE_URL}/users.info", params=params, headers=cls.HEADERS,
+        )
+        response_json = response.json()
+        if not response_json["ok"]:
+            raise cls.SlackClientError(response_json)
+
+        return response_json["user"]
+
     @classmethod
     def send_message(cls, channel, message, thread: Optional[Text] = None):
         payload = {
@@ -28,11 +53,9 @@ class SlackAPI:
         if thread:
             payload["thread_ts"] = thread
 
-        response = requestts.post(
+        response = requests.post(
-                f"{cls.BASE_URL}/chat.postMessage",
+            f"{cls.BASE_URL}/chat.postMessage", data=payload, headers=cls.HEADERS,
-                data=payload,
+        )
-                headers={"Authorization": f"Bearer {SLACK_BOT_OAUTH_ACCESS_TOKEN}"},
-            )
         response_json = response.json()
         if not response_json["ok"]:
             raise cls.SlackClientError(response_json)

butterrobot/lib/telegram.py

@@ -51,8 +51,8 @@ class TelegramAPI:
             "disable_notification": disable_notification,
             "reply_to_message_id": reply_to_message_id,
         }
-        
+
         response = requests.post(url, json=payload)
         response_json = response.json()
         if not response_json["ok"]:
-            raise cls.TelegramClientError(response_json)
+            raise cls.TelegramClientError(response_json)

butterrobot/logging.py

@@ -14,7 +14,9 @@ structlog.configure(
         structlog.processors.StackInfoRenderer(),
         structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M.%S"),
         structlog.processors.format_exc_info,
-        structlog.dev.ConsoleRenderer() if DEBUG else structlog.processors.JSONRenderer(),
+        structlog.dev.ConsoleRenderer()
+        if DEBUG
+        else structlog.processors.JSONRenderer(),
     ],
     context_class=dict,
     logger_factory=structlog.stdlib.LoggerFactory(),

butterrobot/objects.py

@@ -1,15 +1,61 @@
 from datetime import datetime
 from dataclasses import dataclass, field
-from typing import Text, Optional
+from typing import Text, Optional, Dict
+
+import structlog
+
+
+logger = structlog.get_logger(__name__)
+
+
+@dataclass
+class ChannelPlugin:
+    id: int
+    channel_id: int
+    plugin_id: str
+    enabled: bool = False
+    config: dict = field(default_factory=dict)
+
+
+@dataclass
+class Channel:
+    platform: str
+    platform_channel_id: str
+    channel_raw: dict
+    enabled: bool = False
+    id: Optional[int] = None
+    plugins: Dict[str, ChannelPlugin] = field(default_factory=dict)
+
+    def has_enabled_plugin(self, plugin_id):
+        if plugin_id not in self.plugins:
+            logger.debug("No enabled!", plugin_id=plugin_id, plugins=self.plugins)
+            return False
+
+        return self.plugins[plugin_id].enabled
+
+    @property
+    def channel_name(self):
+        from butterrobot.platforms import PLATFORMS
+
+        return PLATFORMS[self.platform].parse_channel_name_from_raw(self.channel_raw)
 
 
 @dataclass
 class Message:
     text: Text
     chat: Text
+    # TODO: Move chat references to `.channel.platform_channel_id`
+    channel: Optional[Channel] = None
     author: Text = None
     from_bot: bool = False
     date: Optional[datetime] = None
     id: Optional[Text] = None
     reply_to: Optional[Text] = None
     raw: dict = field(default_factory=dict)
+
+
+@dataclass
+class User:
+    id: int
+    username: Text
+    password: Text

butterrobot/platforms/__init__.py

@@ -1,6 +1,30 @@
+from functools import lru_cache
+
+import structlog
+
 from butterrobot.platforms.slack import SlackPlatform
 from butterrobot.platforms.telegram import TelegramPlatform
 from butterrobot.platforms.debug import DebugPlatform
 
 
-PLATFORMS = {platform.ID: platform for platform in (SlackPlatform, TelegramPlatform, DebugPlatform)}
+logger = structlog.get_logger(__name__)
+PLATFORMS = {
+    platform.ID: platform
+    for platform in (SlackPlatform, TelegramPlatform, DebugPlatform)
+}
+
+
+@lru_cache
+def get_available_platforms():
+    from butterrobot.platforms import PLATFORMS
+
+    available_platforms = {}
+    for platform in PLATFORMS.values():
+        logger.debug("Setting up", platform=platform.ID)
+        try:
+            platform.init(app=None)
+            available_platforms[platform.ID] = platform
+            logger.info("platform setup completed", platform=platform.ID)
+        except platform.PlatformInitError as error:
+            logger.error("Platform init error", error=error, platform=platform.ID)
+    return available_platforms

butterrobot/platforms/base.py

@@ -1,4 +1,4 @@
-from abc import abstractclassmethod
+from abc import abstractmethod
 from dataclasses import dataclass
 
 
@@ -17,19 +17,51 @@ class Platform:
         """
         Used when the platform needs to make a response right away instead of async.
         """
+
         data: dict
         status_code: int = 200
 
     @classmethod
     def init(cls, app):
+        """
+        Initialises the platform.
+
+        Used at the application launch to prepare anything required for
+        the platform to work..
+
+        It receives the flask application via parameter in case the platform
+        requires for custom webservice endpoints or configuration.
+        """
+        pass
+
+    @classmethod
+    @abstractmethod
+    def parse_incoming_message(cls, request):
+        """
+        Parses the incoming request and returns a :class:`butterrobot.objects.Message` instance.
+        """
+        pass
+
+    @classmethod
+    @abstractmethod
+    def parse_channel_name_from_raw(cls, channel_raw) -> str:
+        """
+        Extracts the Channel name from :class:`butterrobot.objects.Channel.channel_raw`.
+        """
+        pass
+
+    @classmethod
+    @abstractmethod
+    def parse_channel_from_message(cls, channel_raw):
+        """
+        Extracts the Channel raw data from the message received in the incoming webhook.
+        """
         pass
 
 
 class PlatformMethods:
-    @abstractclassmethod
+    @classmethod
+    @abstractmethod
     def send_message(cls, message):
-        pass
+        """Method used to send a message via the platform"""
-
-    @abstractclassmethod
-    def reply_message(cls, message, reply_to):
         pass

butterrobot/platforms/debug.py

@@ -4,7 +4,7 @@ from datetime import datetime
 import structlog
 
 from butterrobot.platforms.base import Platform, PlatformMethods
-from butterrobot.objects import Message
+from butterrobot.objects import Message, Channel
 
 
 logger = structlog.get_logger(__name__)
@@ -25,7 +25,7 @@ class DebugPlatform(Platform):
 
     @classmethod
     def parse_incoming_message(cls, request):
-        request_data = request.get_json()
+        request_data = request["json"]
         logger.debug("Parsing message", data=request_data, platform=cls.ID)
 
         return Message(
@@ -35,5 +35,10 @@ class DebugPlatform(Platform):
             from_bot=bool(request_data.get("from_bot", False)),
             author=request_data.get("author", "Debug author"),
             chat=request_data.get("chat", "Debug chat ID"),
+            channel=Channel(
+                platform=cls.ID,
+                platform_channel_id=request_data.get("chat"),
+                channel_raw={},
+            ),
             raw={},
         )

butterrobot/platforms/slack.py

@@ -4,7 +4,7 @@ import structlog
 
 from butterrobot.platforms.base import Platform, PlatformMethods
 from butterrobot.config import SLACK_TOKEN, SLACK_BOT_OAUTH_ACCESS_TOKEN
-from butterrobot.objects import Message
+from butterrobot.objects import Message, Channel
 from butterrobot.lib.slack import SlackAPI
 
 
@@ -41,9 +41,27 @@ class SlackPlatform(Platform):
             logger.error("Missing token. platform not enabled.", platform=cls.ID)
             return
 
+    @classmethod
+    def parse_channel_name_from_raw(cls, channel_raw):
+        return channel_raw["name"]
+
+    @classmethod
+    def parse_channel_from_message(cls, message):
+        # Call different APIs for a channel or DM
+        if message["event"]["channel_type"] == "im":
+            chat_raw = SlackAPI.get_user_info(message["event"]["user"])
+        else:
+            chat_raw = SlackAPI.get_conversations_info(message["event"]["channel"])
+
+        return Channel(
+            platform=cls.ID,
+            platform_channel_id=message["event"]["channel"],
+            channel_raw=chat_raw,
+        )
+
     @classmethod
     def parse_incoming_message(cls, request):
-        data = request.get_json()
+        data = request["json"]
 
         # Auth
         if data.get("token") != SLACK_TOKEN:
@@ -58,16 +76,30 @@ class SlackPlatform(Platform):
             logger.debug("Discarding message", data=data)
             return
 
-        if data["event"]["type"] != "message":
+        logger.debug("Parsing message", platform=cls.ID, data=data)
+
+        if data["event"]["type"] not in ("message", "message.groups"):
             return
 
-        logger.debug("Parsing message", platform=cls.ID, data=data)
+        # Surprisingly, this *can* happen.
-        return Message(
+        if "text" not in data["event"]:
+            return
+
+        message = Message(
             id=data["event"].get("thread_ts", data["event"]["ts"]),
-            author=data["event"]["user"],
+            author=data["event"].get("user"),
             from_bot="bot_id" in data["event"],
             date=datetime.fromtimestamp(int(float(data["event"]["event_ts"]))),
             text=data["event"]["text"],
             chat=data["event"]["channel"],
+            channel=cls.parse_channel_from_message(data),
             raw=data,
         )
+
+        logger.info(
+            "New message",
+            platform=message.channel.platform,
+            channel=cls.parse_channel_name_from_raw(message.channel.channel_raw),
+        )
+
+        return message

butterrobot/platforms/telegram.py

@@ -5,7 +5,7 @@ import structlog
 from butterrobot.platforms.base import Platform, PlatformMethods
 from butterrobot.config import TELEGRAM_TOKEN, HOSTNAME
 from butterrobot.lib.telegram import TelegramAPI
-from butterrobot.objects import Message
+from butterrobot.objects import Message, Channel
 
 
 logger = structlog.get_logger(__name__)
@@ -46,23 +46,42 @@ class TelegramPlatform(Platform):
             logger.error(f"Error setting Telegram webhook: {error}", platform=cls.ID)
             raise Platform.PlatformInitError()
 
+    @classmethod
+    def parse_channel_name_from_raw(cls, channel_raw):
+        if channel_raw["id"] < 0:
+            return channel_raw["title"]
+        else:
+            if channel_raw["username"]:
+                return f"@{channel_raw['username']}"
+        return f"{channel_raw['first_name']} {channel_raw['last_name']}"
+
+    @classmethod
+    def parse_channel_from_message(cls, channel_raw):
+        return Channel(
+            platform=cls.ID,
+            platform_channel_id=channel_raw["id"],
+            channel_raw=channel_raw,
+        )
+
     @classmethod
     def parse_incoming_message(cls, request):
-        token = request.path.split("/")[-1]
+        token = request["path"].split("/")[-1]
         if token != TELEGRAM_TOKEN:
             raise cls.PlatformAuthError("Authentication error")
 
-        request_data = request.get_json()
+        logger.debug("Parsing message", data=request["json"], platform=cls.ID)
-        logger.debug("Parsing message", data=request_data, platform=cls.ID)
 
-        if "text" in request_data["message"]:
+        if "text" in request["json"]["message"]:
             # Ignore all messages but text messages
             return Message(
-                id=request_data["message"]["message_id"],
+                id=request["json"]["message"]["message_id"],
-                date=datetime.fromtimestamp(request_data["message"]["date"]),
+                date=datetime.fromtimestamp(request["json"]["message"]["date"]),
-                text=str(request_data["message"]["text"]),
+                text=str(request["json"]["message"]["text"]),
-                from_bot=request_data["message"]["from"]["is_bot"],
+                from_bot=request["json"]["message"]["from"]["is_bot"],
-                author=request_data["message"]["from"]["id"],
+                author=request["json"]["message"]["from"]["id"],
-                chat=str(request_data["message"]["chat"]["id"]),
+                chat=str(request["json"]["message"]["chat"]["id"]),
-                raw=request_data,
+                channel=cls.parse_channel_from_message(
+                    request["json"]["message"]["chat"]
+                ),
+                raw=request["json"],
             )

butterrobot/plugins.py

@@ -1,24 +1,54 @@
 import traceback
 import pkg_resources
 from abc import abstractclassmethod
+from functools import lru_cache
+from typing import Optional, Dict
 
 import structlog
 
 from butterrobot.objects import Message
 
+
 logger = structlog.get_logger(__name__)
 
 
 class Plugin:
+    """
+    Base Plugin class.
+
+    All attributes are required except for `requires_config`.
+    """
+
+    id: str
+    name: str
+    help: str
+    requires_config: bool = False
+
     @abstractclassmethod
-    def on_message(cls, message: Message):
+    def on_message(cls, message: Message, channel_config: Optional[Dict] = None):
+        """
+        Function called for each message received on the chat.
+
+        It should exit as soon as possible (usually checking for a keyword or something)
+        similar just at the start.
+
+        If the plugin needs to be executed (keyword matches), keep it as fast as possible
+        as this currently blocks the execution of the rest of the plugins on the channel
+        until this does not finish.
+        TODO: Update this once we go proper async plugin/message integration
+
+        In case something needs to be answered to the channel, you can `yield` a `Message`
+        instance and it will be relayed using the appropriate provider.
+        """
         pass
 
 
+@lru_cache
 def get_available_plugins():
-    """Retrieves every available plugin"""
+    """
+    Retrieves every available auto discovered plugin
+    """
     plugins = {}
-    logger.debug("Loading plugins")
     for ep in pkg_resources.iter_entry_points("butterrobot.plugins"):
         try:
             plugin_cls = ep.load()
@@ -34,5 +64,4 @@ def get_available_plugins():
                 module=ep.module_name,
             )
 
-    logger.info(f"Plugins loaded", plugins=list(plugins.keys()))
     return plugins

butterrobot/queue.py

@@ -0,0 +1,64 @@
+import threading
+import traceback
+import queue
+
+import structlog
+
+from butterrobot.db import ChannelQuery
+from butterrobot.platforms import get_available_platforms
+from butterrobot.platforms.base import Platform
+from butterrobot.plugins import get_available_plugins
+
+logger = structlog.get_logger(__name__)
+q = queue.Queue()
+
+
+def handle_message(platform: str, request: dict):
+    try:
+        message = get_available_platforms()[platform].parse_incoming_message(
+            request=request
+        )
+    except Platform.PlatformAuthResponse as response:
+        return response.data, response.status_code
+    except Exception as error:
+        logger.error(
+            "Error parsing message",
+            platform=platform,
+            error=error,
+            traceback=traceback.format_exc(),
+        )
+        return
+
+    if not message or message.from_bot:
+        return
+
+    try:
+        channel = ChannelQuery.get_by_platform(platform, message.chat)
+    except ChannelQuery.NotFound:
+        # If channel is still not present on the database, create it (defaults to disabled)
+        channel = ChannelQuery.create(
+            platform, message.chat, channel_raw=message.channel.channel_raw
+        )
+
+    if not channel.enabled:
+        return
+
+    for plugin_id, channel_plugin in channel.plugins.items():
+        if not channel.has_enabled_plugin(plugin_id):
+            continue
+
+        for response_message in get_available_plugins()[plugin_id].on_message(
+            message, plugin_config=channel_plugin.config
+        ):
+            get_available_platforms()[platform].methods.send_message(response_message)
+
+
+def worker_thread():
+    while True:
+        item = q.get()
+        handle_message(item["platform"], item["request"])
+        q.task_done()
+
+
+# turn-on the worker thread
+worker = threading.Thread(target=worker_thread, daemon=True).start()

butterrobot_plugins_contrib/dev.py

@@ -5,10 +5,11 @@ from butterrobot.objects import Message
 
 
 class PingPlugin(Plugin):
-    id = "contrib/dev/ping"
+    name = "Ping command"
+    id = "contrib.dev.ping"
 
     @classmethod
-    def on_message(cls, message):
+    def on_message(cls, message, **kwargs):
         if message.text == "!ping":
             delta = datetime.now() - message.date
             delta_ms = delta.seconds * 1000 + delta.microseconds / 1000

butterrobot_plugins_contrib/fun.py

@@ -1,34 +1,51 @@
 import random
 
 import dice
+import structlog
 
 from butterrobot.plugins import Plugin
 from butterrobot.objects import Message
 
 
+logger = structlog.get_logger(__name__)
+
+
 class LoquitoPlugin(Plugin):
-    id = "contrib/fun/loquito"
+    name = "Loquito reply"
+    id = "contrib.fun.loquito"
 
     @classmethod
-    def on_message(cls, message):
+    def on_message(cls, message, **kwargs):
         if "lo quito" in message.text.lower():
-            yield Message(chat=message.chat, reply_to=message.id, text="Loquito tu.",)
+            yield Message(
+                chat=message.chat, reply_to=message.id, text="Loquito tu.",
+            )
 
 
 class DicePlugin(Plugin):
-    id = "contrib/fun/dice"
+    name = "Dice command"
+    id = "contrib.fun.dice"
+    DEFAULT_FORMULA = "1d20"
 
     @classmethod
-    def on_message(cls, message: Message):
+    def on_message(cls, message: Message, **kwargs):
         if message.text.startswith("!dice"):
-            roll = int(dice.roll(message.text.replace("!dice ", "")))
+            dice_formula = message.text.replace("!dice", "").strip()
+            if not dice_formula:
+                dice_formula = cls.DEFAULT_FORMULA
+            roll = int(dice.roll(dice_formula))
             yield Message(chat=message.chat, reply_to=message.id, text=roll)
 
 
 class CoinPlugin(Plugin):
-    id = "contrib/fun/coin"
+    name = "Coin command"
+    id = "contrib.fun.coin"
 
     @classmethod
-    def on_message(cls, message: Message):
+    def on_message(cls, message: Message, **kwargs):
         if message.text.startswith("!coin"):
-            yield Message(chat=message.chat, reply_to=message.id, text=random.choice(("heads", "tails")))
+            yield Message(
+                chat=message.chat,
+                reply_to=message.id,
+                text=random.choice(("heads", "tails")),
+            )

docker/Dockerfile

@@ -2,12 +2,15 @@ FROM alpine:3.11
 
 ENV PYTHON_VERSION=3.8.2-r1
 ENV APP_PORT 8080
-ENV BUTTERROBOT_VERSION 0.0.2a4
+ENV BUTTERROBOT_VERSION 0.0.3
 ENV EXTRA_DEPENDENCIES ""
+ENV APP_PATH /etc/butterrobot
 
 COPY bin/start-server.sh /usr/local/bin/start-server
 RUN apk --update add curl python3-dev==${PYTHON_VERSION} gcc musl-dev libffi-dev openssl-dev && \
-    pip3 install butterrobot==${BUTTERROBOT_VERSION} ${EXTRA_DEPENDENCIES}
+    pip3 install butterrobot==${BUTTERROBOT_VERSION} ${EXTRA_DEPENDENCIES} && \
+    mkdir ${APP_PATH} && \
+    chown -R 1000:1000 ${APP_PATH}
 
 USER 1000
 

docs/README.md

@@ -0,0 +1,8 @@
+# Butterrobot Documentation
+
+## Index
+- [Contributing](./contributing.md)
+- [Platforms](./platforms.md)
+- Plugins
+    - [Creating a Plugin](./creating-a-plugin.md)
+    - [Provided plugins](./plugins.md)

docs/contributing.md

@@ -0,0 +1,23 @@
+## Contributing
+
+To run the project locally you will need [poetry](https://python-poetry.org/).
+
+```
+git clone git@github.com:fmartingr/butterrobot.git
+cd butterrobot
+make setup
+```
+
+Create a `.env-local` file with the required environment variables, you have [an example file](.env-example).
+
+```
+SLACK_TOKEN=xxx
+TELEGRAM_TOKEN=xxx
+...
+```
+
+And then you can run it directly with poetry:
+
+```
+poetry run python -m butterrobot
+```

docs/creating-a-plugin.md

@@ -0,0 +1,37 @@
+# Creating a Plugin
+
+## Example
+
+This simple "Marco Polo" plugin will answer _Polo_ to the user that say _Marco_:
+
+``` python
+# mypackage/plugins.py
+from butterrobot.plugins import Plugin
+from butterrobot.objects import Message
+
+
+class PingPlugin(Plugin):
+    name = "Marco/Polo"
+    id = "test.marco"
+
+    @classmethod
+    def on_message(cls, message, **kwargs):
+        if message.text == "Marco":
+            yield Message(
+                chat=message.chat, reply_to=message.id, text=f"polo",
+            )
+```
+
+``` python
+# setup.py
+# ...
+entrypoints = {
+    "test.marco" = "mypackage.plugins:MarcoPlugin"
+}
+
+setup(
+    # ...
+    entry_points=entrypoints,
+    # ...
+)
+```

docs/platforms.md

@@ -0,0 +1,8 @@
+## Supported platforms
+
+TODO: Create better actions matrix
+
+| Name            | Receive messages | Send messages |
+| --------------- | ---------------- | ------------- |
+| Slack (app)     | Yes              | Yes           |
+| Telegram        | Yes              | Yes           |

docs/plugins.md

@@ -0,0 +1,11 @@
+## Provided plugins
+
+### Development
+
+- `!ping`: Say `!ping` to get response with time elapsed.
+
+### Fun and entertainment
+
+
+- Lo quito: What happens when you say _"lo quito"_...? (Spanish pun)
+- Dice: Put `!dice` and wathever roll you want to perform.

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "butterrobot"
-version = "0.0.2a4"
+version = "0.0.3"
 description = "What is my purpose?"
 authors = ["Felipe Martin <me@fmartingr.com>"]
 license = "GPL-2.0"
@@ -19,6 +19,7 @@ dice = "^3.1.0"
 flask = "^1.1.2"
 requests = "^2.24.0"
 waitress = "^1.4.4"
+dataset = "^1.3.2"
 
 [tool.poetry.dev-dependencies]
 black = "^19.10b0"
@@ -26,6 +27,9 @@ flake8 = "^3.7.9"
 rope = "^0.16.0"
 isort = "^4.3.21"
 ipdb = "^0.13.2"
+pytest = "^6.1.2"
+pytest-cov = "^2.10.1"
+pre-commit = "^2.10.0"
 
 [tool.poetry.plugins]
 [tool.poetry.plugins."butterrobot.plugins"]

setup.cfg

@@ -11,6 +11,6 @@ include_trailing_comma = True
 length_sort = 1
 lines_between_types = 0
 line_length = 88
-known_third_party = click,django,docker,factory,pydantic,pytest,requests,toml
+known_third_party = dataset,dice,flask,pkg_resources,pytest,requests,structlog
 sections = FUTURE, STDLIB, DJANGO, THIRDPARTY, FIRSTPARTY, LOCALFOLDER
 no_lines_before = LOCALFOLDER

tests/test_db.py

@@ -0,0 +1,109 @@
+import os.path
+import tempfile
+from dataclasses import dataclass
+from unittest import mock
+
+import dataset
+import pytest
+
+from butterrobot import db
+
+
+@dataclass
+class DummyItem:
+    id: int
+    foo: str
+
+
+class DummyQuery(db.Query):
+    tablename = "dummy"
+    obj = DummyItem
+
+
+class MockDatabase:
+    def __init__(self):
+        self.temp_dir = tempfile.TemporaryDirectory()
+
+    def __enter__(self):
+        db_path = os.path.join(self.temp_dir.name, "db.sqlite")
+        db.db = dataset.connect(f"sqlite:///{db_path}")
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.temp_dir.cleanup()
+
+
+def test_query_create_ok():
+    with MockDatabase():
+        assert DummyQuery.create(foo="bar")
+
+
+def test_query_delete_ok():
+    with MockDatabase():
+        item_id = DummyQuery.create(foo="bar")
+        assert DummyQuery.delete(item_id)
+
+
+def test_query_exists_by_id_ok():
+    with MockDatabase():
+        assert not DummyQuery.exists(id=1)
+        item_id = DummyQuery.create(foo="bar")
+        assert DummyQuery.exists(id=item_id)
+
+
+def test_query_exists_by_attribute_ok():
+    with MockDatabase():
+        assert not DummyQuery.exists(id=1)
+        item_id = DummyQuery.create(foo="bar")
+        assert DummyQuery.exists(foo="bar")
+
+
+def test_query_get_ok():
+    with MockDatabase():
+        item_id = DummyQuery.create(foo="bar")
+        item = DummyQuery.get(id=item_id)
+        assert item.id
+
+
+def test_query_all_ok():
+    with MockDatabase():
+        assert len(list(DummyQuery.all())) == 0
+        [DummyQuery.create(foo="bar") for i in range(0, 3)]
+        assert len(list(DummyQuery.all())) == 3
+
+
+def test_update_ok():
+    with MockDatabase():
+        expected = "bar2"
+        item_id = DummyQuery.create(foo="bar")
+        assert DummyQuery.update(item_id, foo=expected)
+        item = DummyQuery.get(id=item_id)
+        assert item.foo == expected
+
+
+def test_create_user_sets_password_ok():
+    password = "password"
+    with MockDatabase():
+        user_id = db.UserQuery.create(username="foo", password=password)
+        user = db.UserQuery.get(id=user_id)
+        assert user.password == db.UserQuery._hash_password(password)
+
+
+def test_user_check_credentials_ok():
+    with MockDatabase():
+        username = "foo"
+        password = "bar"
+        user_id = db.UserQuery.create(username=username, password=password)
+        user = db.UserQuery.get(id=user_id)
+        user = db.UserQuery.check_credentials(username, password)
+        assert isinstance(user, db.UserQuery.obj)
+
+
+def test_user_check_credentials_ko():
+    with MockDatabase():
+        username = "foo"
+        password = "bar"
+        user_id = db.UserQuery.create(username=username, password=password)
+        user = db.UserQuery.get(id=user_id)
+        assert not db.UserQuery.check_credentials(username, "error")
+        assert not db.UserQuery.check_credentials("error", password)
+        assert not db.UserQuery.check_credentials("error", "error")

tests/test_objects.py

@@ -0,0 +1,18 @@
+from butterrobot.objects import Channel, ChannelPlugin
+
+
+def test_channel_has_enabled_plugin_ok():
+    channel = Channel(
+        platform="debug",
+        platform_channel_id="debug",
+        channel_raw={},
+        plugins={
+            "enabled": ChannelPlugin(
+                id=1, channel_id="test", plugin_id="enabled", enabled=True
+            ),
+            "existant": ChannelPlugin(id=2, channel_id="test", plugin_id="existant"),
+        },
+    )
+    assert not channel.has_enabled_plugin("non.existant")
+    assert not channel.has_enabled_plugin("existant")
+    assert channel.has_enabled_plugin("enabled")