Compare commits

..

30 Commits

Author SHA1 Message Date
R. Miles McCain
bca3faa597 Remove tests directory 2021-11-13 21:09:09 -08:00
R. Miles McCain
822f7fb74c Code cleanup 2021-11-13 21:08:08 -08:00
Sergio
a47edbfa03 Add factories and first dashboard tests 2021-09-19 16:26:05 +03:00
R. Miles McCain
03ced00f63 Remove Pipfile 2021-07-31 23:54:08 +00:00
R. Miles McCain
8d04ed5c1f Always install debug toolbar 2021-07-31 23:53:41 +00:00
R. Miles McCain
f2879775ef Set default auto field 2021-07-31 23:49:01 +00:00
R. Miles McCain
c980567fee Formatting 2021-07-31 23:44:05 +00:00
R. Miles McCain
57c8695bcc Debug toolbar fix 2021-07-31 23:44:03 +00:00
R. Miles McCain
31ffa47fd3 Bump version 2021-07-20 13:37:37 +00:00
R. Miles McCain
73f3513dfe Use Poetry, not Pipenv 2021-07-20 04:51:13 +00:00
R. Miles McCain
b2e9d50d78 Update psycopg2 2021-07-20 03:05:18 +00:00
Casper Verswijvelt
de235c02a7 Add ability to toggle between map chart and country/session table (#153)
* Add ability to toggle between geo map and table view

* Add back haaavk's bar visualisation for countries table

* Change text/location of map/table toggle buttons

* Add some common css to reusable class

* CSS consistency

* Use button, not span for interactive elements

Co-authored-by: R. Miles McCain <oci@sendmiles.email>
2021-07-19 23:04:56 -04:00
Kasper Seweryn
31cb616242 Change snippet url to display current host (#159)
* Change snippet url to current host

* Change site.domain to host in page.js

* Remove useless condition

* Change hostname in email messages

* Remove `hostname` command

* Fix startup_checks.sh

* Remove unused variable from startup_checks.py
2021-07-19 22:55:30 -04:00
havk
2d5fbae279 Add sessions key to hits_per_day dict (#160) 2021-07-19 22:38:45 -04:00
Casper Verswijvelt
0153b1f847 Fix ellipsis in multiple tables (#152) 2021-06-15 15:29:59 -04:00
R. Miles McCain
473ad93081 Properly name manual build action 2021-06-14 20:56:29 +00:00
R. Miles McCain
1225ad90e8 Add manual build option
Signed-off-by: R. Miles McCain <github@sendmiles.email>
2021-06-14 20:55:22 +00:00
R. Miles McCain
e43718f596 Bump version 2021-06-13 20:11:28 +00:00
R. Miles McCain
d9623a9905 Also build image on push to master 2021-06-13 20:08:45 +00:00
R. Miles McCain
011f1f13c8 Also build image on push to master 2021-06-13 19:58:15 +00:00
Casper Verswijvelt
9832de0c19 GeoIP Map (#142)
* First working version of world map chart

* Cleanup code, fix aspect ratio, add GeoIP Map header

* Remove limited-height on session list with already limited content

* Update package lock

* Integrate map into service page

* Adjust map colors

* Adjust colors further

Co-authored-by: R. Miles McCain <github@sendmiles.email>
2021-06-13 15:50:01 -04:00
havk
83b20643d2 Add hits to chart data (#141)
* Add hits to chart data

* Hide legend

Co-authored-by: R. Miles McCain <github@sendmiles.email>
2021-06-13 14:39:45 -04:00
R. Miles McCain
ab44ba8318 Add subtle rounding to bar 2021-06-13 18:32:55 +00:00
havk
fcea6d3be9 Percents + visualization (#139)
* Black autoformat

* Add percent and divide filters

* Remove divide filter

* Add percents in brackets and visualization

* Apply percents and visualization to all data

* Switch absolute to relative

* Increase percent bar height

* Move bar to separated file

* Add USE_RELATIVE_MAX_IN_BAR_VISUALIZATION to settings

* Add flex items-center

* Move bar to left

* Remove spaces

* Fix USE_RELATIVE_MAX_IN_BAR_VISUALIZATION

* Remove unnecessary True

* Add bar_width tag

* Add flex-none to make flag not get squished

* Fix flex-none
2021-06-13 14:11:40 -04:00
Ian MacIntosh
f3a89bff78 Document possible time zone values (#147)
This might be a normal expectation, but it was new for me.

I added a URL to fast-track the next person in the same position.
2021-06-13 13:58:13 -04:00
dependabot[bot]
3c9bc9f3c9 Bump django from 3.1.9 to 3.1.12 (#150)
Bumps [django](https://github.com/django/django) from 3.1.9 to 3.1.12.
- [Release notes](https://github.com/django/django/releases)
- [Commits](https://github.com/django/django/compare/3.1.9...3.1.12)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-13 13:50:48 -04:00
dependabot[bot]
2f5d0ba7e5 Bump django from 3.1.8 to 3.1.9 (#146)
Bumps [django](https://github.com/django/django) from 3.1.8 to 3.1.9.
- [Release notes](https://github.com/django/django/releases)
- [Commits](https://github.com/django/django/compare/3.1.8...3.1.9)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-04 22:32:57 -04:00
dependabot[bot]
1c866209c9 Bump urllib3 from 1.26.4 to 1.26.5 (#145)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.4 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.4...1.26.5)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-02 11:27:14 -04:00
Casper Verswijvelt
a4785b1a0c Datepicker mobile overflow (#143)
* Remove w-auto to fix datepicker overflow on mobile

* Hardcoded width for datepicker input element
2021-05-23 12:14:03 -04:00
R. Miles McCain
2928e663db Move sample Kubernetes deployment to latest 2021-05-14 17:01:15 +00:00
39 changed files with 2778 additions and 626 deletions

43
.github/workflows/build-docker-edge.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
name: Build edge Docker images
on:
push:
branches:
- master
jobs:
publish_to_docker_hub:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Prepare tags
id: prep
run: |
DOCKER_IMAGE=milesmcc/shynet
TAGS="${DOCKER_IMAGE}:edge"
echo ::set-output name=tags::${TAGS}
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Build and push advanced image
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64,linux/arm/v7
push: true
tags: ${{ steps.prep.outputs.tags }}

View File

@@ -0,0 +1,43 @@
name: Build manual Docker images
on:
workflow_dispatch:
inputs:
tag:
description: 'Docker image tag'
jobs:
publish_to_docker_hub:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Prepare tags
id: prep
run: |
DOCKER_IMAGE=milesmcc/shynet
TAGS="${DOCKER_IMAGE}:${{ github.event.inputs.tag }}"
echo ::set-output name=tags::${TAGS}
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Build and push advanced image
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64,linux/arm/v7
push: true
tags: ${{ steps.prep.outputs.tags }}

View File

@@ -1,4 +1,4 @@
name: Build docker images
name: Build release Docker images
on:
push:
@@ -7,9 +7,7 @@ on:
jobs:
publish_to_docker_hub:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2

View File

@@ -1,26 +1,39 @@
FROM python:alpine3.12
FROM python:alpine3.14
# Getting things ready
WORKDIR /usr/src/shynet
COPY Pipfile.lock Pipfile ./
COPY package.json package-lock.json ../
# Django expects node_modules to be in its parent directory.
# Install dependencies & configure machine
ARG GF_UID="500"
ARG GF_GID="500"
RUN apk update && \
apk add gettext curl bash npm && \
curl -m 180 "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=kKG1ebhL3iWVd0iv&suffix=tar.gz" | tar -xvz -C /tmp && \
apk add gettext curl bash npm libffi-dev rust cargo
# libffi-dev and rust are used for the cryptography package,
# which we indirectly rely on. Necessary for aarch64 support.
# Collect GeoIP Database
RUN curl -m 180 "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=kKG1ebhL3iWVd0iv&suffix=tar.gz" | tar -xvz -C /tmp && \
curl -m 180 "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=kKG1ebhL3iWVd0iv&suffix=tar.gz" | tar -xvz -C /tmp && \
mv /tmp/GeoLite2*/*.mmdb /etc && \
apk del curl && \
apk add --no-cache postgresql-libs && \
apk del curl
# Move dependency files
COPY poetry.lock pyproject.toml ./
COPY package.json package-lock.json ../
# Django expects node_modules to be in its parent directory.
# Install more dependencies
RUN apk add --no-cache postgresql-libs && \
apk add --no-cache --virtual .build-deps gcc musl-dev postgresql-dev && \
npm i -P --prefix .. && \
pip install pipenv~=2020.6.2 && \
pipenv install --system --deploy && \
apk --purge del .build-deps && \
pip install poetry==1.1.7
# Install Python dependencies
RUN poetry config virtualenvs.create false && \
poetry install --no-dev --no-interaction --no-ansi
# Cleanup dependencies & setup user group
RUN apk --purge del .build-deps && \
rm -rf /var/lib/apt/lists/* && \
rm /var/cache/apk/* && \
addgroup --system -g $GF_GID appgroup && \

View File

@@ -41,15 +41,13 @@ Before continuing, please be sure to have the latest version of Docker installed
5. Create an admin user by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py registeradmin <your email>`. A temporary password will be printed to the console.
6. Set the hostname of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py hostname <your public hostname>`, where `<your public hostname>` is the _publicly accessible hostname_ of your instance, including port. This setting affects the URL that the tracking script sends its results to, so make sure it's correct. (Example hostnames: `shynet.example.com` or `example.com:8000`.)
6. Set the whitelabel of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: `"My Shynet Instance"` or `"Acme Analytics"`.)
7. Set the whitelabel of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: `"My Shynet Instance"` or `"Acme Analytics"`.)
7. Launch your webserver by running `docker run --env-file=<your env file> milesmcc/shynet:latest`. You may need to bind Docker's port 8080 (where Shynet runs) to your local port 80 (http); this can be done using the flag `-p 80:8080` after `run`. Visit your service's homepage, and verify everything looks right! You should see a login prompt. Log in with the credentials from step 5. You'll probably be prompted to "confirm your email"—if you haven't set up an email server, the confirmation email will be printed to the console instead.
8. Launch your webserver by running `docker run --env-file=<your env file> milesmcc/shynet:latest`. You may need to bind Docker's port 8080 (where Shynet runs) to your local port 80 (http); this can be done using the flag `-p 80:8080` after `run`. Visit your service's homepage, and verify everything looks right! You should see a login prompt. Log in with the credentials from step 5. You'll probably be prompted to "confirm your email"—if you haven't set up an email server, the confirmation email will be printed to the console instead.
8. Create a service by clicking "+ Create Service" in the top right hand corner. Fill out the options as appropriate. Once you're done, press "create" and you'll be redirected to your new service's analytics page.
9. Create a service by clicking "+ Create Service" in the top right hand corner. Fill out the options as appropriate. Once you're done, press "create" and you'll be redirected to your new service's analytics page.
10. Finally, click on "Manage" in the top right of the service's page to get the tracking script code. Inject this script on all pages you'd like the service to track.
9. Finally, click on "Manage" in the top right of the service's page to get the tracking script code. Inject this script on all pages you'd like the service to track.
### Basic Installation with Docker Compose
@@ -66,9 +64,7 @@ Before continuing, please be sure to have the latest version of Docker installed
5. Create an admin user by running `docker exec -it shynet_main ./manage.py registeradmin <your email>`. A temporary password will be printed to the console.
6. Set the hostname of your Shynet instance by running `docker exec -it shynet_main ./manage.py hostname <your public hostname>`, where `<your public hostname>` is the same as the hostname you set in step 3. This setting affects the URL that the tracking script sends its results to, so make sure it's correct. (Example hostnames: shynet.example.com or example.com:8000.)
7. Set the whitelabel of your Shynet instance by running `docker exec -it shynet_main ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: "My Shynet Instance" or "Acme Analytics".)
6. Set the whitelabel of your Shynet instance by running `docker exec -it shynet_main ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: "My Shynet Instance" or "Acme Analytics".)
Your site should now be accessible at `http://hostname:port`. Now you can follow steps 9-10 of the [Basic Installation](#basic-installation) guide above to get Shynet integrated on your sites.
@@ -78,11 +74,10 @@ You may wish to deploy Shynet on Heroku. Note that Heroku's free offerings (name
[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/milesmcc/shynet/tree/master)
Once you deploy, you'll need to setup an admin user, whitelabel, and hostname before you can use Shynet. Do that with the following commands:
Once you deploy, you'll need to setup an admin user and whitelabel before you can use Shynet. Do that with the following commands:
1. `heroku run --app=<your app> ./manage.py registeradmin <your email>`
2. `heroku run --app=<your app> ./manage.py hostname <the hostname where you will run Shynet>`
3. `heroku run --app=<your app> ./manage.py whitelabel "<your Shynet instance's name>"`
2. `heroku run --app=<your app> ./manage.py whitelabel "<your Shynet instance's name>"`
## Render
@@ -93,8 +88,7 @@ Once you deploy, you'll need to setup an admin user, whitelabel, and hostname be
Once your deploy has completed, use the **Render Shell** to configure your app:
1. Set your email: `./manage.py registeradmin your-email@example.com`
1. Add your onrender.com domain: `./manage.py hostname your-shynet-domain.onrender.com`
1. Set your whitelabel: `./manage.py whitelabel "Your Shynet Instance Name"`
2. Set your whitelabel: `./manage.py whitelabel "Your Shynet Instance Name"`
See the [Render docs](https://render.com/docs/deploy-shynet) for more information on deploying your application on Render.
@@ -272,4 +266,4 @@ Here are solutions for some common issues. If your situation isn't described her
#### Shynet can't connect to my database running on `localhost`/`127.0.0.1`
* The problem is likely that to Shynet, `localhost` points to the local network in the container itself, not on the host machine. Try adding the `--network='host'` option when you run Docker.
* The problem is likely that to Shynet, `localhost` points to the local network in the container itself, not on the host machine. Try adding the `--network='host'` option when you run Docker.

25
Pipfile
View File

@@ -1,25 +0,0 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[packages]
django = "~=3.1"
django-allauth = "~=0.42.0"
geoip2 = "~=3.0.0"
whitenoise = "~=5.1.0"
celery = "~=4.4.6"
django-ipware = "~=2.1.0"
pyyaml = "~=5.4"
ua-parser = "~=0.10.0"
user-agents = "~=2.1"
rules = "~=2.2"
gunicorn = "~=20.0.4"
psycopg2-binary = "~=2.8.5"
redis = "~=3.5.3"
django-redis-cache = "~=3.0.0"
pycountry = "~=19.8.18"
html2text = "~=2020.1.16"
django-health-check = "~=3.12.1"
django-npm = "~=1.0.0"
django-debug-toolbar = "*"

343
Pipfile.lock generated
View File

@@ -1,343 +0,0 @@
{
"_meta": {
"hash": {
"sha256": "c51ea0205c9ffe753b9ef5249cd49c2338bb50768ae104113bfb7b97b5f9d70c"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"amqp": {
"hashes": [
"sha256:70cdb10628468ff14e57ec2f751c7aa9e48e7e3651cfd62d431213c0c4e58f21",
"sha256:aa7f313fb887c91f15474c1229907a04dac0b8135822d6603437803424c0aa59"
],
"version": "==2.6.1"
},
"asgiref": {
"hashes": [
"sha256:92906c611ce6c967347bbfea733f13d6313901d54dcca88195eaeb52b2a8e8ee",
"sha256:d1216dfbdfb63826470995d31caed36225dcaf34f182e0fa257a4dd9e86f1b78"
],
"version": "==3.3.4"
},
"billiard": {
"hashes": [
"sha256:299de5a8da28a783d51b197d496bef4f1595dd023a93a4f59dde1886ae905547",
"sha256:87103ea78fa6ab4d5c751c4909bcff74617d985de7fa8b672cf8618afd5a875b"
],
"version": "==3.6.4.0"
},
"celery": {
"hashes": [
"sha256:a92e1d56e650781fb747032a3997d16236d037c8199eacd5217d1a72893bca45",
"sha256:d220b13a8ed57c78149acf82c006785356071844afe0b27012a4991d44026f9f"
],
"index": "pypi",
"version": "==4.4.7"
},
"certifi": {
"hashes": [
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.12.5"
},
"chardet": {
"hashes": [
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
],
"version": "==4.0.0"
},
"defusedxml": {
"hashes": [
"sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69",
"sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61"
],
"version": "==0.7.1"
},
"django": {
"hashes": [
"sha256:c348b3ddc452bf4b62361f0752f71a339140c777ebea3cdaaaa8fdb7f417a862",
"sha256:f8393103e15ec2d2d313ccbb95a3f1da092f9f58d74ac1c61ca2ac0436ae1eac"
],
"index": "pypi",
"version": "==3.1.8"
},
"django-allauth": {
"hashes": [
"sha256:f17209410b7f87da0a84639fd79d3771b596a6d3fc1a8e48ce50dabc7f441d30"
],
"index": "pypi",
"version": "==0.42.0"
},
"django-debug-toolbar": {
"hashes": [
"sha256:a5ff2a54f24bf88286f9872836081078f4baa843dc3735ee88524e89f8821e33",
"sha256:e759e63e3fe2d3110e0e519639c166816368701eab4a47fed75d7de7018467b9"
],
"index": "pypi",
"version": "==3.2.1"
},
"django-health-check": {
"hashes": [
"sha256:2667b89b8f85ad9b2a24c90581b376016d22ea912fedf37f9866413a3c2e0a5d",
"sha256:894738bd7e461b2405c005927403ad5ee8048bbaf5934cf30b2c81a4e047d4b0"
],
"index": "pypi",
"version": "==3.12.3"
},
"django-ipware": {
"hashes": [
"sha256:a7c7a8fd019dbdc9c357e6e582f65034e897572fc79a7e467674efa8aef9d00b"
],
"index": "pypi",
"version": "==2.1.0"
},
"django-npm": {
"hashes": [
"sha256:2e6bba65e728fa18b9db3c8dc0d4490b70cb7f43bacf60eb3654d7dcb6424272"
],
"index": "pypi",
"version": "==1.0.0"
},
"django-redis-cache": {
"hashes": [
"sha256:9a2eebef421d996a82098a19d17ff6b321265cd73178fa398913019764e8394a"
],
"index": "pypi",
"version": "==3.0.0"
},
"geoip2": {
"hashes": [
"sha256:5869e987bc54c0d707264fec4710661332cc38d2dca5a7f9bb5362d0308e2ce0",
"sha256:99ec12d2f1271a73a0a4a2b663fe6ce25fd02289c0a6bef05c0a1c3b30ee95a4"
],
"index": "pypi",
"version": "==3.0.0"
},
"gunicorn": {
"hashes": [
"sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626",
"sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c"
],
"index": "pypi",
"version": "==20.0.4"
},
"html2text": {
"hashes": [
"sha256:c7c629882da0cf377d66f073329ccf34a12ed2adf0169b9285ae4e63ef54c82b",
"sha256:e296318e16b059ddb97f7a8a1d6a5c1d7af4544049a01e261731d2d5cc277bbb"
],
"index": "pypi",
"version": "==2020.1.16"
},
"idna": {
"hashes": [
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
],
"version": "==2.10"
},
"kombu": {
"hashes": [
"sha256:be48cdffb54a2194d93ad6533d73f69408486483d189fe9f5990ee24255b0e0a",
"sha256:ca1b45faac8c0b18493d02a8571792f3c40291cf2bcf1f55afed3d8f3aa7ba74"
],
"version": "==4.6.11"
},
"maxminddb": {
"hashes": [
"sha256:47e86a084dd814fac88c99ea34ba3278a74bc9de5a25f4b815b608798747c7dc"
],
"version": "==2.0.3"
},
"oauthlib": {
"hashes": [
"sha256:bee41cc35fcca6e988463cacc3bcb8a96224f470ca547e697b604cc697b2f889",
"sha256:df884cd6cbe20e32633f1db1072e9356f53638e4361bef4e8b03c9127c9328ea"
],
"version": "==3.1.0"
},
"psycopg2-binary": {
"hashes": [
"sha256:0deac2af1a587ae12836aa07970f5cb91964f05a7c6cdb69d8425ff4c15d4e2c",
"sha256:0e4dc3d5996760104746e6cfcdb519d9d2cd27c738296525d5867ea695774e67",
"sha256:11b9c0ebce097180129e422379b824ae21c8f2a6596b159c7659e2e5a00e1aa0",
"sha256:15978a1fbd225583dd8cdaf37e67ccc278b5abecb4caf6b2d6b8e2b948e953f6",
"sha256:1fabed9ea2acc4efe4671b92c669a213db744d2af8a9fc5d69a8e9bc14b7a9db",
"sha256:2dac98e85565d5688e8ab7bdea5446674a83a3945a8f416ad0110018d1501b94",
"sha256:42ec1035841b389e8cc3692277a0bd81cdfe0b65d575a2c8862cec7a80e62e52",
"sha256:6422f2ff0919fd720195f64ffd8f924c1395d30f9a495f31e2392c2efafb5056",
"sha256:6a32f3a4cb2f6e1a0b15215f448e8ce2da192fd4ff35084d80d5e39da683e79b",
"sha256:7312e931b90fe14f925729cde58022f5d034241918a5c4f9797cac62f6b3a9dd",
"sha256:7d92a09b788cbb1aec325af5fcba9fed7203897bbd9269d5691bb1e3bce29550",
"sha256:833709a5c66ca52f1d21d41865a637223b368c0ee76ea54ca5bad6f2526c7679",
"sha256:89705f45ce07b2dfa806ee84439ec67c5d9a0ef20154e0e475e2b2ed392a5b83",
"sha256:8cd0fb36c7412996859cb4606a35969dd01f4ea34d9812a141cd920c3b18be77",
"sha256:950bc22bb56ee6ff142a2cb9ee980b571dd0912b0334aa3fe0fe3788d860bea2",
"sha256:a0c50db33c32594305b0ef9abc0cb7db13de7621d2cadf8392a1d9b3c437ef77",
"sha256:a0eb43a07386c3f1f1ebb4dc7aafb13f67188eab896e7397aa1ee95a9c884eb2",
"sha256:aaa4213c862f0ef00022751161df35804127b78adf4a2755b9f991a507e425fd",
"sha256:ac0c682111fbf404525dfc0f18a8b5f11be52657d4f96e9fcb75daf4f3984859",
"sha256:ad20d2eb875aaa1ea6d0f2916949f5c08a19c74d05b16ce6ebf6d24f2c9f75d1",
"sha256:b4afc542c0ac0db720cf516dd20c0846f71c248d2b3d21013aa0d4ef9c71ca25",
"sha256:b8a3715b3c4e604bcc94c90a825cd7f5635417453b253499664f784fc4da0152",
"sha256:ba28584e6bca48c59eecbf7efb1576ca214b47f05194646b081717fa628dfddf",
"sha256:ba381aec3a5dc29634f20692349d73f2d21f17653bda1decf0b52b11d694541f",
"sha256:bd1be66dde2b82f80afb9459fc618216753f67109b859a361cf7def5c7968729",
"sha256:c2507d796fca339c8fb03216364cca68d87e037c1f774977c8fc377627d01c71",
"sha256:cec7e622ebc545dbb4564e483dd20e4e404da17ae07e06f3e780b2dacd5cee66",
"sha256:d14b140a4439d816e3b1229a4a525df917d6ea22a0771a2a78332273fd9528a4",
"sha256:d1b4ab59e02d9008efe10ceabd0b31e79519da6fb67f7d8e8977118832d0f449",
"sha256:d5227b229005a696cc67676e24c214740efd90b148de5733419ac9aaba3773da",
"sha256:e1f57aa70d3f7cc6947fd88636a481638263ba04a742b4a37dd25c373e41491a",
"sha256:e74a55f6bad0e7d3968399deb50f61f4db1926acf4a6d83beaaa7df986f48b1c",
"sha256:e82aba2188b9ba309fd8e271702bd0d0fc9148ae3150532bbb474f4590039ffb",
"sha256:ee69dad2c7155756ad114c02db06002f4cded41132cc51378e57aad79cc8e4f4",
"sha256:f5ab93a2cb2d8338b1674be43b442a7f544a0971da062a5da774ed40587f18f5"
],
"index": "pypi",
"version": "==2.8.6"
},
"pycountry": {
"hashes": [
"sha256:3c57aa40adcf293d59bebaffbe60d8c39976fba78d846a018dc0c2ec9c6cb3cb"
],
"index": "pypi",
"version": "==19.8.18"
},
"python3-openid": {
"hashes": [
"sha256:33fbf6928f401e0b790151ed2b5290b02545e8775f982485205a066f874aaeaf",
"sha256:6626f771e0417486701e0b4daff762e7212e820ca5b29fcc0d05f6f8736dfa6b"
],
"version": "==3.2.0"
},
"pytz": {
"hashes": [
"sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da",
"sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798"
],
"version": "==2021.1"
},
"pyyaml": {
"hashes": [
"sha256:08682f6b72c722394747bddaf0aa62277e02557c0fd1c42cb853016a38f8dedf",
"sha256:0f5f5786c0e09baddcd8b4b45f20a7b5d61a7e7e99846e3c799b05c7c53fa696",
"sha256:129def1b7c1bf22faffd67b8f3724645203b79d8f4cc81f674654d9902cb4393",
"sha256:294db365efa064d00b8d1ef65d8ea2c3426ac366c0c4368d930bf1c5fb497f77",
"sha256:3b2b1824fe7112845700f815ff6a489360226a5609b96ec2190a45e62a9fc922",
"sha256:3bd0e463264cf257d1ffd2e40223b197271046d09dadf73a0fe82b9c1fc385a5",
"sha256:4465124ef1b18d9ace298060f4eccc64b0850899ac4ac53294547536533800c8",
"sha256:49d4cdd9065b9b6e206d0595fee27a96b5dd22618e7520c33204a4a3239d5b10",
"sha256:4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc",
"sha256:5accb17103e43963b80e6f837831f38d314a0495500067cb25afab2e8d7a4018",
"sha256:607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e",
"sha256:6c78645d400265a062508ae399b60b8c167bf003db364ecb26dcab2bda048253",
"sha256:72a01f726a9c7851ca9bfad6fd09ca4e090a023c00945ea05ba1638c09dc3347",
"sha256:74c1485f7707cf707a7aef42ef6322b8f97921bd89be2ab6317fd782c2d53183",
"sha256:895f61ef02e8fed38159bb70f7e100e00f471eae2bc838cd0f4ebb21e28f8541",
"sha256:8c1be557ee92a20f184922c7b6424e8ab6691788e6d86137c5d93c1a6ec1b8fb",
"sha256:bb4191dfc9306777bc594117aee052446b3fa88737cd13b7188d0e7aa8162185",
"sha256:bfb51918d4ff3d77c1c856a9699f8492c612cde32fd3bcd344af9be34999bfdc",
"sha256:c20cfa2d49991c8b4147af39859b167664f2ad4561704ee74c1de03318e898db",
"sha256:cb333c16912324fd5f769fff6bc5de372e9e7a202247b48870bc251ed40239aa",
"sha256:d2d9808ea7b4af864f35ea216be506ecec180628aced0704e34aca0b040ffe46",
"sha256:d483ad4e639292c90170eb6f7783ad19490e7a8defb3e46f97dfe4bacae89122",
"sha256:dd5de0646207f053eb0d6c74ae45ba98c3395a571a2891858e87df7c9b9bd51b",
"sha256:e1d4970ea66be07ae37a3c2e48b5ec63f7ba6804bdddfdbd3cfd954d25a82e63",
"sha256:e4fac90784481d221a8e4b1162afa7c47ed953be40d31ab4629ae917510051df",
"sha256:fa5ae20527d8e831e8230cbffd9f8fe952815b2b7dae6ffec25318803a7528fc",
"sha256:fd7f6999a8070df521b6384004ef42833b9bd62cfee11a09bda1079b4b704247",
"sha256:fdc842473cd33f45ff6bce46aea678a54e3d21f1b61a7750ce3c498eedfe25d6",
"sha256:fe69978f3f768926cfa37b867e3843918e012cf83f680806599ddce33c2c68b0"
],
"index": "pypi",
"version": "==5.4.1"
},
"redis": {
"hashes": [
"sha256:0e7e0cfca8660dea8b7d5cd8c4f6c5e29e11f31158c0b0ae91a397f00e5a05a2",
"sha256:432b788c4530cfe16d8d943a09d40ca6c16149727e4afe8c2c9d5580c59d9f24"
],
"index": "pypi",
"version": "==3.5.3"
},
"requests": {
"hashes": [
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
],
"version": "==2.25.1"
},
"requests-oauthlib": {
"hashes": [
"sha256:7f71572defaecd16372f9006f33c2ec8c077c3cfa6f5911a9a90202beb513f3d",
"sha256:b4261601a71fd721a8bd6d7aa1cc1d6a8a93b4a9f5e96626f8e4d91e8beeaa6a"
],
"version": "==1.3.0"
},
"rules": {
"hashes": [
"sha256:9bae429f9d4f91a375402990da1541f9e093b0ac077221d57124d06eeeca4405"
],
"index": "pypi",
"version": "==2.2"
},
"sqlparse": {
"hashes": [
"sha256:017cde379adbd6a1f15a61873f43e8274179378e95ef3fede90b5aa64d304ed0",
"sha256:0f91fd2e829c44362cbcfab3e9ae12e22badaa8a29ad5ff599f9ec109f0454e8"
],
"version": "==0.4.1"
},
"ua-parser": {
"hashes": [
"sha256:46ab2e383c01dbd2ab284991b87d624a26a08f72da4d7d413f5bfab8b9036f8a",
"sha256:47b1782ed130d890018d983fac37c2a80799d9e0b9c532e734c67cf70f185033"
],
"index": "pypi",
"version": "==0.10.0"
},
"urllib3": {
"hashes": [
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
],
"version": "==1.26.4"
},
"user-agents": {
"hashes": [
"sha256:a98c4dc72ecbc64812c4534108806fb0a0b3a11ec3fd1eafe807cee5b0a942e7",
"sha256:d36d25178db65308d1458c5fa4ab39c9b2619377010130329f3955e7626ead26"
],
"index": "pypi",
"version": "==2.2.0"
},
"vine": {
"hashes": [
"sha256:133ee6d7a9016f177ddeaf191c1f58421a1dcc6ee9a42c58b34bed40e1d2cd87",
"sha256:ea4947cc56d1fd6f2095c8d543ee25dad966f78692528e68b4fada11ba3f98af"
],
"version": "==1.3.0"
},
"whitenoise": {
"hashes": [
"sha256:60154b976a13901414a25b0273a841145f77eb34a141f9ae032a0ace3e4d5b27",
"sha256:6dd26bfda3af29177d8ab7333a0c7b7642eb615ce83764f4d15a9aecda3201c4"
],
"index": "pypi",
"version": "==5.1.0"
}
},
"develop": {}
}

View File

@@ -33,6 +33,8 @@ ACCOUNT_SIGNUPS_ENABLED=False
ACCOUNT_EMAIL_VERIFICATION=none
# The timezone of the admin panel. Affects how dates are displayed.
# This must match a value from the IANA's tz database.
# Wikipedia has a list of valid strings: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TIME_ZONE=America/New_York
# Set to "False" if you will not be serving content over HTTPS
@@ -96,3 +98,6 @@ LOCATION_URL=https://www.openstreetmap.org/?mlat=$LATITUDE&mlon=$LONGITUDE
# How many services should be displayed on dashboard page?
# Set to big number if you don't want pagination at all.
DASHBOARD_PAGE_SIZE=5
# Should background bars be scaled to full width?
USE_RELATIVE_MAX_IN_BAR_VISUALIZATION=True

View File

@@ -132,6 +132,11 @@
"description": "How many services should be displayed on dashboard page?",
"value": "5",
"required": false
},
"USE_RELATIVE_MAX_IN_BAR_VISUALIZATION": {
"description": "Should background bars be scaled to full width?",
"value": "True",
"required": false
}
}
}

View File

@@ -17,7 +17,7 @@ spec:
spec:
containers:
- name: "shynet-webserver"
image: "milesmcc/shynet:dev"
image: "milesmcc/shynet:latest"
imagePullPolicy: Always
envFrom:
- secretRef:
@@ -42,7 +42,7 @@ spec:
spec:
containers:
- name: "shynet-celeryworker"
image: "milesmcc/shynet:dev"
image: "milesmcc/shynet:latest"
command: ["./celeryworker.sh"]
imagePullPolicy: Always
envFrom:

1020
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -20,6 +20,7 @@
"@fortawesome/fontawesome-free": "^5.15.1",
"a17t": "^0.5.1",
"apexcharts": "^3.24.0",
"datamaps": "^0.5.9",
"flag-icon-css": "^3.5.0",
"inter-ui": "^3.15.0",
"litepicker": "^2.0.11"

1093
poetry.lock generated Normal file

File diff suppressed because it is too large Load Diff

32
pyproject.toml Normal file
View File

@@ -0,0 +1,32 @@
[tool.poetry]
name = "shynet"
version = "0.10.0"
description = "Modern, privacy-friendly, and cookie-free web analytics."
authors = ["R. Miles McCain <github@sendmiles.email>"]
license = "Apache-2.0"
[tool.poetry.dependencies]
python = "^3.8"
Django = "^3.2.5"
django-allauth = "^0.45.0"
geoip2 = "^4.2.0"
whitenoise = "^5.3.0"
celery = "^5.1.2"
django-ipware = "^3.0.2"
PyYAML = "^5.4.1"
user-agents = "^2.2.0"
rules = "^3.0"
gunicorn = "^20.1.0"
psycopg2-binary = "^2.9.1"
redis = "^3.5.3"
django-redis-cache = "^3.0.0"
pycountry = "^20.7.3"
html2text = "^2020.1.16"
django-health-check = "^3.16.4"
django-npm = "^1.0.0"
python-dotenv = "^0.18.0"
django-debug-toolbar = "^3.2.1"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

View File

@@ -19,7 +19,7 @@ var Shynet = {
var xhr = new XMLHttpRequest();
xhr.open(
"POST",
"{{protocol}}://{{request.site.domain|default:request.META.HTTP_HOST}}{{endpoint}}",
"{{protocol}}://{{request.get_host}}{{endpoint}}",
true
);
xhr.setRequestHeader("Content-Type", "application/json");

37
shynet/core/factories.py Normal file
View File

@@ -0,0 +1,37 @@
from django.contrib.auth import get_user_model
import factory
from factory.django import DjangoModelFactory
from .models import Service
class UserFactory(DjangoModelFactory):
username = factory.Faker("user_name")
email = factory.Faker("email")
name = factory.Faker("name")
@post_generation
def password(self, create, extracted, **kwargs):
password = (
extracted
if extracted
else factory.Faker(
"password",
length=42,
special_chars=True,
digits=True,
upper_case=True,
lower_case=True,
).evaluate(None, None, extra={"locale": None})
)
self.set_password(password)
class Meta:
model = get_user_model()
django_get_or_create = ["username"]
class ServiceFactory(DjangoModelFactory):
class Meta:
model = Service
name = factory.Faker("company")

View File

@@ -1,35 +0,0 @@
import traceback
import uuid
from django.conf import settings
from django.contrib.sites.models import Site
from django.core.management.base import BaseCommand, CommandError
from django.utils.crypto import get_random_string
from core.models import User
class Command(BaseCommand):
help = "Configures the Shynet hostname"
def add_arguments(self, parser):
parser.add_argument(
"hostname",
type=str,
)
def handle(self, *args, **options):
site = Site.objects.get(pk=settings.SITE_ID)
site.domain = options.get("hostname")
if options.get("hostname").lower().startswith("http"):
self.stdout.write(
self.style.WARNING(
f"Warning: the hostname '{options.get('hostname')}' starts with `http`. You almost certainly don't want this. The hostname is supposed to be the raw domain name of your Shynet instance, without `http://` or `https://`. For example, if your Shynet instance will eventually be hosted at `https://analytics.example.com`, the hostname should be `analytics.example.com`."
)
)
site.save()
self.stdout.write(
self.style.SUCCESS(
f"Successfully set the hostname to '{options.get('hostname')}'"
)
)

View File

@@ -35,15 +35,9 @@ class Command(BaseCommand):
def handle(self, *args, **options):
migration = self.check_migrations()
admin, hostname, whitelabel = [True] * 3
admin, whitelabel = [True] * 2
if not migration:
admin = not User.objects.all().exists()
hostname = (
not Site.objects.filter(domain__isnull=False)
.exclude(domain__exact="")
.exclude(domain__exact="example.com")
.exists()
)
whitelabel = (
not Site.objects.filter(name__isnull=False)
.exclude(name__exact="")
@@ -51,6 +45,4 @@ class Command(BaseCommand):
.exists()
)
self.stdout.write(
self.style.SUCCESS(f"{migration} {admin} {hostname} {whitelabel}")
)
self.stdout.write(self.style.SUCCESS(f"{migration} {admin} {whitelabel}"))

View File

@@ -198,6 +198,36 @@ class Service(models.Model):
avg_hits_per_session = hit_count / session_count if session_count > 0 else None
avg_session_duration = self._get_avg_session_duration(sessions, session_count)
chart_data, chart_tooltip_format, chart_granularity = self._get_chart_data(
sessions, hits, start_time, end_time, tz_now
)
return {
"currently_online": currently_online,
"session_count": session_count,
"hit_count": hit_count,
"has_hits": has_hits,
"bounce_rate_pct": bounce_count * 100 / session_count
if session_count > 0
else None,
"avg_session_duration": avg_session_duration,
"avg_load_time": avg_load_time,
"avg_hits_per_session": avg_hits_per_session,
"locations": locations,
"referrers": referrers,
"countries": countries,
"operating_systems": operating_systems,
"browsers": browsers,
"devices": devices,
"device_types": device_types,
"chart_data": chart_data,
"chart_tooltip_format": chart_tooltip_format,
"chart_granularity": chart_granularity,
"online": True,
}
def _get_avg_session_duration(self, sessions, session_count):
try:
avg_session_duration = sessions.annotate(
duration=models.F("last_seen") - models.F("start_time")
@@ -212,68 +242,75 @@ class Service(models.Model):
if session_count == 0:
avg_session_duration = None
return avg_session_duration
def _get_chart_data(self, sessions, hits, start_time, end_time, tz_now):
# Show hourly chart for date ranges of 3 days or less, otherwise daily chart
if (end_time - start_time).days < 3:
session_chart_tooltip_format = "MM/dd HH:mm"
session_chart_granularity = "hourly"
session_chart_data = {
k["hour"]: k["count"]
for k in sessions.annotate(hour=TruncHour("start_time"))
chart_tooltip_format = "MM/dd HH:mm"
chart_granularity = "hourly"
sessions_per_hour = (
sessions.annotate(hour=TruncHour("start_time"))
.values("hour")
.annotate(count=models.Count("uuid"))
.order_by("hour")
)
chart_data = {
k["hour"]: {"sessions": k["count"]} for k in sessions_per_hour
}
for hour_offset in range(int((end_time - start_time).total_seconds() / 3600) + 1):
hour = (start_time + timezone.timedelta(hours=hour_offset))
if hour not in session_chart_data:
session_chart_data[hour] = 0 if hour <= tz_now else None
hits_per_hour = (
hits.annotate(hour=TruncHour("start_time"))
.values("hour")
.annotate(count=models.Count("id"))
.order_by("hour")
)
for k in hits_per_hour:
if k["hour"] not in chart_data:
chart_data[k["hour"]] = {"hits": k["count"], "sessions": 0}
else:
chart_data[k["hour"]]["hits"] = k["count"]
hours_range = range(int((end_time - start_time).total_seconds() / 3600) + 1)
for hour_offset in hours_range:
hour = start_time + timezone.timedelta(hours=hour_offset)
if hour not in chart_data and hour <= tz_now:
chart_data[hour] = {"sessions": 0, "hits": 0}
else:
session_chart_tooltip_format = "MMM d"
session_chart_granularity = "daily"
session_chart_data = {
k["date"]: k["count"]
for k in sessions.annotate(date=TruncDate("start_time"))
chart_tooltip_format = "MMM d"
chart_granularity = "daily"
sessions_per_day = (
sessions.annotate(date=TruncDate("start_time"))
.values("date")
.annotate(count=models.Count("uuid"))
.order_by("date")
}
)
chart_data = {k["date"]: {"sessions": k["count"]} for k in sessions_per_day}
hits_per_day = (
hits.annotate(date=TruncDate("start_time"))
.values("date")
.annotate(count=models.Count("id"))
.order_by("date")
)
for k in hits_per_day:
if k["date"] not in chart_data:
chart_data[k["date"]] = {"hits": k["count"], "sessions": 0}
else:
chart_data[k["date"]]["hits"] = k["count"]
for day_offset in range((end_time - start_time).days + 1):
day = (start_time + timezone.timedelta(days=day_offset)).date()
if day not in session_chart_data:
session_chart_data[day] = 0 if day <= tz_now.date() else None
if day not in chart_data and day <= tz_now.date():
chart_data[day] = {"sessions": 0, "hits": 0}
return {
"currently_online": currently_online,
"session_count": session_count,
"hit_count": hit_count,
"has_hits": has_hits,
"avg_hits_per_session": hit_count / (max(session_count, 1)),
"bounce_rate_pct": bounce_count * 100 / session_count
if session_count > 0
else None,
"avg_session_duration": avg_session_duration,
"avg_load_time": avg_load_time,
"avg_hits_per_session": avg_hits_per_session,
"locations": locations,
"referrers": referrers,
"countries": countries,
"operating_systems": operating_systems,
"browsers": browsers,
"devices": devices,
"device_types": device_types,
"session_chart_data": json.dumps(
[
{"x": str(key), "y": value}
for key, value in sorted(
session_chart_data.items(), key=lambda k: k[0]
)
]
),
"session_chart_tooltip_format": session_chart_tooltip_format,
"session_chart_granularity": session_chart_granularity,
"online": True,
chart_data = sorted(chart_data.items(), key=lambda k: k[0])
chart_data = {
"sessions": [v["sessions"] for k, v in chart_data],
"hits": [v["hits"] for k, v in chart_data],
"labels": [str(k) for k, v in chart_data],
}
return chart_data, chart_tooltip_format, chart_granularity
def get_absolute_url(self):
return reverse(
"dashboard:service",

View File

@@ -0,0 +1 @@

View File

@@ -26,34 +26,34 @@ class DateRangeMixin:
now = timezone.now()
return [
{
'name': 'Last 3 days',
'start': now - timezone.timedelta(days=2),
'end': now,
"name": "Last 3 days",
"start": now - timezone.timedelta(days=2),
"end": now,
},
{
'name': 'Last 30 days',
'start': now - timezone.timedelta(days=29),
'end': now,
"name": "Last 30 days",
"start": now - timezone.timedelta(days=29),
"end": now,
},
{
'name': 'Last 90 days',
'start': now - timezone.timedelta(days=89),
'end': now,
"name": "Last 90 days",
"start": now - timezone.timedelta(days=89),
"end": now,
},
{
'name': 'This month',
'start': now.replace(day=1),
'end': now,
"name": "This month",
"start": now.replace(day=1),
"end": now,
},
{
'name': 'Last month',
'start': now.replace(day=1, month=now.month - 1),
'end': now.replace(day=1, month=now.month) - timezone.timedelta(days=1),
"name": "Last month",
"start": now.replace(day=1, month=now.month - 1),
"end": now.replace(day=1, month=now.month) - timezone.timedelta(days=1),
},
{
'name': 'This year',
'start': now.replace(day=1, month=1),
'end': now,
"name": "This year",
"start": now.replace(day=1, month=1),
"end": now,
},
]

View File

@@ -23,6 +23,22 @@
max-width: 0;
}
.min-w-48 {
min-width: 48px;
}
.geo-table {
display: none;
}
.geo-card--use-table-view .geo-map {
display: none;
}
.geo-card--use-table-view .geo-table {
display: inline-block;
}
:root {
--color-neutral-000: white;
--color-neutral-50: #F8FAFC;

View File

@@ -1,10 +1,10 @@
{% load account %}{% user_display user as user_display %}{% load i18n %}{% autoescape off %}{% blocktrans with site_name=current_site.name site_domain=current_site.domain %}Hi there,
{% load account %}{% user_display user as user_display %}{% load i18n %}{% autoescape off %}{% blocktrans with site_name=current_site.name site_domain=request.get_host %}Hi there,
You're receiving this email because {{ user_display }} has listed this email as a valid contact address for their account.
To confirm this is correct, go to {{ activate_url }}
{% endblocktrans %}
{% blocktrans with site_name=current_site.name site_domain=current_site.domain %}Thank you,
{% blocktrans with site_name=current_site.name site_domain=request.get_host %}Thank you,
{{ site_name }}
{% endblocktrans %}
{% endautoescape %}

View File

@@ -1,4 +1,4 @@
{% load i18n %}{% autoescape off %}{% blocktrans with site_name=current_site.name site_domain=current_site.domain %}Hi there,
{% load i18n %}{% autoescape off %}{% blocktrans with site_name=current_site.name site_domain=request.get_host %}Hi there,
You're receiving this email because you or someone else has requested a password for your account.
@@ -6,7 +6,7 @@ This message can be safely ignored if you did not request a password reset. Clic
{{ password_reset_url }}
{% blocktrans with site_name=current_site.name site_domain=current_site.domain %}Thank you,
{% blocktrans with site_name=current_site.name site_domain=request.get_host %}Thank you,
{{ site_name }}
{% endblocktrans %}
{% endautoescape %}

View File

@@ -12,11 +12,13 @@
<script src="{% static 'apexcharts/dist/apexcharts.min.js'%}"></script>
<script src="{% static 'litepicker/dist/nocss/litepicker.js' %}"></script>
<script src="{% static 'litepicker/dist/plugins/ranges.js' %}"></script>
<link rel="stylesheet" href="{% static 'litepicker/dist/css/litepicker.css' %}">
<script src="{% static 'd3/d3.min.js' %}"></script>
<script src="{% static 'topojson/build/topojson.min.js' %}"></script>
<script src="{% static 'datamaps/dist/datamaps.world.min.js' %}"></script>
<script src="{% static 'dashboard/js/base.js' %}"></script>
<link rel="stylesheet" href="{% static 'dashboard/css/global.css' %}">
<link rel="stylesheet" href="{% static 'flag-icon-css/css/flag-icon.min.css' %}">
<link rel="stylesheet" href="{% static 'litepicker/dist/css/litepicker.css' %}">
{% block extra_head %}
{% endblock %}
</head>

View File

@@ -0,0 +1,6 @@
{% load helpers %}
<div
class="absolute h-6 rounded-md"
style="width: {% bar_width count max total %}; top: 6px; left: 0px; height: calc(100% - 12px); background-color: var(--color-urge-100-fallback)"
>
</div>

View File

@@ -2,7 +2,7 @@
<input type="hidden" name="startDate" value="{{start_date.isoformat}}" id="startDate">
<input type="hidden" name="endDate" value="{{end_date.isoformat}}" id="endDate">
</form>
<input type="input" id="rangePicker" placeholder="Date range" class="input ~neutral bg-neutral-000 cursor-pointer w-auto" readonly>
<input type="input" id="rangePicker" placeholder="Date range" class="input ~neutral bg-neutral-000 cursor-pointer" style="max-width: 200px;" readonly>
<style>
:root {
--litepicker-button-prev-month-color-hover: var(--color-urge);

View File

@@ -0,0 +1,66 @@
{% load helpers %}
<div id="map-chart" class="relative"></div>
<script>
// Colors
const lightBlue = "#C4B5FD";
const highlightBlue = "#8B5CF6";
const white = "#ffffff";
// Data maps
const countryMapData = {};
const countryMapColors = {};
const countryMap = {
{% for country in countries %}"{{country.country|safe|datamap_id}}": {{country.count}},
{% endfor %}
};
// Max session count will be full opacity
const maxSessionCount = Math.max(...Object.values(countryMap));
// Color scale starts from opacity 0.1 - 1.0, 0 sessions gets opacity 0
const minPercentage = 0.1
// Loop over country map and transform data for Datamaps use
const keys = Object.keys(countryMap);
const length = keys.length;
for (let i = 0; i < length; i++) {
countryMapData[keys[i]] = {
sessionCount: countryMap[keys[i]],
color: `rgba(124, 58, 237, ${countryMap[keys[i]] === 0 ? 0 : minPercentage + (countryMap[keys[i]] / maxSessionCount * (1 - minPercentage))})`
};
countryMapColors[keys[i]] = countryMapData[keys[i]].color;
}
// Create datamap
var geoMap = new Datamap({
element: document.getElementById('map-chart'),
projection: 'mercator',
responsive: true,
geographyConfig: {
borderColor: lightBlue,
highlightBorderColor: highlightBlue,
highlightBorderWidth: 1.5,
highlightFillColor: (geography) => geography.color || white,
highlightFillOpacity: 0.9,
popupTemplate: (geography, data) => '<div class="hoverinfo"><strong>' + geography.properties.name + '</strong>: ' + data.sessionCount + ' sessions</div>'
},
fills: {
defaultFill: white
},
data: countryMapData,
aspectRatio: 0.68
});
geoMap.updateChoropleth(countryMapColors);
let debounceTimeout
const debounce = (func, debounce) => {
return function(event){
if(debounceTimeout) clearTimeout(debounceTimeout);
debounceTimeout = setTimeout(func,debounce,event);
};
}
window.addEventListener("resize",debounce(() => geoMap.resize(), 100))
</script>

View File

@@ -51,7 +51,7 @@
</div>
<hr class="sep h-4">
<div style="bottom: -1px;">
{% include 'dashboard/includes/time_chart.html' with data=stats.session_chart_data sparkline=True height=100 name=object.uuid tooltip_format=stats.session_chart_tooltip_format granularity=stats.session_chart_granularity %}
{% include 'dashboard/includes/time_chart.html' with data=stats.chart_data sparkline=True height=100 name=object.uuid tooltip_format=stats.chart_tooltip_format granularity=stats.chart_granularity %}
</div>
{% endwith %}
</a>
</a>

View File

@@ -1,5 +1,5 @@
<div class="card ~neutral !high font-mono text-sm whitespace-pre-wrap break-all">{% filter force_escape %}<noscript>
<img src="{{script_protocol}}{{request.site.domain}}{% url 'ingress:endpoint_pixel' object.uuid %}">
<img src="{{script_protocol}}{{request.get_host}}{% url 'ingress:endpoint_pixel' object.uuid %}">
</noscript>
<script defer src="{{script_protocol}}{{request.site.domain}}{% url 'ingress:endpoint_script' object.uuid %}"></script>{% endfilter %}
</div>
<script defer src="{{script_protocol}}{{request.get_host}}{% url 'ingress:endpoint_script' object.uuid %}"></script>{% endfilter %}
</div>

View File

@@ -5,12 +5,14 @@
enabled: false
},
tooltip: {
shared: false,
shared: true,
x: {
format: '{{tooltip_format|default:"MMM d"}}',
},
},
colors: ["#805AD5"],
legend: {
show: false,
},
chart: {
zoom: {
enabled: false,
@@ -18,7 +20,7 @@
toolbar: {
show: false,
},
type: 'area',
type: 'line',
height: {{height|default:"200"}},
offsetY: -1,
animations: {
@@ -27,20 +29,10 @@
sparkline: {
enabled: {% if sparkline %}true{% else %}false{% endif %},
},
fill: {
type: 'gradient',
gradient: {
shadeIntensity: 1,
inverseColors: false,
opacityFrom: 0.8,
opacityTo: 0,
stops: [0, 75, 100]
},
},
{% if granularity == "daily" and click_zoom %}
events: {
markerClick: function(event, chartContext, { seriesIndex, dataPointIndex, w: {config}}) {
const day = config.series[seriesIndex].data[dataPointIndex].x
const day = config.labels[dataPointIndex]
window.location.href = `?startDate=${day}&endDate=${day}`
},
},
@@ -79,13 +71,22 @@
},
},
stroke: {
width: 1.5,
width: 2,
curve: 'smooth',
},
series: [{
name: "{{unit|default:'Sessions'}}",
data: {{data|safe}}
}]
name: "Hits",
type: 'area',
color: "#ddd6fe",
data: {{data.hits|safe}}
}, {
name: "Sessions",
type: 'line',
color: "#805AD5",
data: {{data.sessions|safe}}
}],
labels: {{data.labels|safe}}
};
var triggerMatchesChart = new ApexCharts(document.querySelector("#chart{{name|default:'Main'}}"), triggerMatchesChartOptions);
triggerMatchesChart.render();
</script>
</script>

View File

@@ -94,10 +94,10 @@
{% endwith %}
</div>
<div class="card overflow-visible ~neutral !low py-0 mb-6">
{% include 'dashboard/includes/time_chart.html' with data=stats.session_chart_data tooltip_format=stats.session_chart_tooltip_format granularity=stats.session_chart_granularity click_zoom=True %}
{% include 'dashboard/includes/time_chart.html' with data=stats.chart_data tooltip_format=stats.chart_tooltip_format granularity=stats.chart_granularity click_zoom=True %}
</div>
{% endif %}
<div class="grid grid-cols-1 md:grid-cols-2 gap-6 mb-6">
<div id="card-grid" class="grid grid-cols-1 md:grid-cols-2 gap-6 mb-6">
<div class="card ~neutral !low limited-height py-2">
<table class="table">
<thead class="text-sm">
@@ -109,8 +109,68 @@
<tbody>
{% for location in stats.locations %}
<tr>
<td class="truncate w-full max-w-0">{{location.location|default:"Unknown"|urldisplay}}</td>
<td class="rf">{{location.count|intcomma}}</td>
<td class="truncate w-full max-w-0 relative">
{% include 'dashboard/includes/bar.html' with count=location.count max=stats.locations.0.count total=stats.hit_count %}
<div class="relative flex items-center">
{{location.location|default:"Unknown"|urldisplay}}
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{location.count|intcomma}}
<span class="text-xs rf min-w-48">
({{location.count|percent:stats.hit_count}})
</span>
</div>
</td>
</tr>
{% empty %}
<tr>
<td><span class="text-gray-600">No data yet...</span></td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="geo-map card ~neutral !low py-2 overflow-y-hidden">
<p class="text-sm font-semibold p-2 border-b mb-2" style="color: var(--color-title)">
Sessions by Geography &nbsp
<button onclick="document.getElementById('card-grid').classList.add('geo-card--use-table-view')" class="text-xs select-none p-0 button ~urge !low">
(view table)
</button>
</p>
{% include 'dashboard/includes/map_chart.html' with countries=stats.countries %}
</div>
<div class="geo-table card ~neutral !low limited-height py-2">
<table class="table">
<thead class="text-sm">
<tr>
<th>
Country &nbsp
<button onclick="document.getElementById('card-grid').classList.remove('geo-card--use-table-view'); geoMap.resize()" class="text-xs select-none p-0 button ~urge !low">
(view map)
</button>
</th>
<th class="rf">Sessions</th>
</tr>
</thead>
<tbody>
{% for country in stats.countries %}
<tr>
<td class="truncate w-full max-w-0 relative" title="{{country.country|country_name}}">
{% include 'dashboard/includes/bar.html' with count=country.count max=stats.countries.0.count total=stats.session_count %}
<div class="relative flex items-center">
<span class="flex-none {{country.country|flag_class}}"></span> <span class="truncate">{{country.country|country_name}}</span>
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{country.count|intcomma}}
<span class="text-xs rf min-w-48">
({{country.count|percent:stats.session_count}})
</span>
</div>
</td>
</tr>
{% empty %}
<tr>
@@ -131,32 +191,20 @@
<tbody>
{% for referrer in stats.referrers %}
<tr>
<td class="truncate w-full max-w-0">{{referrer.referrer|default:"Direct"|urldisplay}}</td>
<td class="rf">{{referrer.count|intcomma}}</td>
</tr>
{% empty %}
<tr>
<td><span class="text-gray-600">No data yet...</span></td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="card ~neutral !low limited-height py-2">
<table class="table">
<thead class="text-sm">
<tr>
<th>Country</th>
<th class="rf">Sessions</th>
</tr>
</thead>
<tbody>
{% for country in stats.countries %}
<tr>
<td class="truncate w-full max-w-0" title="{{country.country|country_name}}">
<span class="{{country.country|flag_class}}"></span> {{country.country|country_name}}
<td class="truncate w-full max-w-0 relative">
{% include 'dashboard/includes/bar.html' with count=referrer.count max=stats.referrers.0.count total=stats.session_count %}
<div class="relative flex items-center">
{{referrer.referrer|default:"Direct"|urldisplay}}
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{referrer.count|intcomma}}
<span class="text-xs rf min-w-48">
({{referrer.count|percent:stats.session_count}})
</span>
</div>
</td>
<td class="rf">{{country.count|intcomma}}</td>
</tr>
{% empty %}
<tr>
@@ -177,10 +225,20 @@
<tbody>
{% for os in stats.operating_systems %}
<tr>
<td class="flex items-center truncate w-full max-w-0" title="{{os.os|default:'Unknown'}}">
{{os.os|iconify}}<span>{{os.os|default:"Unknown"}}</span>
<td class="flex items-center truncate w-full max-w-0 relative" title="{{os.os|default:'Unknown'}}">
{% include 'dashboard/includes/bar.html' with count=os.count max=stats.operating_systems.0.count total=stats.session_count %}
<div class="relative flex items-center">
{{os.os|iconify}}<span class="truncate">{{os.os|default:"Unknown"}}</span>
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{os.count|intcomma}}
<span class="text-xs rf min-w-48">
({{os.count|percent:stats.session_count}})
</span>
</div>
</td>
<td class="rf">{{os.count|intcomma}}</td>
</tr>
{% empty %}
<tr>
@@ -201,9 +259,21 @@
<tbody>
{% for browser in stats.browsers %}
<tr>
<td class="flex items-center truncate w-full max-w-0" title="{{browser.browser|default:'Unknown'}}">
{{browser.browser|iconify}}<span>{{browser.browser|default:"Unknown"}}</span></td>
<td class="rf">{{browser.count|intcomma}}</td>
<td class="flex items-center truncate w-full max-w-0 relative" title="{{browser.browser|default:'Unknown'}}">
{% include 'dashboard/includes/bar.html' with count=browser.count max=stats.browsers.0.count total=stats.session_count %}
</div>
<div class="relative flex items-center">
{{browser.browser|iconify}}<span class="truncate">{{browser.browser|default:"Unknown"}}</span>
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{browser.count|intcomma}}
<span class="text-xs rf min-w-48">
({{browser.count|percent:stats.session_count}})
</span>
</div>
</td>
</tr>
{% empty %}
<tr>
@@ -224,8 +294,20 @@
<tbody>
{% for device_type in stats.device_types %}
<tr>
<td class="truncate w-full max-w-0">{{device_type.device_type|default:"Unknown"|title}}</td>
<td class="rf">{{device_type.count|intcomma}}</td>
<td class="truncate w-full max-w-0 relative">
{% include 'dashboard/includes/bar.html' with count=device_type.count max=stats.device_types.0.count total=stats.session_count %}
<div class="relative flex items-center">
<span class="truncate">{{device_type.device_type|default:"Unknown"|title}}</span>
</div>
</td>
<td>
<div class="flex justify-end items-center">
{{device_type.count|intcomma}}
<span class="text-xs rf min-w-48">
({{device_type.count|percent:stats.session_count}})
</span>
</div>
</td>
</tr>
{% empty %}
<tr>
@@ -236,11 +318,11 @@
</table>
</div>
</div>
<div class="card ~neutral !low limited-height py-2">
<div class="card ~neutral !low py-2 overflow-auto">
{% include 'dashboard/includes/session_list.html' %}
<hr class="sep h-8 md:h-12">
<a href="{% contextual_url 'dashboard:service_session_list' service.uuid %}" class="button ~neutral w-auto mb-2">View more
sessions
&rarr;</a>
</div>
{% endblock %}
{% endblock %}

View File

@@ -43,6 +43,14 @@ def country_name(isocode):
return "Unknown"
@register.filter
def datamap_id(isocode):
try:
return pycountry.countries.get(alpha_2=isocode).alpha_3
except:
return "UNKNOWN"
@register.simple_tag
def relative_stat_tone(
start,
@@ -181,11 +189,12 @@ def urldisplay(url):
if url.startswith("http"):
display_url = url.replace("http://", "").replace("https://", "")
return SafeString(
f"<a href='{url}' title='{url}' rel='nofollow' class='flex items-center mr-1'>{iconify(url)}<span class='truncate'>{escape(display_url)}</span></a>"
f"<a href='{url}' title='{url}' rel='nofollow' class='flex items-center mr-1 truncate'>{iconify(url)}<span class='truncate'>{escape(display_url)}</span></a>"
)
else:
return url
class ContextualURLNode(template.Node):
"""Extension of the Django URLNode to support including contextual parameters in URL outputs. In other words, URLs generated will keep the start and end date parameters."""
@@ -205,9 +214,13 @@ class ContextualURLNode(template.Node):
url_parts = list(urlparse(url))
query = dict(urllib.parse.parse_qsl(url_parts[4]))
query.update({
param: context.request.GET.get(param) for param in self.CONTEXT_PARAMS if param in context.request.GET and param not in query
})
query.update(
{
param: context.request.GET.get(param)
for param in self.CONTEXT_PARAMS
if param in context.request.GET and param not in query
}
)
url_parts[4] = urllib.parse.urlencode(query)
@@ -215,7 +228,7 @@ class ContextualURLNode(template.Node):
if self.urlnode.asvar:
context[self.urlnode.asvar] = url_final
return ''
return ""
else:
return url_final
@@ -224,7 +237,39 @@ class ContextualURLNode(template.Node):
def contextual_url(*args, **kwargs):
urlnode = url_tag(*args, **kwargs)
return ContextualURLNode(urlnode)
@register.filter
def location_url(session):
return settings.LOCATION_URL.replace("$LATITUDE", str(session.latitude)).replace("$LONGITUDE", str(session.longitude))
return settings.LOCATION_URL.replace("$LATITUDE", str(session.latitude)).replace(
"$LONGITUDE", str(session.longitude)
)
@register.filter
def percent(value, total):
if total == 0:
return "N/A"
percent = value / total
if percent < 0.001:
return "<0.1%"
return f"{percent:.1%}"
@register.simple_tag
def bar_width(count, max, total):
if total == 0 or max == 0:
return "0"
if settings.USE_RELATIVE_MAX_IN_BAR_VISUALIZATION:
percent = count / max
else:
percent = count / total
if percent < 0.001:
return "0"
return f"{percent:.1%}"

View File

View File

@@ -0,0 +1,43 @@
from django.test import TestCase, RequestFactory
from django.conf import settings
from django.urls import reverse
from core.factories import UserFactory
from dashboard.views import DashboardView
class QuestionModelTests(TestCase):
def setUp(self):
# Every test needs access to the request factory.
self.factory = RequestFactory()
self.user = UserFactory()
def tearDown(self):
pass
def tests_unauthenticated_dashboard_view(self):
"""
GIVEN: Unauthenticated user
WHEN: Accessing the dashboard view
THEN: It's redirected to login page with NEXT url to dashboard
"""
login_url = settings.LOGIN_URL
response = self.client.get(reverse("dashboard:dashboard"))
self.assertEqual(response.status_code, 302)
self.assertEqual(
response.url, f"{login_url}?next={reverse('dashboard:dashboard')}"
)
def tests_authenticated_dashboard_view(self):
"""
GIVEN: Authenticated user
WHEN: Accessing the dashboard view
THEN: It should respond with 200 and render the view
"""
request = self.factory.get(reverse("dashboard:dashboard"))
request.user = self.user
# Use this syntax for class-based views.
response = DashboardView.as_view()(request)
self.assertEqual(response.status_code, 200)

View File

@@ -9,6 +9,7 @@ https://docs.djangoproject.com/en/2.2/ref/settings/
"""
import os
from dotenv import load_dotenv
# import module sys to get the type of exception
import sys
@@ -17,8 +18,11 @@ import urllib.parse as urlparse
# Messages
from django.contrib.messages import constants as messages
# Load environment variables
load_dotenv()
# Increment on new releases
VERSION = "v0.9.1"
VERSION = "0.11.0"
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -294,6 +298,9 @@ else:
EMAIL_USE_SSL = os.environ.get("EMAIL_USE_SSL")
EMAIL_USE_TLS = os.environ.get("EMAIL_USE_TLS")
# Auto fields
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
# NPM
NPM_ROOT_PATH = "../"
@@ -310,7 +317,13 @@ NPM_FILE_PATTERNS = {
"stimulus": [os.path.join("dist", "stimulus.umd.js")],
"inter-ui": [os.path.join("Inter (web)", "*")],
"@fortawesome": [os.path.join("fontawesome-free", "js", "all.min.js")],
"flag-icon-css": [os.path.join("css", "flag-icon.min.css"), os.path.join("flags", "*")],
"datamaps": [os.path.join("dist", "datamaps.world.min.js")],
"d3": ["d3.min.js"],
"topojson": [os.path.join("build", "topojson.min.js")],
"flag-icon-css": [
os.path.join("css", "flag-icon.min.css"),
os.path.join("flags", "*"),
],
}
# Shynet
@@ -346,7 +359,14 @@ BLOCK_ALL_IPS = os.getenv("BLOCK_ALL_IPS", "False") == "True"
AGGRESSIVE_HASH_SALTING = os.getenv("AGGRESSIVE_HASH_SALTING", "False") == "True"
# What location url should be linked to in the frontend?
LOCATION_URL = os.getenv("LOCATION_URL", "https://www.openstreetmap.org/?mlat=$LATITUDE&mlon=$LONGITUDE")
LOCATION_URL = os.getenv(
"LOCATION_URL", "https://www.openstreetmap.org/?mlat=$LATITUDE&mlon=$LONGITUDE"
)
# How many services should be displayed on dashboard page?
DASHBOARD_PAGE_SIZE = int(os.getenv("DASHBOARD_PAGE_SIZE", "5"))
# Should background bars be scaled to full width?
USE_RELATIVE_MAX_IN_BAR_VISUALIZATION = (
os.getenv("USE_RELATIVE_MAX_IN_BAR_VISUALIZATION", "True") == "True"
)

View File

@@ -16,9 +16,6 @@ if [[ ${startup_results[1]} == True ]]; then
echo "Warning: no admin user available. Consult docs for instructions."
fi
if [[ ${startup_results[2]} == True ]]; then
echo "Warning: Shynet's hostname is not set. The script won't work correctly. Consult docs for instructions."
fi
if [[ ${startup_results[3]} == True ]]; then
echo "Warning: Shynet's whitelabel is not set. Consult docs for instructions."
fi
echo "Startup checks complete!"

View File

@@ -1,16 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<title>JS test</title>
</head>
<body>
<noscript>
<img src="http://localhost:8000/ingress/0ca733e8-c41f-462b-a11a-4ba0cea29948/pixel.gif">
</noscript>
<script defer src="http://localhost:8000/ingress/0ca733e8-c41f-462b-a11a-4ba0cea29948/script.js"></script>
</body>
</html>

View File

@@ -1,13 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<title>Pixel test</title>
</head>
<body>
<img src="http://localhost:8000/ingress/9b2c4e2f-8d29-4418-82d4-b68e06795025/pixel.gif">
</body>
</html>