Compare commits

..

113 Commits

Author SHA1 Message Date
R. Miles McCain
c2234ec647 Bump version 2021-03-28 22:15:30 +00:00
R. Miles McCain
02cbee5c8c Cache bounce 2021-03-28 21:55:38 +00:00
R. Miles McCain
518436ffd2 Relock npm packages 2021-03-28 21:37:09 +00:00
R. Miles McCain
311aa2b1ac Drop Turbolinks 2021-03-28 21:36:53 +00:00
R. Miles McCain
8ad44ddc23 Add pagination to dashboard 2021-03-28 21:29:54 +00:00
R. Miles McCain
874aad87a8 Store service directly in Hit 2021-03-28 20:54:19 +00:00
R. Miles McCain
f2e875d03d Add indexes to key Hit fields 2021-03-28 19:18:57 +00:00
R. Miles McCain
45fd32c8ca Index last_seen 2021-03-28 19:15:03 +00:00
R. Miles McCain
08b36ba69f Integrate debug toolbar 2021-03-28 19:14:56 +00:00
R. Miles McCain
d5cfe577a0 Add debug toolbar 2021-03-28 19:14:33 +00:00
R. Miles McCain
c131cfef27 Merge branch 'patch-1' into dev 2021-03-28 18:54:24 +00:00
R. Miles McCain
526d4cd133 Relock Pipfile 2021-03-28 18:53:49 +00:00
R. Miles McCain
8e09871b44 Merge branch 'dependabot/pip/django-3.1.6' into dev 2021-03-28 18:51:33 +00:00
R. Miles McCain
6aa3ce0b32 Merge branch 'dependabot/pip/pyyaml-5.4' into dev 2021-03-28 18:49:49 +00:00
dependabot[bot]
23ea8e493e Bump pyyaml from 5.3.1 to 5.4
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 5.3.1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/compare/5.3.1...5.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-25 23:41:11 +00:00
dependabot[bot]
22d996bed7 Bump django from 3.1.3 to 3.1.6
Bumps [django](https://github.com/django/django) from 3.1.3 to 3.1.6.
- [Release notes](https://github.com/django/django/releases)
- [Commits](https://github.com/django/django/compare/3.1.3...3.1.6)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-19 02:04:53 +00:00
Kasper Seweryn
9df864787c Fix #99 2021-02-17 02:24:35 +00:00
dependabot[bot]
b7a6ac9ec0 Bump apexcharts from 3.23.1 to 3.24.0 (#97)
Bumps [apexcharts](https://github.com/apexcharts/apexcharts.js) from 3.23.1 to 3.24.0.
- [Release notes](https://github.com/apexcharts/apexcharts.js/releases)
- [Commits](https://github.com/apexcharts/apexcharts.js/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-02-15 18:56:29 -05:00
R. Miles McCain
38d8d416e1 Update kubernetes deployments 2021-01-25 13:39:53 -05:00
R. Miles McCain
592613a99a Use dev shynet in kubernetes deployments 2021-01-23 23:28:24 -05:00
R. Miles McCain
e9f43c6a53 Bump version 2021-01-23 23:23:19 -05:00
R. Miles McCain
89c6800913 Fix formatting 2021-01-23 23:16:33 -05:00
R. Miles McCain
db9c807289 Add optional more aggressive salting (fixes #95) 2021-01-23 23:13:44 -05:00
R. Miles McCain
6e48a3eac7 Merge branch 'global-ip-block' into dev 2021-01-23 23:01:53 -05:00
R. Miles McCain
ba9a716913 Merge branch 'heartbeat-frequency' into dev 2021-01-23 22:41:25 -05:00
R. Miles McCain
6d7292a60a Fix duration change being unknown (fixes #89) 2021-01-23 22:40:19 -05:00
Oliver Kamer
c0d02732e7 Add additional env variable to template 2021-01-19 22:05:33 +01:00
Oliver Kamer
d071a91917 Block Collect IP option if disabled globally 2021-01-19 22:02:57 +01:00
Oliver Kamer
d67e14b08f Block IP collection from settings 2021-01-19 21:41:54 +01:00
Oliver Kamer
174a386f54 Add block all ips to settings 2021-01-19 21:31:02 +01:00
Oliver Kamer
ce23cfc5b5 Add pycharm gitignore stuff 2021-01-19 21:20:30 +01:00
Oliver Kamer
8be690c417 Use heartbeat frequency for currently active
If the heartbeat frequency is more than 10 seconds, shynet will display as not active, even though it still is.

Using 2x the heartbeat frequency should give better results.
2021-01-19 11:32:36 +01:00
R. Miles McCain
2f778dc4b4 Bump version to v0.7.3 2021-01-11 12:12:15 -05:00
R. Miles McCain
e0c165313b Add fallback to percent_change_display (fixes #89) 2021-01-11 12:11:27 -05:00
R. Miles McCain
c86192d301 Improve a17t colors 2021-01-10 12:25:08 -05:00
R. Miles McCain
775c105d1d Bump version to 0.7.2 2021-01-10 12:20:17 -05:00
R. Miles McCain
be85c0a560 Update and trim dependencies 2021-01-10 12:19:54 -05:00
R. Miles McCain
70e1af15cc Fix division by zero error 2021-01-10 12:17:53 -05:00
R. Miles McCain
6afea91c5f Bump version 2020-11-26 21:03:52 +00:00
R. Miles McCain
7a4c892804 Remove background from favicon 2020-11-26 21:03:18 +00:00
imgbot[bot]
9b50b1ea42 [ImgBot] Optimize images (#87)
*Total -- 765.10kb -> 502.59kb (34.31%)

/images/service.png -- 366.95kb -> 239.64kb (34.69%)
/images/homepage.png -- 398.15kb -> 262.95kb (33.96%)

Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com>

Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com>
2020-11-26 16:00:51 -05:00
R. Miles McCain
52a18d21f1 Small visual improvements 2020-11-26 20:59:03 +00:00
R. Miles McCain
8aaf312c67 Bump version 2020-11-26 20:10:28 +00:00
R. Miles McCain
ede06900e5 Format code 2020-11-26 20:09:55 +00:00
R. Miles McCain
a42455c9dc Add browser icon 2020-11-26 20:09:34 +00:00
R. Miles McCain
547a84f2fc Update screenshots 2020-11-26 20:04:48 +00:00
R. Miles McCain
f56ea99dc2 Add demo data command 2020-11-26 19:44:07 +00:00
R. Miles McCain
4cea5d2310 Add additional icons 2020-11-26 19:43:44 +00:00
R. Miles McCain
e4f09b4e68 Ensure times are always correct 2020-11-26 19:43:25 +00:00
R. Miles McCain
cc094fe04e Add icons to dashboard 2020-11-26 18:09:42 +00:00
R. Miles McCain
ac5c743390 Update dependencies 2020-11-26 18:00:19 +00:00
R. Miles McCain
963db18642 Bump version 2020-10-17 19:11:25 +00:00
R. Miles McCain
748fb76eaf Bump version 2020-10-17 19:03:37 +00:00
R. Miles McCain
d93a698e87 Update django-redis-cache 2020-10-17 19:03:18 +00:00
R. Miles McCain
ca9ee2f1f5 Bump version 2020-10-17 18:54:22 +00:00
R. Miles McCain
9146c889ac Show Shynet version when envvar is true 2020-10-17 18:53:46 +00:00
R. Miles McCain
8b1034ebb0 Fix prerelease usage 2020-10-17 18:34:09 +00:00
R. Miles McCain
a8fd263855 Bump version 2020-10-14 18:00:35 +00:00
R. Miles McCain
31fa3d55d5 Update dependencies 2020-10-14 18:00:31 +00:00
Sudipto Ghosh
13229f64aa removed unused import 2020-08-31 18:27:55 +05:30
Sudipto Ghosh
101d26d356 added SHOW_SHYNET_VERSION env var to control display of version info 2020-08-31 18:08:00 +05:30
R. Miles McCain
c524325f0a Bump version to v0.6.5 2020-08-28 17:20:16 +00:00
R. Miles McCain
4a07ab80ce Prevent multiple emails from pointing to same collaborator (fixes #78) 2020-08-28 17:19:57 +00:00
R. Miles McCain
c8dead4457 Prevent services from showing up twice on homepage 2020-08-28 17:19:07 +00:00
R. Miles McCain
4a06357137 Bump version 2020-08-19 23:08:45 +00:00
Nicholas Bentley
29ac82a91b smtp ssl/tls fix 2020-08-18 23:34:20 -04:00
R. Miles McCain
fecea17a9d Bump version 2020-08-18 15:42:17 +00:00
R. Miles McCain
03062e3de5 Fix session detail page for collaborators (fixes #74) 2020-08-18 15:41:50 +00:00
R. Miles McCain
6652acdf14 Bump version 2020-08-11 21:56:59 +00:00
R. Miles McCain
1dfbec06e1 Split testing options 2020-08-11 21:56:26 +00:00
R. Miles McCain
3e315f06ed Enforce origin checking on pixel trackers (indirectly fixes #65) 2020-08-11 21:56:20 +00:00
R. Miles McCain
2d42674e1a Add warning when hostname starts with http (fixes #68) 2020-08-11 21:39:08 +00:00
R. Miles McCain
e4deab2072 Fix file path creation (fixes #69) 2020-08-11 21:34:39 +00:00
R. Miles McCain
c5ed5ef0e7 Merge branch 'MagnumDingusEdu/master' into dev 2020-08-11 21:32:03 +00:00
R. Miles McCain
7268a4ea84 Improve GUIDE language 2020-08-11 21:31:39 +00:00
Vividh Mariy
2cbc5ac441 Added deployment using docker-compose. Fixed #70 2020-08-10 00:00:52 +05:30
R. Miles McCain
058601d669 Fix button styling on session page (fixes #63) 2020-07-31 16:32:15 +00:00
Jake Malachowski
213c44a45a Add Render as a deployment option (#62)
* Add Render deployment option

Add Render as deployment option

* Remove Render feature descriptions
2020-07-21 11:45:35 -04:00
R. Miles McCain
8b98cf2277 Update pixel cache control 2020-07-11 17:26:53 +00:00
R. Miles McCain
4c53b94588 Add SPA section to guide TOC 2020-07-07 03:25:31 +00:00
R. Miles McCain
a70e07be05 Finish transition to startup checks 2020-07-07 03:15:07 +00:00
R. Miles McCain
0195c4595b Document SPA behavior 2020-07-07 03:01:48 +00:00
R. Miles McCain
a54d9e6840 Bump version 2020-07-07 02:45:11 +00:00
R. Miles McCain
a4245eb733 Update dependencies 2020-07-07 02:45:03 +00:00
R. Miles McCain
7e0584b5d2 Fix button styling 2020-07-07 02:44:38 +00:00
R. Miles McCain
37396cde63 Improve service form 2020-07-07 02:23:48 +00:00
R. Miles McCain
a1e4bef08f Use a17t v0.2.2 2020-07-07 02:23:09 +00:00
R. Miles McCain
c3510278e3 Improve origin explanation language 2020-07-07 01:41:01 +00:00
R. Miles McCain
da61b9b400 Document primary key integration (fixes #56) 2020-07-07 01:38:16 +00:00
R. Miles McCain
98187a39f8 Document health check endpoint (fixes #59) 2020-07-07 01:27:08 +00:00
R. Miles McCain
3d27efba8b Check IP versions before comparing (fixes #57) 2020-07-07 00:22:29 +00:00
R. Miles McCain
80c66ceb8e Remove unnecessary ipaddress dependency 2020-07-07 00:18:33 +00:00
R. Miles McCain
a2776e64f6 Rename from "sanity results" to "startup results" 2020-07-07 00:18:24 +00:00
imgbot[bot]
c73f96525a [ImgBot] Optimize images (#55)
*Total -- 911.50kb -> 583.94kb (35.94%)

/images/slogo.png -- 2.51kb -> 0.91kb (63.77%)
/images/service.png -- 589.15kb -> 359.70kb (38.95%)
/images/homepage.png -- 307.99kb -> 214.75kb (30.28%)
/images/logo.png -- 11.85kb -> 8.58kb (27.54%)

Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com>

Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com>
2020-06-28 15:11:48 -04:00
R. Miles McCain
510df192d8 Add script injection option 2020-06-28 18:59:05 +00:00
R. Miles McCain
2e7620f1eb Bump version 2020-06-28 17:56:29 +00:00
R. Miles McCain
93d4ee5241 Use specific version of pipenv in dockerfile 2020-06-28 17:56:09 +00:00
R. Miles McCain
1a7594be93 Use SHA256 for more secure session association 2020-06-28 17:55:59 +00:00
R. Miles McCain
f464a7ee67 Add automatic release drafter 2020-06-28 17:47:58 +00:00
R. Miles McCain
a1cd3d4609 Code cleanup 2020-06-28 17:36:20 +00:00
R. Miles McCain
358fb234a7 Fix multiple origin support (closes #52) 2020-06-28 17:36:12 +00:00
R. Miles McCain
94fed58de3 Fix charts 2020-06-28 17:18:58 +00:00
R. Miles McCain
49f452d9f2 Lock Python package versions 2020-06-28 17:11:08 +00:00
R. Miles McCain
40d07fe159 Add IE11 support 2020-06-28 04:10:06 +00:00
R. Miles McCain
e150e6bede Bump version 2020-06-28 04:01:50 +00:00
R. Miles McCain
87a411f42d Make ingress processing more resilient 2020-06-28 03:48:40 +00:00
R. Miles McCain
88f25b6743 Analytics script cleanup (closes #54) 2020-06-28 03:31:10 +00:00
R. Miles McCain
bb0dc2e90f Remove all external dependencies 2020-06-28 02:58:49 +00:00
R. Miles McCain
4a8939796e Arbitrarily update the pixel test 2020-06-28 02:45:46 +00:00
R. Miles McCain
ba795ccd5c Clarify collaborators description 2020-06-28 02:44:59 +00:00
Alexandre Bulté
c9b5a677d3 SIGNUPS_ENABLED -> ACCOUNT_SIGNUPS_ENABLED in docs (#51) 2020-06-18 15:15:10 -04:00
Alexandre Bulté
affcb893fa Add timeout to curl / geoip (#50)
W/o timeout, if the download is unresponsive it can be a pain on some environments (eg `dokku`) because the build hangs forever.
2020-06-18 14:00:06 -04:00
R. Miles McCain
e030807acb Fix GeoIP2 license keys 2020-06-18 16:46:39 +00:00
62 changed files with 1229 additions and 423 deletions

4
.github/release-drafter.yml vendored Normal file
View File

@@ -0,0 +1,4 @@
template: |
## Whats Changed
$CHANGES

14
.github/workflows/draft.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
name: Release Drafter
on:
push:
branches:
- master
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v5.11.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

8
.gitignore vendored
View File

@@ -3,6 +3,9 @@ __pycache__/
*.py[cod]
*$py.class
# JavaScript packages
node_modules/
# C extensions
*.so
@@ -135,4 +138,7 @@ dmypy.json
secrets.yml
.vscode
.DS_Store
compiledstatic/
compiledstatic/
# Pycharm
.idea

View File

@@ -3,20 +3,22 @@ FROM python:3-alpine
# Getting things ready
WORKDIR /usr/src/shynet
COPY Pipfile.lock Pipfile ./
COPY package.json package-lock.json ../
# Django expects node_modules to be in its parent directory.
# Install dependencies & configure machine
ARG GF_UID="500"
ARG GF_GID="500"
RUN apk update && \
apk add gettext curl bash && \
# URL from https://github.com/shlinkio/shlink/issues/596 :)
curl "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=G4Lm0C60yJsnkdPi&suffix=tar.gz" | tar -xvz -C /tmp && \
curl "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=G4Lm0C60yJsnkdPi&suffix=tar.gz" | tar -xvz -C /tmp && \
apk add gettext curl bash npm && \
curl -m 180 "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=kKG1ebhL3iWVd0iv&suffix=tar.gz" | tar -xvz -C /tmp && \
curl -m 180 "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=kKG1ebhL3iWVd0iv&suffix=tar.gz" | tar -xvz -C /tmp && \
mv /tmp/GeoLite2*/*.mmdb /etc && \
apk del curl && \
apk add --no-cache postgresql-libs && \
apk add --no-cache --virtual .build-deps gcc musl-dev postgresql-dev && \
pip install pipenv && \
npm i -P --prefix .. && \
pip install pipenv~=2020.6.2 && \
pipenv install --system --deploy && \
apk --purge del .build-deps && \
rm -rf /var/lib/apt/lists/* && \

View File

@@ -4,21 +4,26 @@
- [Installation](#installation)
- [Heroku](#heroku)
- [Render](#render)
- [Updating Your Configuration](#updating-your-configuration)
- [Enhancements](#enhancements)
- [Advanced Usage](#advanced-usage)
* [Installation with SSL](#installation-with-ssl)
* [Configuring a Reverse Proxy](#configuring-a-reverse-proxy)
+ [Cloudflare](#cloudflare)
+ [Nginx](#nginx)
* [Health Checks](#health-checks)
* [Primary Key Integration](#primary-key-integration)
* [Usage with Single-Page Applications](#usage-with-single-page-applications)
+ [Troubleshooting](#troubleshooting)
---
## Staying Updated
**If you install Shynet, you should strongly consider enabling notifications when new versions are released.** You can do this under the "Watch" tab on GitHub (above). This will ensure that you are notified when new versions are available, some of which may be security updates. (Shynet will never automatically update itself.)
## Installation
Installation of Shynet is easy! Follow the [Basic Installation](#basic-installation) guide below if you'd like to run Shynet over HTTP or if you are going to be running it over HTTPS through a reverse proxy. If you'd like to run Shynet over HTTPS without a reverse proxy, skip ahead to [Installation with SSL](#installation-with-ssl) instead.
Installation of Shynet is easy! Follow the [Basic Installation](#basic-installation) guide or the [Basic Installation with Docker Compose](#basic-installation-with-docker-compose) below for a minimal installation, or if you are going to be running Shynet over HTTPS through a reverse proxy. If you'd like to run Shynet over HTTPS without a reverse proxy, skip ahead to [Installation with SSL](#installation-with-ssl) instead.
> **These commands assume Ubuntu.** If you're installing Shynet on a different platform, the process will be different.
@@ -36,7 +41,7 @@ Before continuing, please be sure to have the latest version of Docker installed
5. Create an admin user by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py registeradmin <your email>`. A temporary password will be printed to the console.
6. Set the hostname of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py hostname <your public hostname>`, where `<your public hostname>` is the _publicly accessible hostname_ of your instance, including port. This setting affects the URL that the tracking script sends its results to, so make sure it's correct. (Example hostnames: `shynet.rmrm.io` or `example.com:8000`.)
6. Set the hostname of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py hostname <your public hostname>`, where `<your public hostname>` is the _publicly accessible hostname_ of your instance, including port. This setting affects the URL that the tracking script sends its results to, so make sure it's correct. (Example hostnames: `shynet.example.com` or `example.com:8000`.)
7. Set the whitelabel of your Shynet instance by running `docker run --env-file=<your env file> milesmcc/shynet:latest ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: `"My Shynet Instance"` or `"Acme Analytics"`.)
@@ -46,6 +51,27 @@ Before continuing, please be sure to have the latest version of Docker installed
10. Finally, click on "Manage" in the top right of the service's page to get the tracking script code. Inject this script on all pages you'd like the service to track.
### Basic Installation with Docker Compose
> Make sure you have `docker-compose` installed. If not, [install it](https://docs.docker.com/compose/install/)
1. Clone the repository.
2. Using [TEMPLATE.env](/TEMPLATE.env) as a template, confiure the environment for your Shynet instance and place the modified config in a file called `.env` in the root of the repository. Do _not_ change the port number at the end; you can set the public facing port in the next step.
3. On line 2 of the `nginx.conf` file located in the root of the repository, replace `example.com` with your hostname. Then, in the `docker-compose.yml` file, set the port number by replacing `8080` in line 38 ( `- 8080:80` ) with whatever local port you want to bind it to. For example, set the port number to `- 80:80` if you want your site will be available via HTTP (port 80) at `http://<your hostname>`.
4. Launch the Shynet server for the first time by running `docker-compose up -d`. If you get an error like "permission denied" or "Couldn't connect to Docker daemon", either prefix the command with `sudo` or add your user to the `docker` group.
5. Create an admin user by running `docker exec -it shynet_main ./manage.py registeradmin <your email>`. A temporary password will be printed to the console.
6. Set the hostname of your Shynet instance by running `docker exec -it shynet_main ./manage.py hostname <your public hostname>`, where `<your public hostname>` is the same as the hostname you set in step 3. This setting affects the URL that the tracking script sends its results to, so make sure it's correct. (Example hostnames: shynet.example.com or example.com:8000.)
7. Set the whitelabel of your Shynet instance by running `docker exec -it shynet_main ./manage.py whitelabel <whitelabel>`. While this setting doesn't affect any core operations of Shynet, it lets you rename Shynet to whatever you want. (Example whitelabels: "My Shynet Instance" or "Acme Analytics".)
Your site should now be accessible at `http://hostname:port`. Now you can follow steps 9-10 of the [Basic Installation](#basic-installation) guide above to get Shynet integrated on your sites.
## Heroku
You may wish to deploy Shynet on Heroku. Note that Heroku's free offerings (namely the free Postgres addon) are unlikely to support running any Shynet instance that records more than a few hundred requests per day &mdash; the database will quickly fill up. In most cases, the more cost-effective option for running Shynet is renting a VPS from a full cloud service provider. However, if you're sure Heroku is the right option for you, or you just want to try Shynet out, you can use the Quick Deploy button then follow the steps below.
@@ -58,9 +84,23 @@ Once you deploy, you'll need to setup an admin user, whitelabel, and hostname be
2. `heroku run --app=<your app> ./manage.py hostname <the hostname where you will run Shynet>`
3. `heroku run --app=<your app> ./manage.py whitelabel "<your Shynet instance's name>"`
## Render
[Render](https://render.com) is a modern cloud platform to build and run all your apps and websites with free SSL, a global CDN, private networks and auto deploys from Git. To deploy Shynet, click the `Deploy to Render` button and follow the steps below.
[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/render-examples/shynet)
Once your deploy has completed, use the **Render Shell** to configure your app:
1. Set your email: `./manage.py registeradmin your-email@example.com`
1. Add your onrender.com domain: `./manage.py hostname your-shynet-domain.onrender.com`
1. Set your whitelabel: `./manage.py whitelabel "Your Shynet Instance Name"`
See the [Render docs](https://render.com/docs/deploy-shynet) for more information on deploying your application on Render.
---
## Enhancements
## Advanced Usage
### Installation with SSL
@@ -175,6 +215,40 @@ Nginx is a self hosted, highly configurable webserver. Nginx can be configured t
* [How to add SSL/HTTPS to Nginx (Ubuntu 16.04)](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04)
* [Nginx Documentation](https://nginx.org/en/docs/)
### Health Checks
By default, Shynet includes a default health check endpoint at `/healthz/`. If the instance is running normally, this endpoint will return an HTTP status code of 200; if something is wrong, it will have a non-200 status code. To view the health data as JSON, send your request to `/healthz/?format=json`.
This feature is helpful when running Shynet with Kubernetes, as it allows you to setup [startup readiness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) that prevent traffic from being sent to your Shynet instances before they are ready.
### Primary-Key Integration
In some cases, it is useful to associate particular users on your platform with their sessions in Shynet. In Shynet, this is called _primary key integration_, and is done by adding an additional element to the Shynet script url for each particular user.
If the Shynet script location (for either the pixel or the script) is, for example, `//shynet.example.com/ingress/your_service_uuid/pixel.gif` and `//shynet.example.com/ingress/your_service_uuid/script.js`, the URLs for primary-key enabled users would be `//shynet.example.com/ingress/your_service_uuid/USER_PRIMARY_KEY/pixel.gif` and `//shynet.example.com/ingress/your_service_uuid/USER_PRIMARY_KEY/script.js`.
Adding this path can be done easily using server-side rendering. For example, here is a Django template that adds users' primary keys to the Shynet tracking script:
```html
{% if request.user.is_authenticated %}
<noscript>
<img src="//shynet.example.com/ingress/service-uuid/{{request.user.email|urlencode:""}}/pixel.gif">
</noscript>
<script src="//shynet.example.com/ingress/service-uuid/{{request.user.email|urlencode:""}}/script.js"></script>
{% else %}
<noscript>
<img src="//shynet.example.com/ingress/service-uuid/pixel.gif">
</noscript>
<script src="//shynet.example.com/ingress/service-uuid/script.js"></script>
{% endif %}
```
### Usage with Single-Page Applications
In a single-page application, the page never reloads. (That's the entire point of single-page applications, after all!) Unfortunately, this also means that Shynet will not automatically recognize and track when the user navigates between pages _within_ your application.
Fortunately, Shynet offers a simple method you can call from anywhere within your JavaScript to indicate that a new page has been loaded: `Shynet.newPageLoad()`. Add this method call to the code that handles routing in your app, and you'll be ready to go.
---
## Troubleshooting

45
Pipfile
View File

@@ -3,29 +3,24 @@ name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
black = "*"
[packages]
django = "*"
django-allauth = "*"
geoip2 = "*"
whitenoise = "*"
celery = "*"
django-ipware = "*"
pyyaml = "*"
ua-parser = "*"
user-agents = "*"
emoji-country-flag = "*"
rules = "*"
gunicorn = "*"
psycopg2-binary = "*"
redis = "*"
django-redis-cache = "*"
pycountry = "*"
ipaddress = "*"
html2text = "*"
django-health-check = "*"
[pipenv]
allow_prereleases = true
django = "~=3.1"
django-allauth = "~=0.42.0"
geoip2 = "~=3.0.0"
whitenoise = "~=5.1.0"
celery = "~=4.4.6"
django-ipware = "~=2.1.0"
pyyaml = "~=5.4"
ua-parser = "~=0.10.0"
user-agents = "~=2.1"
emoji-country-flag = "~=1.2.1"
rules = "~=2.2"
gunicorn = "~=20.0.4"
psycopg2-binary = "~=2.8.5"
redis = "~=3.5.3"
django-redis-cache = "~=3.0.0"
pycountry = "~=19.8.18"
html2text = "~=2020.1.16"
django-health-check = "~=3.12.1"
django-npm = "~=1.0.0"
django-debug-toolbar = "*"

375
Pipfile.lock generated
View File

@@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
"sha256": "327b897f359bad486c08fc88fb70a1f9d2edaf1aadafcb1d31e5b3e144125ff7"
"sha256": "f8c76565a776f1bd36364077a86d6c16fccc522d9d2024bb9b51be5cb9f8b4b5"
},
"pipfile-spec": 6,
"requires": {},
@@ -16,17 +16,19 @@
"default": {
"amqp": {
"hashes": [
"sha256:24dbaff8ce4f30566bb88976b398e8c4e77637171af3af6f1b9650f48890e60b",
"sha256:bb68f8d2bced8f93ccfd07d96c689b716b3227720add971be980accfc2952139"
"sha256:70cdb10628468ff14e57ec2f751c7aa9e48e7e3651cfd62d431213c0c4e58f21",
"sha256:aa7f313fb887c91f15474c1229907a04dac0b8135822d6603437803424c0aa59"
],
"version": "==2.6.0"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==2.6.1"
},
"asgiref": {
"hashes": [
"sha256:8036f90603c54e93521e5777b2b9a39ba1bad05773fcf2d208f0299d1df58ce5",
"sha256:9ca8b952a0a9afa61d30aa6d3d9b570bb3fd6bafcf7ec9e6bed43b936133db1c"
"sha256:5ee950735509d04eb673bd7f7120f8fa1c9e2df495394992c73234d526907e17",
"sha256:7162a3cb30ab0609f1a4c95938fd73e8604f63bdba516a7f7d64b83ff09478f0"
],
"version": "==3.2.7"
"markers": "python_version >= '3.5'",
"version": "==3.3.1"
},
"billiard": {
"hashes": [
@@ -37,40 +39,42 @@
},
"celery": {
"hashes": [
"sha256:c3f4173f83ceb5a5c986c5fdaefb9456de3b0729a72a5776e46bd405fda7b647",
"sha256:d1762d6065522879f341c3d67c2b9fe4615eb79756d59acb1434601d4aca474b"
"sha256:a92e1d56e650781fb747032a3997d16236d037c8199eacd5217d1a72893bca45",
"sha256:d220b13a8ed57c78149acf82c006785356071844afe0b27012a4991d44026f9f"
],
"index": "pypi",
"version": "==4.4.5"
"version": "==4.4.7"
},
"certifi": {
"hashes": [
"sha256:5ad7e9a056d25ffa5082862e36f119f7f7cec6457fa07ee2f8c339814b80c9b1",
"sha256:9cd41137dc19af6a5e03b630eefe7d1f458d964d406342dd3edf625839b944cc"
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.4.5.2"
"version": "==2020.12.5"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
],
"version": "==3.0.4"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.0.0"
},
"defusedxml": {
"hashes": [
"sha256:8ede8ba04cf5bf7999e1492fa77df545db83717f52c5eab625f97228ebd539bf",
"sha256:aa621655d72cdd30f57073893b96cd0c3831a85b08b8e4954531bdac47e3e8c8"
"sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69",
"sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61"
],
"version": "==0.7.0rc1"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==0.7.1"
},
"django": {
"hashes": [
"sha256:db4c9b29615d17f808f2b1914d5cd73cd457c9fd90581195172c0888c210d944",
"sha256:dd96f98ec1c3e60877d45cea7350215f16de409848d23cced8443db1b188bd9b"
"sha256:32ce792ee9b6a0cbbec340123e229ac9f765dff8c2a4ae9247a14b2ba3a365a7",
"sha256:baf099db36ad31f970775d0be5587cc58a6256a6771a44eb795b554d45f211b8"
],
"index": "pypi",
"version": "==3.1a1"
"version": "==3.1.7"
},
"django-allauth": {
"hashes": [
@@ -79,13 +83,21 @@
"index": "pypi",
"version": "==0.42.0"
},
"django-health-check": {
"django-debug-toolbar": {
"hashes": [
"sha256:0563827e003d25fd4d9ebbd7467dea5f390435628d645aaa63f8889deaded73a",
"sha256:9e6b7d93d4902901474efd4e25d31b5aaea7563b570c0260adce52cd3c3a9e36"
"sha256:84e2607d900dbd571df0a2acf380b47c088efb787dce9805aefeb407341961d2",
"sha256:9e5a25d0c965f7e686f6a8ba23613ca9ca30184daa26487706d4829f5cfb697a"
],
"index": "pypi",
"version": "==3.12.1"
"version": "==3.2"
},
"django-health-check": {
"hashes": [
"sha256:2667b89b8f85ad9b2a24c90581b376016d22ea912fedf37f9866413a3c2e0a5d",
"sha256:894738bd7e461b2405c005927403ad5ee8048bbaf5934cf30b2c81a4e047d4b0"
],
"index": "pypi",
"version": "==3.12.3"
},
"django-ipware": {
"hashes": [
@@ -94,27 +106,27 @@
"index": "pypi",
"version": "==2.1.0"
},
"django-redis-cache": {
"django-npm": {
"hashes": [
"sha256:06d4e48545243883f88dc9263dda6c8a0012cb7d0cee2d8758d8917eca92cece",
"sha256:b19ee6654cc2f2c89078c99255e07e19dc2dba8792351d76ba7ea899d465fbb0"
"sha256:2e6bba65e728fa18b9db3c8dc0d4490b70cb7f43bacf60eb3654d7dcb6424272"
],
"index": "pypi",
"version": "==2.1.1"
"version": "==1.0.0"
},
"django-redis-cache": {
"hashes": [
"sha256:9a2eebef421d996a82098a19d17ff6b321265cd73178fa398913019764e8394a"
],
"index": "pypi",
"version": "==3.0.0"
},
"emoji-country-flag": {
"hashes": [
"sha256:67c0cb6a3765fb53f31b34160d6b1c8a5f44b297bc278d1835c6f2e5b0a9a592",
"sha256:ae7edb38077b0840210fa9e37673f481f2b9c032446e13ad6dab2b1108cd7ad6"
"sha256:338f5e374119dcde093cfeaa8ca3af372d4b8d984d89a7fb2fb0db0011662560",
"sha256:a3a068191294294143d8ef294fdfe9792c5c243753eac130798bf2fa5de38185"
],
"index": "pypi",
"version": "==1.2.1"
},
"future": {
"hashes": [
"sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"
],
"version": "==0.18.2"
"version": "==1.2.4"
},
"geoip2": {
"hashes": [
@@ -142,74 +154,75 @@
},
"idna": {
"hashes": [
"sha256:7588d1c14ae4c77d74036e8c22ff447b26d0fde8f007354fd48a7814db15b7cb",
"sha256:a068a21ceac8a4d63dbfd964670474107f541babbd2250d61922f029858365fa"
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
],
"version": "==2.9"
},
"ipaddress": {
"hashes": [
"sha256:6e0f4a39e66cb5bb9a137b00276a2eff74f93b71dcbdad6f10ff7df9d3557fcc",
"sha256:b7f8e0369580bb4a24d5ba1d7cc29660a4a6987763faf1d8a8046830e020e7e2"
],
"index": "pypi",
"version": "==1.0.23"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.10"
},
"kombu": {
"hashes": [
"sha256:437b9cdea193cc2ed0b8044c85fd0f126bb3615ca2f4d4a35b39de7cacfa3c1a",
"sha256:dc282bb277197d723bccda1a9ba30a27a28c9672d0ab93e9e51bb05a37bd29c3"
"sha256:be48cdffb54a2194d93ad6533d73f69408486483d189fe9f5990ee24255b0e0a",
"sha256:ca1b45faac8c0b18493d02a8571792f3c40291cf2bcf1f55afed3d8f3aa7ba74"
],
"version": "==4.6.10"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.6.11"
},
"maxminddb": {
"hashes": [
"sha256:f4d28823d9ca23323d113dc7af8db2087aa4f657fafc64ff8f7a8afda871425b"
"sha256:47e86a084dd814fac88c99ea34ba3278a74bc9de5a25f4b815b608798747c7dc"
],
"version": "==1.5.4"
"markers": "python_version >= '3.6'",
"version": "==2.0.3"
},
"oauthlib": {
"hashes": [
"sha256:bee41cc35fcca6e988463cacc3bcb8a96224f470ca547e697b604cc697b2f889",
"sha256:df884cd6cbe20e32633f1db1072e9356f53638e4361bef4e8b03c9127c9328ea"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==3.1.0"
},
"psycopg2-binary": {
"hashes": [
"sha256:008da3ab51adc70a5f1cfbbe5db3a22607ab030eb44bcecf517ad11a0c2b3cac",
"sha256:07cf82c870ec2d2ce94d18e70c13323c89f2f2a2628cbf1feee700630be2519a",
"sha256:08507efbe532029adee21b8d4c999170a83760d38249936038bd0602327029b5",
"sha256:107d9be3b614e52a192719c6bf32e8813030020ea1d1215daa86ded9a24d8b04",
"sha256:17a0ea0b0eabf07035e5e0d520dabc7950aeb15a17c6d36128ba99b2721b25b1",
"sha256:3286541b9d85a340ee4ed42732d15fc1bb441dc500c97243a768154ab8505bb5",
"sha256:3939cf75fc89c5e9ed836e228c4a63604dff95ad19aed2bbf71d5d04c15ed5ce",
"sha256:40abc319f7f26c042a11658bf3dd3b0b3bceccf883ec1c565d5c909a90204434",
"sha256:51f7823f1b087d2020d8e8c9e6687473d3d239ba9afc162d9b2ab6e80b53f9f9",
"sha256:6bb2dd006a46a4a4ce95201f836194eb6a1e863f69ee5bab506673e0ca767057",
"sha256:702f09d8f77dc4794651f650828791af82f7c2efd8c91ae79e3d9fe4bb7d4c98",
"sha256:7036ccf715925251fac969f4da9ad37e4b7e211b1e920860148a10c0de963522",
"sha256:7b832d76cc65c092abd9505cc670c4e3421fd136fb6ea5b94efbe4c146572505",
"sha256:8f74e631b67482d504d7e9cf364071fc5d54c28e79a093ff402d5f8f81e23bfa",
"sha256:930315ac53dc65cbf52ab6b6d27422611f5fb461d763c531db229c7e1af6c0b3",
"sha256:96d3038f5bd061401996614f65d27a4ecb62d843eb4f48e212e6d129171a721f",
"sha256:a20299ee0ea2f9cca494396ac472d6e636745652a64a418b39522c120fd0a0a4",
"sha256:a34826d6465c2e2bbe9d0605f944f19d2480589f89863ed5f091943be27c9de4",
"sha256:a69970ee896e21db4c57e398646af9edc71c003bc52a3cc77fb150240fefd266",
"sha256:b9a8b391c2b0321e0cd7ec6b4cfcc3dd6349347bd1207d48bcb752aa6c553a66",
"sha256:ba13346ff6d3eb2dca0b6fa0d8a9d999eff3dcd9b55f3a890f12b0b6362b2b38",
"sha256:bb0608694a91db1e230b4a314e8ed00ad07ed0c518f9a69b83af2717e31291a3",
"sha256:c8830b7d5f16fd79d39b21e3d94f247219036b29b30c8270314c46bf8b732389",
"sha256:cac918cd7c4c498a60f5d2a61d4f0a6091c2c9490d81bc805c963444032d0dab",
"sha256:cc30cb900f42c8a246e2cb76539d9726f407330bc244ca7729c41a44e8d807fb",
"sha256:ccdc6a87f32b491129ada4b87a43b1895cf2c20fdb7f98ad979647506ffc41b6",
"sha256:d1a8b01f6a964fec702d6b6dac1f91f2b9f9fe41b310cbb16c7ef1fac82df06d",
"sha256:e004db88e5a75e5fdab1620fb9f90c9598c2a195a594225ac4ed2a6f1c23e162",
"sha256:eb2f43ae3037f1ef5e19339c41cf56947021ac892f668765cd65f8ab9814192e",
"sha256:fa466306fcf6b39b8a61d003123d442b23707d635a5cb05ac4e1b62cc79105cd"
"sha256:0deac2af1a587ae12836aa07970f5cb91964f05a7c6cdb69d8425ff4c15d4e2c",
"sha256:0e4dc3d5996760104746e6cfcdb519d9d2cd27c738296525d5867ea695774e67",
"sha256:11b9c0ebce097180129e422379b824ae21c8f2a6596b159c7659e2e5a00e1aa0",
"sha256:15978a1fbd225583dd8cdaf37e67ccc278b5abecb4caf6b2d6b8e2b948e953f6",
"sha256:1fabed9ea2acc4efe4671b92c669a213db744d2af8a9fc5d69a8e9bc14b7a9db",
"sha256:2dac98e85565d5688e8ab7bdea5446674a83a3945a8f416ad0110018d1501b94",
"sha256:42ec1035841b389e8cc3692277a0bd81cdfe0b65d575a2c8862cec7a80e62e52",
"sha256:6422f2ff0919fd720195f64ffd8f924c1395d30f9a495f31e2392c2efafb5056",
"sha256:6a32f3a4cb2f6e1a0b15215f448e8ce2da192fd4ff35084d80d5e39da683e79b",
"sha256:7312e931b90fe14f925729cde58022f5d034241918a5c4f9797cac62f6b3a9dd",
"sha256:7d92a09b788cbb1aec325af5fcba9fed7203897bbd9269d5691bb1e3bce29550",
"sha256:833709a5c66ca52f1d21d41865a637223b368c0ee76ea54ca5bad6f2526c7679",
"sha256:89705f45ce07b2dfa806ee84439ec67c5d9a0ef20154e0e475e2b2ed392a5b83",
"sha256:8cd0fb36c7412996859cb4606a35969dd01f4ea34d9812a141cd920c3b18be77",
"sha256:950bc22bb56ee6ff142a2cb9ee980b571dd0912b0334aa3fe0fe3788d860bea2",
"sha256:a0c50db33c32594305b0ef9abc0cb7db13de7621d2cadf8392a1d9b3c437ef77",
"sha256:a0eb43a07386c3f1f1ebb4dc7aafb13f67188eab896e7397aa1ee95a9c884eb2",
"sha256:aaa4213c862f0ef00022751161df35804127b78adf4a2755b9f991a507e425fd",
"sha256:ac0c682111fbf404525dfc0f18a8b5f11be52657d4f96e9fcb75daf4f3984859",
"sha256:ad20d2eb875aaa1ea6d0f2916949f5c08a19c74d05b16ce6ebf6d24f2c9f75d1",
"sha256:b4afc542c0ac0db720cf516dd20c0846f71c248d2b3d21013aa0d4ef9c71ca25",
"sha256:b8a3715b3c4e604bcc94c90a825cd7f5635417453b253499664f784fc4da0152",
"sha256:ba28584e6bca48c59eecbf7efb1576ca214b47f05194646b081717fa628dfddf",
"sha256:ba381aec3a5dc29634f20692349d73f2d21f17653bda1decf0b52b11d694541f",
"sha256:bd1be66dde2b82f80afb9459fc618216753f67109b859a361cf7def5c7968729",
"sha256:c2507d796fca339c8fb03216364cca68d87e037c1f774977c8fc377627d01c71",
"sha256:cec7e622ebc545dbb4564e483dd20e4e404da17ae07e06f3e780b2dacd5cee66",
"sha256:d14b140a4439d816e3b1229a4a525df917d6ea22a0771a2a78332273fd9528a4",
"sha256:d1b4ab59e02d9008efe10ceabd0b31e79519da6fb67f7d8e8977118832d0f449",
"sha256:d5227b229005a696cc67676e24c214740efd90b148de5733419ac9aaba3773da",
"sha256:e1f57aa70d3f7cc6947fd88636a481638263ba04a742b4a37dd25c373e41491a",
"sha256:e74a55f6bad0e7d3968399deb50f61f4db1926acf4a6d83beaaa7df986f48b1c",
"sha256:e82aba2188b9ba309fd8e271702bd0d0fc9148ae3150532bbb474f4590039ffb",
"sha256:ee69dad2c7155756ad114c02db06002f4cded41132cc51378e57aad79cc8e4f4",
"sha256:f5ab93a2cb2d8338b1674be43b442a7f544a0971da062a5da774ed40587f18f5"
],
"index": "pypi",
"version": "==2.8.5"
"version": "==2.8.6"
},
"pycountry": {
"hashes": [
@@ -220,34 +233,52 @@
},
"python3-openid": {
"hashes": [
"sha256:0086da6b6ef3161cfe50fb1ee5cceaf2cda1700019fda03c2c5c440ca6abe4fa",
"sha256:628d365d687e12da12d02c6691170f4451db28d6d68d050007e4a40065868502"
"sha256:33fbf6928f401e0b790151ed2b5290b02545e8775f982485205a066f874aaeaf",
"sha256:6626f771e0417486701e0b4daff762e7212e820ca5b29fcc0d05f6f8736dfa6b"
],
"version": "==3.1.0"
"version": "==3.2.0"
},
"pytz": {
"hashes": [
"sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed",
"sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048"
"sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da",
"sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798"
],
"version": "==2020.1"
"version": "==2021.1"
},
"pyyaml": {
"hashes": [
"sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97",
"sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76",
"sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2",
"sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648",
"sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf",
"sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f",
"sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2",
"sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee",
"sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d",
"sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c",
"sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a"
"sha256:08682f6b72c722394747bddaf0aa62277e02557c0fd1c42cb853016a38f8dedf",
"sha256:0f5f5786c0e09baddcd8b4b45f20a7b5d61a7e7e99846e3c799b05c7c53fa696",
"sha256:129def1b7c1bf22faffd67b8f3724645203b79d8f4cc81f674654d9902cb4393",
"sha256:294db365efa064d00b8d1ef65d8ea2c3426ac366c0c4368d930bf1c5fb497f77",
"sha256:3b2b1824fe7112845700f815ff6a489360226a5609b96ec2190a45e62a9fc922",
"sha256:3bd0e463264cf257d1ffd2e40223b197271046d09dadf73a0fe82b9c1fc385a5",
"sha256:4465124ef1b18d9ace298060f4eccc64b0850899ac4ac53294547536533800c8",
"sha256:49d4cdd9065b9b6e206d0595fee27a96b5dd22618e7520c33204a4a3239d5b10",
"sha256:4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc",
"sha256:5accb17103e43963b80e6f837831f38d314a0495500067cb25afab2e8d7a4018",
"sha256:607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e",
"sha256:6c78645d400265a062508ae399b60b8c167bf003db364ecb26dcab2bda048253",
"sha256:72a01f726a9c7851ca9bfad6fd09ca4e090a023c00945ea05ba1638c09dc3347",
"sha256:74c1485f7707cf707a7aef42ef6322b8f97921bd89be2ab6317fd782c2d53183",
"sha256:895f61ef02e8fed38159bb70f7e100e00f471eae2bc838cd0f4ebb21e28f8541",
"sha256:8c1be557ee92a20f184922c7b6424e8ab6691788e6d86137c5d93c1a6ec1b8fb",
"sha256:bb4191dfc9306777bc594117aee052446b3fa88737cd13b7188d0e7aa8162185",
"sha256:bfb51918d4ff3d77c1c856a9699f8492c612cde32fd3bcd344af9be34999bfdc",
"sha256:c20cfa2d49991c8b4147af39859b167664f2ad4561704ee74c1de03318e898db",
"sha256:cb333c16912324fd5f769fff6bc5de372e9e7a202247b48870bc251ed40239aa",
"sha256:d2d9808ea7b4af864f35ea216be506ecec180628aced0704e34aca0b040ffe46",
"sha256:d483ad4e639292c90170eb6f7783ad19490e7a8defb3e46f97dfe4bacae89122",
"sha256:dd5de0646207f053eb0d6c74ae45ba98c3395a571a2891858e87df7c9b9bd51b",
"sha256:e1d4970ea66be07ae37a3c2e48b5ec63f7ba6804bdddfdbd3cfd954d25a82e63",
"sha256:e4fac90784481d221a8e4b1162afa7c47ed953be40d31ab4629ae917510051df",
"sha256:fa5ae20527d8e831e8230cbffd9f8fe952815b2b7dae6ffec25318803a7528fc",
"sha256:fd7f6999a8070df521b6384004ef42833b9bd62cfee11a09bda1079b4b704247",
"sha256:fdc842473cd33f45ff6bce46aea678a54e3d21f1b61a7750ce3c498eedfe25d6",
"sha256:fe69978f3f768926cfa37b867e3843918e012cf83f680806599ddce33c2c68b0"
],
"index": "pypi",
"version": "==5.3.1"
"version": "==5.4.1"
},
"redis": {
"hashes": [
@@ -259,15 +290,17 @@
},
"requests": {
"hashes": [
"sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee",
"sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6"
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
],
"version": "==2.23.0"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==2.25.1"
},
"requests-oauthlib": {
"hashes": [
"sha256:7f71572defaecd16372f9006f33c2ec8c077c3cfa6f5911a9a90202beb513f3d",
"sha256:b4261601a71fd721a8bd6d7aa1cc1d6a8a93b4a9f5e96626f8e4d91e8beeaa6a"
"sha256:b4261601a71fd721a8bd6d7aa1cc1d6a8a93b4a9f5e96626f8e4d91e8beeaa6a",
"sha256:fa6c47b933f01060936d87ae9327fead68768b69c6c9ea2109c48be30f2d4dbc"
],
"version": "==1.3.0"
},
@@ -278,19 +311,13 @@
"index": "pypi",
"version": "==2.2"
},
"six": {
"hashes": [
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
],
"version": "==1.15.0"
},
"sqlparse": {
"hashes": [
"sha256:022fb9c87b524d1f7862b3037e541f68597a730a8843245c349fc93e1643dc4e",
"sha256:e162203737712307dfe78860cc56c8da8a852ab2ee33750e33aeadf38d12c548"
"sha256:017cde379adbd6a1f15a61873f43e8274179378e95ef3fede90b5aa64d304ed0",
"sha256:0f91fd2e829c44362cbcfab3e9ae12e22badaa8a29ad5ff599f9ec109f0454e8"
],
"version": "==0.3.1"
"markers": "python_version >= '3.5'",
"version": "==0.4.1"
},
"ua-parser": {
"hashes": [
@@ -302,24 +329,26 @@
},
"urllib3": {
"hashes": [
"sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527",
"sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115"
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
],
"version": "==1.25.9"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.4"
},
"user-agents": {
"hashes": [
"sha256:da54371d856c35d8ead0622da24ad5ef6d667eda3629a750e3373a3e847a054b",
"sha256:e727ab6f169e829bc25d41dbd25b9ff679b4631bd81959bcf7de1e246da67194"
"sha256:a98c4dc72ecbc64812c4534108806fb0a0b3a11ec3fd1eafe807cee5b0a942e7",
"sha256:d36d25178db65308d1458c5fa4ab39c9b2619377010130329f3955e7626ead26"
],
"index": "pypi",
"version": "==2.1"
"version": "==2.2.0"
},
"vine": {
"hashes": [
"sha256:133ee6d7a9016f177ddeaf191c1f58421a1dcc6ee9a42c58b34bed40e1d2cd87",
"sha256:ea4947cc56d1fd6f2095c8d543ee25dad966f78692528e68b4fada11ba3f98af"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.3.0"
},
"whitenoise": {
@@ -331,101 +360,5 @@
"version": "==5.1.0"
}
},
"develop": {
"appdirs": {
"hashes": [
"sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41",
"sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"
],
"version": "==1.4.4"
},
"attrs": {
"hashes": [
"sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
"sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
],
"version": "==19.3.0"
},
"black": {
"hashes": [
"sha256:1b30e59be925fafc1ee4565e5e08abef6b03fe455102883820fe5ee2e4734e0b",
"sha256:c2edb73a08e9e0e6f65a0e6af18b059b8b1cdd5bef997d7a0b181df93dc81539"
],
"index": "pypi",
"version": "==19.10b0"
},
"click": {
"hashes": [
"sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
"sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
],
"version": "==7.1.2"
},
"pathspec": {
"hashes": [
"sha256:7d91249d21749788d07a2d0f94147accd8f845507400749ea19c1ec9054a12b0",
"sha256:da45173eb3a6f2a5a487efba21f050af2b41948be6ab52b6a1e3ff22bb8b7061"
],
"version": "==0.8.0"
},
"regex": {
"hashes": [
"sha256:08997a37b221a3e27d68ffb601e45abfb0093d39ee770e4257bd2f5115e8cb0a",
"sha256:112e34adf95e45158c597feea65d06a8124898bdeac975c9087fe71b572bd938",
"sha256:1700419d8a18c26ff396b3b06ace315b5f2a6e780dad387e4c48717a12a22c29",
"sha256:2f6f211633ee8d3f7706953e9d3edc7ce63a1d6aad0be5dcee1ece127eea13ae",
"sha256:52e1b4bef02f4040b2fd547357a170fc1146e60ab310cdbdd098db86e929b387",
"sha256:55b4c25cbb3b29f8d5e63aeed27b49fa0f8476b0d4e1b3171d85db891938cc3a",
"sha256:5aaa5928b039ae440d775acea11d01e42ff26e1561c0ffcd3d805750973c6baf",
"sha256:654cb773b2792e50151f0e22be0f2b6e1c3a04c5328ff1d9d59c0398d37ef610",
"sha256:690f858d9a94d903cf5cada62ce069b5d93b313d7d05456dbcd99420856562d9",
"sha256:6ad8663c17db4c5ef438141f99e291c4d4edfeaacc0ce28b5bba2b0bf273d9b5",
"sha256:89cda1a5d3e33ec9e231ece7307afc101b5217523d55ef4dc7fb2abd6de71ba3",
"sha256:92d8a043a4241a710c1cf7593f5577fbb832cf6c3a00ff3fc1ff2052aff5dd89",
"sha256:95fa7726d073c87141f7bbfb04c284901f8328e2d430eeb71b8ffdd5742a5ded",
"sha256:97712e0d0af05febd8ab63d2ef0ab2d0cd9deddf4476f7aa153f76feef4b2754",
"sha256:b2ba0f78b3ef375114856cbdaa30559914d081c416b431f2437f83ce4f8b7f2f",
"sha256:bae83f2a56ab30d5353b47f9b2a33e4aac4de9401fb582b55c42b132a8ac3868",
"sha256:c78e66a922de1c95a208e4ec02e2e5cf0bb83a36ceececc10a72841e53fbf2bd",
"sha256:cf59bbf282b627130f5ba68b7fa3abdb96372b24b66bdf72a4920e8153fc7910",
"sha256:e3cdc9423808f7e1bb9c2e0bdb1c9dc37b0607b30d646ff6faf0d4e41ee8fee3",
"sha256:e9b64e609d37438f7d6e68c2546d2cb8062f3adb27e6336bc129b51be20773ac",
"sha256:fbff901c54c22425a5b809b914a3bfaf4b9570eee0e5ce8186ac71eb2025191c"
],
"version": "==2020.6.8"
},
"toml": {
"hashes": [
"sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f",
"sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"
],
"version": "==0.10.1"
},
"typed-ast": {
"hashes": [
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355",
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919",
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa",
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652",
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75",
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01",
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d",
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1",
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907",
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c",
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3",
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b",
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614",
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb",
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b",
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41",
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6",
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34",
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe",
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4",
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"
],
"version": "==1.4.1"
}
}
"develop": {}
}

View File

@@ -14,7 +14,10 @@ EMAIL_HOST_USER=example
EMAIL_HOST_PASSWORD=example_password
EMAIL_HOST=smtp.example.com
EMAIL_PORT=465
SERVER_EMAIL=<Shynet> noreply@shynet.example.com
EMAIL_USE_SSL=True
# Comment out EMAIL_USE_SSL & uncomment EMAIL_USE_TLS if your SMTP server uses TLS.
# EMAIL_USE_TLS=True
SERVER_EMAIL=Shynet <noreply@shynet.example.com>
# General Django settings
DJANGO_SECRET_KEY=random_string
@@ -23,7 +26,7 @@ DJANGO_SECRET_KEY=random_string
ALLOWED_HOSTS=*
# Set to True (capitalized) if you want people to be able to sign up for your Shynet instance (not recommended)
SIGNUPS_ENABLED=False
ACCOUNT_SIGNUPS_ENABLED=False
# Should user email addresses be verified? Only set this to `required` if you've setup the email settings and allow
# public sign-ups; otherwise, it's unnecessary.
@@ -55,12 +58,28 @@ PERFORM_CHECKS_AND_SETUP=True
# The port that Shynet should bind to. Don't set this if you're deploying on Heroku.
PORT=8080
# Set to "False" if you do not want the version to be displayed on the frontend.
SHOW_SHYNET_VERSION=True
# Redis, queue, and parellization settings; not necessary for single-instance deployments.
# Don't uncomment these unless you know what you are doing!
# NUM_WORKERS=1
# Make sure you set a REDIS_CACHE_LOCATION if you have more than one frontend worker/instance.
# REDIS_CACHE_LOCATION=redis://redis.default.svc.cluster.local/0
# REDIS_CACHE_LOCATION=redis://redis.default.svc.cluster.local/0
# If CELERY_BROKER_URL is set, make sure CELERY_TASK_ALWAYS_EAGER is False and
# that you have a separate queue consumer running somewhere via `celeryworker.sh`.
# CELERY_TASK_ALWAYS_EAGER=False
# CELERY_BROKER_URL=redis://redis.default.svc.cluster.local/1
# CELERY_BROKER_URL=redis://redis.default.svc.cluster.local/1
# Should Shynet show third-party icons in the dashboard?
SHOW_THIRD_PARTY_ICONS=True
# Should Shynet block collection of IP addresses globally?
BLOCK_ALL_IPS=False
# Should Shynet include the date and site ID when hashing users?
# This will prevent any possibility of cross-site tracking provided
# that IP collection is also disabled, and external keys (primary
# keys) aren't supplied. It will also prevent sessions from spanning
# one day to another.
AGGRESSIVE_HASH_SALTING=True

View File

@@ -1,13 +1,13 @@
{
"name": "Shynet",
"description":"Modern, privacy-friendly, and detailed web analytics that works without cookies or JS.",
"keywords":[
"app.json",
"shynet",
"heroku",
"analytics",
"privacy",
"friendly"
"description": "Modern, privacy-friendly, and detailed web analytics that works without cookies or JS.",
"keywords": [
"app.json",
"shynet",
"heroku",
"analytics",
"privacy",
"friendly"
],
"website": "https://github.com/milesmcc/shynet",
"repository": "https://github.com/milesmcc/shynet",
@@ -83,7 +83,7 @@
"value": "*",
"required": false
},
"SIGNUPS_ENABLED": {
"ACCOUNT_SIGNUPS_ENABLED": {
"description": "Set to True (capitalized) if you want people to be able to sign up for your Shynet instance (not recommended).",
"value": "False",
"required": false
@@ -117,6 +117,11 @@
"description": "Whether to perform checks and setup at startup. Recommended value is 'True' for Heroku users.",
"value": "True",
"required": false
},
"SHOW_SHYNET_VERSION": {
"description": "Set to 'False' if you do not want the version to be displayed on the frontend.",
"value": "True",
"required": false
}
}
}
}

View File

@@ -1,6 +1,7 @@
version: '3'
services:
shynet:
container_name: shynet_main
image: milesmcc/shynet:latest
restart: unless-stopped
expose:
@@ -16,6 +17,7 @@ services:
depends_on:
- db
db:
container_name: shynet_database
image: postgres
restart: always
environment:
@@ -26,6 +28,18 @@ services:
- shynet_db:/var/lib/postgresql/data
networks:
- internal
webserver:
container_name: shynet_webserver
image: nginx
restart: always
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- 8080:80
depends_on:
- shynet
networks:
- internal
volumes:
shynet_db:
networks:

Binary file not shown.

Before

Width:  |  Height:  |  Size: 308 KiB

After

Width:  |  Height:  |  Size: 263 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 8.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 589 KiB

After

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.5 KiB

After

Width:  |  Height:  |  Size: 931 B

View File

@@ -17,7 +17,7 @@ spec:
spec:
containers:
- name: "shynet-webserver"
image: "milesmcc/shynet:latest"
image: "milesmcc/shynet:dev"
imagePullPolicy: Always
envFrom:
- secretRef:
@@ -42,7 +42,7 @@ spec:
spec:
containers:
- name: "shynet-celeryworker"
image: "milesmcc/shynet:latest"
image: "milesmcc/shynet:dev"
command: ["./celeryworker.sh"]
imagePullPolicy: Always
envFrom:
@@ -61,7 +61,7 @@ spec:
selector:
app: shynet-redis
---
apiVersion: apps/v1beta2
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: shynet-redis
@@ -83,3 +83,37 @@ spec:
ports:
- containerPort: 6379
name: redis
---
apiVersion: v1
kind: Service
metadata:
name: shynet-webserver-service
spec:
type: ClusterIP
ports:
- port: 8080
selector:
app: shynet-webserver
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: shynet-webserver-ingress
annotations:
kubernetes.io/ingress.class: addon-http-application-routing
spec:
rules:
- host: shynet.rmrm.io
http:
paths:
- backend:
serviceName: shynet-webserver-service
servicePort: 8080
path: /
- host: shynet-beta.rmrm.io
http:
paths:
- backend:
serviceName: shynet-webserver-service
servicePort: 8080
path: /

View File

@@ -8,7 +8,7 @@ stringData:
DEBUG: "False"
ALLOWED_HOSTS: "*" # For better security, set this to your deployment's domain. Comma separated.
DJANGO_SECRET_KEY: ""
SIGNUPS_ENABLED: "False"
ACCOUNT_SIGNUPS_ENABLED: "False"
TIME_ZONE: "America/New_York"
# Redis configuration (if you use the default Kubernetes config, this will work)

19
nginx.conf Normal file
View File

@@ -0,0 +1,19 @@
server {
server_name example.com;
access_log /var/log/nginx/bin.access.log;
error_log /var/log/nginx/bin.error.log error;
location / {
proxy_pass http://shynet:8080;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Url-Scheme $scheme;
}
listen 80;
}

104
package-lock.json generated Normal file
View File

@@ -0,0 +1,104 @@
{
"name": "shynet",
"requires": true,
"lockfileVersion": 1,
"dependencies": {
"@fortawesome/fontawesome-free": {
"version": "5.15.1",
"resolved": "https://registry.npmjs.org/@fortawesome/fontawesome-free/-/fontawesome-free-5.15.1.tgz",
"integrity": "sha512-OEdH7SyC1suTdhBGW91/zBfR6qaIhThbcN8PUXtXilY4GYnSBbVqOntdHbC1vXwsDnX0Qix2m2+DSU1J51ybOQ=="
},
"a17t": {
"version": "0.5.1",
"resolved": "https://registry.npmjs.org/a17t/-/a17t-0.5.1.tgz",
"integrity": "sha512-peIPrH9eDiu49LLzLlSTFFrXj6WLlEX3TRsUkqyyOHi/i58ilJ/eERnu7AcswXhuCBx+/2W9EUuHM+8iAq4ipg=="
},
"apexcharts": {
"version": "3.24.0",
"resolved": "https://registry.npmjs.org/apexcharts/-/apexcharts-3.24.0.tgz",
"integrity": "sha512-iT6czJCIVrmAtrcO90MZTQCvC+xi6R6Acf0jNH/d40FVTtCfcqECuKIh5iAMyOTtgUb7+fQ8rbadH2bm1kbL9Q==",
"requires": {
"svg.draggable.js": "^2.2.2",
"svg.easing.js": "^2.0.0",
"svg.filter.js": "^2.0.2",
"svg.pathmorphing.js": "^0.1.3",
"svg.resize.js": "^1.4.3",
"svg.select.js": "^3.0.1"
}
},
"inter-ui": {
"version": "3.15.0",
"resolved": "https://registry.npmjs.org/inter-ui/-/inter-ui-3.15.0.tgz",
"integrity": "sha512-6v0WK8FHkVYbNQZ7L9O5tP8280pgTBR9ydxqYwssMuUH6SZO70ZFK/NQ1Ob8nNmOOzpUJAzT0WE73ty96z1tAQ=="
},
"litepicker": {
"version": "1.5.7",
"resolved": "https://registry.npmjs.org/litepicker/-/litepicker-1.5.7.tgz",
"integrity": "sha512-4L2ZcF8iqCE4A/qGWS3PbdFplZR1g751x5SsZ87zCRZ4LQN1Fgezarnvqi0eHk/kDWK7Qx0HZ9Y4bNznJMF1xA=="
},
"svg.draggable.js": {
"version": "2.2.2",
"resolved": "https://registry.npmjs.org/svg.draggable.js/-/svg.draggable.js-2.2.2.tgz",
"integrity": "sha512-JzNHBc2fLQMzYCZ90KZHN2ohXL0BQJGQimK1kGk6AvSeibuKcIdDX9Kr0dT9+UJ5O8nYA0RB839Lhvk4CY4MZw==",
"requires": {
"svg.js": "^2.0.1"
}
},
"svg.easing.js": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/svg.easing.js/-/svg.easing.js-2.0.0.tgz",
"integrity": "sha1-iqmUawqOJ4V6XEChDrpAkeVpHxI=",
"requires": {
"svg.js": ">=2.3.x"
}
},
"svg.filter.js": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/svg.filter.js/-/svg.filter.js-2.0.2.tgz",
"integrity": "sha1-kQCOFROJ3ZIwd5/L5uLJo2LRwgM=",
"requires": {
"svg.js": "^2.2.5"
}
},
"svg.js": {
"version": "2.7.1",
"resolved": "https://registry.npmjs.org/svg.js/-/svg.js-2.7.1.tgz",
"integrity": "sha512-ycbxpizEQktk3FYvn/8BH+6/EuWXg7ZpQREJvgacqn46gIddG24tNNe4Son6omdXCnSOaApnpZw6MPCBA1dODA=="
},
"svg.pathmorphing.js": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/svg.pathmorphing.js/-/svg.pathmorphing.js-0.1.3.tgz",
"integrity": "sha512-49HWI9X4XQR/JG1qXkSDV8xViuTLIWm/B/7YuQELV5KMOPtXjiwH4XPJvr/ghEDibmLQ9Oc22dpWpG0vUDDNww==",
"requires": {
"svg.js": "^2.4.0"
}
},
"svg.resize.js": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/svg.resize.js/-/svg.resize.js-1.4.3.tgz",
"integrity": "sha512-9k5sXJuPKp+mVzXNvxz7U0uC9oVMQrrf7cFsETznzUDDm0x8+77dtZkWdMfRlmbkEEYvUn9btKuZ3n41oNA+uw==",
"requires": {
"svg.js": "^2.6.5",
"svg.select.js": "^2.1.2"
},
"dependencies": {
"svg.select.js": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/svg.select.js/-/svg.select.js-2.1.2.tgz",
"integrity": "sha512-tH6ABEyJsAOVAhwcCjF8mw4crjXSI1aa7j2VQR8ZuJ37H2MBUbyeqYr5nEO7sSN3cy9AR9DUwNg0t/962HlDbQ==",
"requires": {
"svg.js": "^2.2.5"
}
}
}
},
"svg.select.js": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/svg.select.js/-/svg.select.js-3.0.1.tgz",
"integrity": "sha512-h5IS/hKkuVCbKSieR9uQCj9w+zLHoPh+ce19bBYyqF53g6mnPB8sAtIbe1s9dh2S2fCmYX2xel1Ln3PJBbK4kw==",
"requires": {
"svg.js": "^2.6.5"
}
}
}
}

26
package.json Normal file
View File

@@ -0,0 +1,26 @@
{
"name": "shynet",
"description": "Modern, privacy-friendly, and cookie-free web analytics.",
"repository": {
"type": "git",
"url": "git+https://github.com/milesmcc/shynet.git"
},
"keywords": [
"privacy",
"analytics",
"self-host"
],
"author": "R. Miles McCain <shynet@sendmiles.email>",
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/milesmcc/shynet/issues"
},
"homepage": "https://github.com/milesmcc/shynet#readme",
"dependencies": {
"@fortawesome/fontawesome-free": "^5.15.1",
"a17t": "^0.5.1",
"apexcharts": "^3.24.0",
"inter-ui": "^3.15.0",
"litepicker": "^1.5.7"
}
}

View File

@@ -1,10 +0,0 @@
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/a17t@0.1.3/dist/a17t.css">
<script async src="https://use.fontawesome.com/releases/v5.3.1/js/all.js"></script>
<link href="https://cdn.jsdelivr.net/npm/tailwindcss@^1.0/dist/tailwind.min.css" rel="stylesheet">
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&display=swap" rel="stylesheet">
<style>
:root {
--family-primary: "Inter", system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
--family-secondary: var(--family-primary);
}
</style>

View File

@@ -0,0 +1,12 @@
{% load static %}
<link rel="stylesheet" href="{% static 'a17t/dist/a17t.css' %}">
<script async src="{% static '@fortawesome/fontawesome-free/js/all.min.js' %}" data-mutate-approach="sync"></script>
<link href="{% static 'a17t/dist/tailwind.css' %}" rel="stylesheet">
<link href="{% static 'inter-ui/Inter (web)/inter.css' %}" rel="stylesheet">
<style>
:root {
--family-primary: "Inter", system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
--family-secondary: var(--family-primary);
}
</style>

View File

@@ -1,23 +1,23 @@
<nav class="flex w-full flex-wrap items-center justify-between" role="navigation" aria-label="pagination">
<div class="w-full md:w-auto mb-2">
{% if page.has_previous %}
<a href="?page={{ page.previous_page_number }}{{url_parameters}}" class="button field w-auto mr-1">Previous</a>
<a href="?page={{ page.previous_page_number }}&{{url_parameters}}" class="button field bg-neutral-000 w-auto mr-1">Previous</a>
{% else %}
<a class="button field w-auto mr-1" disabled>Previous</a>
<a class="button field bg-neutral-000 w-auto mr-1" disabled>Previous</a>
{% endif %}
{% if page.has_next %}
<a href="?page={{ page.next_page_number }}{{url_parameters}}" class="button field w-auto">Next</a>
<a href="?page={{ page.next_page_number }}&{{url_parameters}}" class="button field bg-neutral-000 w-auto">Next</a>
{% else %}
<a class="button field w-auto" disabled>Next</a>
<a class="button field bg-neutral-000 w-auto" disabled>Next</a>
{% endif %}
</div>
<ul class="pagination-list w-full md:w-auto mb-2 flex">
{% for pnum in begin %}
{% ifequal page.number pnum %}
<li><a class="button field w-auto mx-1 text-white bg-gray-700">{{ pnum }}</a></li>
<li><a class="button field w-auto mx-1 text-white bg-neutral-700">{{ pnum }}</a></li>
{% else %}
<li><a class="button field w-auto mx-1" href="?page={{ pnum }}{{url_parameters}}">{{ pnum }}</a></li>
<li><a class="button field bg-neutral-000 w-auto mx-1" href="?page={{ pnum }}&{{url_parameters}}">{{ pnum }}</a></li>
{% endifequal %}
{% endfor %}
@@ -25,9 +25,9 @@
<li><span class="pagination-ellipsis">&hellip;</span></li>
{% for pnum in middle %}
{% ifequal page.number pnum %}
<li><a class="button field w-auto mx-1 text-white bg-gray-700">{{ pnum }}</a></li>
<li><a class="button field w-auto mx-1 text-white bg-neutral-700">{{ pnum }}</a></li>
{% else %}
<li><a class="button field w-auto mx-1" href="?page={{ pnum }}{{url_parameters}}">{{ pnum }}</a></li>
<li><a class="button field bg-neutral-000 w-auto mx-1" href="?page={{ pnum }}&{{url_parameters}}">{{ pnum }}</a></li>
{% endifequal %}
{% endfor %}
{% endif %}
@@ -36,9 +36,9 @@
<li><span class="pagination-ellipsis">&hellip;</span></li>
{% for pnum in end %}
{% ifequal page.number pnum %}
<li><a class="button field w-auto mx-1 text-white bg-gray-700">{{ pnum }}</a></li>
<li><a class="button field w-auto mx-1 text-white bg-neutral-700">{{ pnum }}</a></li>
{% else %}
<li><a class="button field w-auto mx-1" href="?page={{ pnum }}{{url_parameters}}">{{ pnum }}</a></li>
<li><a class="button field bg-neutral-000 w-auto mx-1" href="?page={{ pnum }}&{{url_parameters}}">{{ pnum }}</a></li>
{% endifequal %}
{% endfor %}
{% endif %}

View File

@@ -92,4 +92,6 @@ def is_file(field):
def add_class(field, css_class):
if len(field.errors) > 0:
css_class += " ~critical"
if field.field.widget.attrs.get("class") != None:
css_class += " " + field.field.widget.attrs["class"]
return field.as_widget(attrs={"class": field.css_classes(extra_classes=css_class)})

View File

@@ -15,9 +15,9 @@ def pagination(
before_current_pages=4,
after_current_pages=4,
):
url_parameters = "".join(
url_parameters = urlencode(
[
f"&{urlencode(key)}={urlencode(value)}"
(key, value)
for key, value in request.GET.items()
if key != "page"
]

View File

@@ -0,0 +1,38 @@
# Generated by Django 3.1.7 on 2021-03-28 19:14
from django.db import migrations, models
import django.utils.timezone
class Migration(migrations.Migration):
dependencies = [
('analytics', '0003_auto_20200502_1227'),
]
operations = [
migrations.AlterField(
model_name='hit',
name='last_seen',
field=models.DateTimeField(default=django.utils.timezone.now),
),
migrations.AlterField(
model_name='hit',
name='start_time',
field=models.DateTimeField(db_index=True, default=django.utils.timezone.now),
),
migrations.AlterField(
model_name='session',
name='last_seen',
field=models.DateTimeField(db_index=True, default=django.utils.timezone.now),
),
migrations.AlterField(
model_name='session',
name='start_time',
field=models.DateTimeField(db_index=True, default=django.utils.timezone.now),
),
migrations.AddIndex(
model_name='session',
index=models.Index(fields=['service', '-last_seen'], name='analytics_s_service_10bb96_idx'),
),
]

View File

@@ -0,0 +1,24 @@
# Generated by Django 3.1.7 on 2021-03-28 19:18
from django.db import migrations, models
import django.utils.timezone
class Migration(migrations.Migration):
dependencies = [
('analytics', '0004_auto_20210328_1514'),
]
operations = [
migrations.AlterField(
model_name='hit',
name='last_seen',
field=models.DateTimeField(db_index=True, default=django.utils.timezone.now),
),
migrations.AlterField(
model_name='hit',
name='load_time',
field=models.FloatField(db_index=True, null=True),
),
]

View File

@@ -0,0 +1,38 @@
# Generated by Django 3.1.7 on 2021-03-28 19:36
from ..models import Hit, Session
from django.db import migrations, models
import django.db.models.deletion
from django.db.models import Subquery, OuterRef
def add_service_to_hits(_a, _b):
service = Session.objects.filter(
pk=OuterRef('session')
).values_list(
'service'
)[:1]
Hit.objects.update(
service=Subquery(service)
)
class Migration(migrations.Migration):
dependencies = [
('core', '0008_auto_20200628_1403'),
('analytics', '0005_auto_20210328_1518'),
]
operations = [
migrations.AddField(
model_name='hit',
name='service',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='core.service'),
),
migrations.RunPython(add_service_to_hits, lambda: ()),
migrations.AlterField(
model_name='hit',
name='service',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='core.service'),
),
]

View File

@@ -0,0 +1,17 @@
# Generated by Django 3.1.7 on 2021-03-28 20:34
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('analytics', '0006_hit_service'),
]
operations = [
migrations.AddIndex(
model_name='hit',
index=models.Index(fields=['service', '-start_time'], name='analytics_h_service_f4f41e_idx'),
),
]

View File

@@ -0,0 +1,24 @@
# Generated by Django 3.1.7 on 2021-03-28 21:38
from django.db.models.expressions import F
from ..models import Session, Hit
from django.db import migrations, models
from django.db.models import Subquery, OuterRef
def update_bounce_stats(_a, _b):
Session.objects.all().annotate(hit_count=models.Count("hit")).filter(hit_count__gt=1).update(is_bounce=False)
class Migration(migrations.Migration):
dependencies = [
('analytics', '0007_auto_20210328_1634'),
]
operations = [
migrations.AddField(
model_name='session',
name='is_bounce',
field=models.BooleanField(db_index=True, default=True),
),
migrations.RunPython(update_bounce_stats, lambda: ()),
]

View File

@@ -1,6 +1,7 @@
import json
import uuid
from django.conf import settings
from django.db import models
from django.shortcuts import reverse
from django.utils import timezone
@@ -20,8 +21,8 @@ class Session(models.Model):
identifier = models.TextField(blank=True, db_index=True)
# Time
start_time = models.DateTimeField(auto_now_add=True, db_index=True)
last_seen = models.DateTimeField(auto_now_add=True)
start_time = models.DateTimeField(default=timezone.now, db_index=True)
last_seen = models.DateTimeField(default=timezone.now, db_index=True)
# Core request information
user_agent = models.TextField()
@@ -48,16 +49,21 @@ class Session(models.Model):
latitude = models.FloatField(null=True)
time_zone = models.TextField(blank=True)
is_bounce = models.BooleanField(default=True, db_index=True)
class Meta:
ordering = ["-start_time"]
indexes = [
models.Index(fields=["service", "-start_time"]),
models.Index(fields=["service", "-last_seen"]),
models.Index(fields=["service", "identifier"]),
]
@property
def is_currently_active(self):
return timezone.now() - self.last_seen < timezone.timedelta(seconds=10)
return timezone.now() - self.last_seen < timezone.timedelta(
milliseconds=settings.SCRIPT_HEARTBEAT_FREQUENCY * 2
)
@property
def duration(self):
@@ -72,14 +78,20 @@ class Session(models.Model):
kwargs={"pk": self.service.pk, "session_pk": self.uuid},
)
def recalculate_bounce(self):
bounce = self.hit_set.count() == 1
if bounce != self.is_bounce:
self.is_bounce = bounce
self.save()
class Hit(models.Model):
session = models.ForeignKey(Session, on_delete=models.CASCADE, db_index=True)
initial = models.BooleanField(default=True, db_index=True)
# Base request information
start_time = models.DateTimeField(auto_now_add=True, db_index=True)
last_seen = models.DateTimeField(auto_now_add=True)
start_time = models.DateTimeField(default=timezone.now, db_index=True)
last_seen = models.DateTimeField(default=timezone.now, db_index=True)
heartbeats = models.IntegerField(default=0)
tracker = models.TextField(
choices=[("JS", "JavaScript"), ("PIXEL", "Pixel (noscript)")]
@@ -88,12 +100,17 @@ class Hit(models.Model):
# Advanced page information
location = models.TextField(blank=True, db_index=True)
referrer = models.TextField(blank=True, db_index=True)
load_time = models.FloatField(null=True)
load_time = models.FloatField(null=True, db_index=True)
# While not necessary, we store the root service directly for performance.
# It makes querying much easier; no need for inner joins.
service = models.ForeignKey(Service, on_delete=models.CASCADE, db_index=True)
class Meta:
ordering = ["-start_time"]
indexes = [
models.Index(fields=["session", "-start_time"]),
models.Index(fields=["service", "-start_time"]),
models.Index(fields=["session", "location"]),
models.Index(fields=["session", "referrer"]),
]
@@ -105,5 +122,5 @@ class Hit(models.Model):
def get_absolute_url(self):
return reverse(
"dashboard:service_session",
kwargs={"pk": self.session.service.pk, "session_pk": self.session.pk},
kwargs={"pk": self.service.pk, "session_pk": self.session.pk},
)

View File

@@ -1,7 +1,6 @@
import ipaddress
import json
import logging
from hashlib import sha1
from hashlib import sha256
import geoip2.database
import user_agents
@@ -64,7 +63,10 @@ def ingress_request(
try:
remote_ip = ipaddress.ip_network(ip)
for ignored_network in service.get_ignored_networks():
if ignored_network.supernet_of(remote_ip):
if (
ignored_network.version == remote_ip.version
and ignored_network.supernet_of(remote_ip)
):
return
except ValueError as e:
log.exception(e)
@@ -73,9 +75,14 @@ def ingress_request(
if payload.get("loadTime", 1) <= 0:
payload["loadTime"] = None
association_id_hash = sha1()
association_id_hash = sha256()
association_id_hash.update(str(ip).encode("utf-8"))
association_id_hash.update(str(user_agent).encode("utf-8"))
if settings.AGGRESSIVE_HASH_SALTING:
association_id_hash.update(str(service.pk).encode("utf-8"))
association_id_hash.update(
str(timezone.now().date().isoformat()).encode("utf-8")
)
session_cache_path = (
f"session_association_{service.pk}_{association_id_hash.hexdigest()}"
)
@@ -114,18 +121,20 @@ def ingress_request(
return
session = Session.objects.create(
service=service,
ip=ip if service.collect_ips else None,
ip=ip if service.collect_ips and not settings.BLOCK_ALL_IPS else None,
user_agent=user_agent,
identifier=identifier.strip(),
browser=ua.browser.family or "",
device=ua.device.family or ua.device.model or "",
device_type=device_type,
start_time=time,
last_seen=time,
os=ua.os.family or "",
asn=ip_data.get("asn", ""),
country=ip_data.get("country", ""),
asn=ip_data.get("asn") or "",
country=ip_data.get("country") or "",
longitude=ip_data.get("longitude"),
latitude=ip_data.get("latitude"),
time_zone=ip_data.get("time_zone", ""),
time_zone=ip_data.get("time_zone") or "",
)
cache.set(
session_cache_path, session.pk, timeout=settings.SESSION_MEMORY_TIMEOUT
@@ -136,7 +145,7 @@ def ingress_request(
log.debug("Updating old session with new data...")
# Update last seen time
session.last_seen = timezone.now()
session.last_seen = time
if session.identifier == "" and identifier.strip() != "":
session.identifier = identifier.strip()
session.save()
@@ -157,7 +166,7 @@ def ingress_request(
# this is a heartbeat.
log.debug("Hit is a heartbeat; updating old hit with new data...")
hit.heartbeats += 1
hit.last_seen = timezone.now()
hit.last_seen = time
hit.save()
if hit is None:
@@ -173,7 +182,14 @@ def ingress_request(
location=payload.get("location", location),
referrer=payload.get("referrer", ""),
load_time=payload.get("loadTime"),
start_time=time,
last_seen=time,
service=service
)
# Recalculate whether the session is a bounce
session.recalculate_bounce()
# Set idempotency (if applicable)
if idempotency is not None:
cache.set(

View File

@@ -1,8 +1,14 @@
window.onload = function () {
var idempotency =
Math.random().toString(36).substring(2, 15) +
Math.random().toString(36).substring(2, 15);
function sendUpdate() {
// This is a lightweight and privacy-friendly analytics script from Shynet, a self-hosted
// analytics tool. To give you full visibility into how your data is being monitored, this
// file is intentionally not minified or obfuscated. To learn more about Shynet (and to view
// its source code), visit <https://github.com/milesmcc/shynet>.
//
// This script only sends the current URL, the referrer URL, and the page load time. That's it!
var Shynet = {
idempotency: null,
heartbeatTaskId: null,
sendHeartbeat: function () {
try {
if (document.hidden) {
return;
@@ -16,7 +22,7 @@ window.onload = function () {
xhr.setRequestHeader("Content-Type", "application/json");
xhr.send(
JSON.stringify({
idempotency: idempotency,
idempotency: Shynet.idempotency,
referrer: document.referrer,
location: window.location.href,
loadTime:
@@ -24,8 +30,25 @@ window.onload = function () {
window.performance.timing.navigationStart,
})
);
} catch { }
} catch (e) { }
},
newPageLoad: function () {
if (Shynet.heartbeatTaskId != null) {
clearInterval(Shynet.heartbeatTaskId);
}
Shynet.idempotency = Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15);
Shynet.heartbeatTaskId = setInterval(Shynet.sendHeartbeat, parseInt("{{heartbeat_frequency}}"));
Shynet.sendHeartbeat();
}
setInterval(sendUpdate, parseInt("{{heartbeat_frequency}}"));
sendUpdate();
};
window.addEventListener("load", Shynet.newPageLoad);
{% if script_inject %}
// The following is script is not part of Shynet, and was instead
// provided by this site's administrator.
//
// -- START --
{{script_inject|safe}}
// -- END --
{% endif %}

View File

@@ -1,10 +1,16 @@
import base64
import json
from urllib.parse import urlparse
from django.conf import settings
from django.core.cache import cache
from django.core.exceptions import ValidationError
from django.http import Http404, HttpResponse, HttpResponseBadRequest
from django.http import (
Http404,
HttpResponse,
HttpResponseBadRequest,
HttpResponseForbidden,
)
from django.shortcuts import render, reverse
from django.utils import timezone
from django.utils.decorators import method_decorator
@@ -49,7 +55,23 @@ class ValidateServiceOriginsMixin:
cache.set(f"service_origins_{service_uuid}", origins, timeout=3600)
resp = super().dispatch(request, *args, **kwargs)
resp["Access-Control-Allow-Origin"] = origins
if origins != "*":
remote_origin = request.META.get("HTTP_ORIGIN")
if (
remote_origin is None
and request.META.get("HTTP_REFERER") is not None
):
parsed = urlparse(request.META.get("HTTP_REFERER"))
remote_origin = f"{parsed.scheme}://{parsed.netloc}".lower()
origins = [origin.strip().lower() for origin in origins.split(",")]
if remote_origin in origins:
resp["Access-Control-Allow-Origin"] = remote_origin
else:
return HttpResponseForbidden()
else:
resp["Access-Control-Allow-Origin"] = "*"
resp["Access-Control-Allow-Methods"] = "GET,HEAD,OPTIONS,POST"
resp[
"Access-Control-Allow-Headers"
@@ -78,7 +100,7 @@ class PixelView(ValidateServiceOriginsMixin, View):
"R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw=="
)
resp = HttpResponse(data, content_type="image/gif")
resp["Cache-Control"] = "no-cache"
resp["Cache-Control"] = "no-cache, no-store, must-revalidate"
resp["Access-Control-Allow-Origin"] = "*"
return resp
@@ -105,11 +127,14 @@ class ScriptView(ValidateServiceOriginsMixin, View):
return render(
self.request,
"analytics/scripts/page.js",
context={
"endpoint": endpoint,
"protocol": protocol,
"heartbeat_frequency": heartbeat_frequency,
},
context=dict(
{
"endpoint": endpoint,
"protocol": protocol,
"heartbeat_frequency": heartbeat_frequency,
"script_inject": self.get_script_inject(),
}
),
content_type="application/javascript",
)
@@ -125,3 +150,12 @@ class ScriptView(ValidateServiceOriginsMixin, View):
return HttpResponse(
json.dumps({"status": "OK"}), content_type="application/json"
)
def get_script_inject(self):
service_uuid = self.kwargs.get("service_uuid")
script_inject = cache.get(f"script_inject_{service_uuid}")
if script_inject == None:
service = Service.objects.get(uuid=service_uuid)
script_inject = service.script_inject
cache.set(f"script_inject_{service_uuid}", script_inject, timeout=3600)
return script_inject

View File

@@ -0,0 +1,114 @@
import traceback
from django.utils.timezone import now
from django.utils.timezone import timedelta
import random
import uuid
from django.conf import settings
from django.contrib.sites.models import Site
from django.core.management.base import BaseCommand, CommandError
from django.utils.crypto import get_random_string
import user_agents
from logging import info
from core.models import User, Service
from analytics.models import Session, Hit
from analytics.tasks import ingress_request
LOCATIONS = [
"/",
"/post/{rand}",
"/login",
"/me",
]
REFERRERS = [
"https://news.ycombinator.com/item?id=11116274",
"https://news.ycombinator.com/item?id=24872911",
"https://reddit.com",
"https://facebook.com",
"https://twitter.com/milesmccain",
"https://twitter.com",
"https://stanford.edu/~mccain/",
"https://tiktok.com",
"https://io.stanford.edu",
"https://en.wikipedia.org",
"https://stackoverflow.com",
"",
"",
"",
"",
]
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:42.0) Gecko/20100101 Firefox/43.4",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
"Mozilla/5.0 (iPhone; CPU iPhone OS 11_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko)",
"Version/10.0 Mobile/14E304 Safari/602.1",
]
class Command(BaseCommand):
help = "Configures a Shynet demo service"
def add_arguments(self, parser):
parser.add_argument(
"name", type=str,
)
parser.add_argument("owner_email", type=str)
parser.add_argument(
"avg", type=int,
)
parser.add_argument("deviation", type=float, default=0.4)
parser.add_argument(
"days", type=int,
)
parser.add_argument("load_time", type=float, default=1000)
def handle(self, *args, **options):
owner = User.objects.get(email=options.get("owner_email"))
service = Service.objects.create(name=options.get("name"), owner=owner)
print(
f"Created demo service `{service.name}` (uuid: `{service.uuid}`, owner: {owner})"
)
# Go through each day requested, creating sessions and hits
for days in range(options.get("days")):
day = (now() - timedelta(days=days)).replace(hour=0, minute=0, second=0)
print(f"Populating info for {day}...")
avg = options.get("avg")
deviation = options.get("deviation")
ips = [
".".join(map(str, (random.randint(0, 255) for _ in range(4))))
for _ in range(avg)
]
n = avg + random.randrange(-1 * deviation * avg, deviation * avg)
for _ in range(n):
time = day + timedelta(
hours=random.randrange(0, 23),
minutes=random.randrange(0, 59),
seconds=random.randrange(0, 59),
)
ip = random.choice(ips)
load_time = random.normalvariate(options.get("load_time"), 500)
referrer = random.choice(REFERRERS)
location = "https://example.com" + random.choice(LOCATIONS).replace(
"{rand}", str(random.randint(0, n))
)
user_agent = random.choice(USER_AGENTS)
ingress_request(
service.uuid,
"JS",
time,
{"loadTime": load_time, "referrer": referrer},
ip,
location,
user_agent,
)
print(f"Created {n} demo hits on {day}!")
self.stdout.write(self.style.SUCCESS(f"Successfully created demo data!"))

View File

@@ -20,6 +20,12 @@ class Command(BaseCommand):
def handle(self, *args, **options):
site = Site.objects.get(pk=settings.SITE_ID)
site.domain = options.get("hostname")
if options.get("hostname").lower().startswith("http"):
self.stdout.write(
self.style.WARNING(
f"Warning: the hostname '{options.get('hostname')}' starts with `http`. You almost certainly don't want this. The hostname is supposed to be the raw domain name of your Shynet instance, without `http://` or `https://`. For example, if your Shynet instance will eventually be hosted at `https://analytics.example.com`, the hostname should be `analytics.example.com`."
)
)
site.save()
self.stdout.write(
self.style.SUCCESS(

View File

@@ -13,7 +13,7 @@ from core.models import User
class Command(BaseCommand):
help = "Internal command to perform startup sanity checks."
help = "Internal command to perform startup checks."
def check_migrations(self):
from django.db.migrations.executor import MigrationExecutor

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0006_service_hide_referrer_regex'),
("core", "0006_service_hide_referrer_regex"),
]
operations = [
migrations.AddField(
model_name='service',
name='ignore_robots',
model_name="service",
name="ignore_robots",
field=models.BooleanField(default=False),
)
]

View File

@@ -0,0 +1,25 @@
# Generated by Django 3.1b1 on 2020-06-28 18:03
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0007_service_ignore_robots"),
]
operations = [
migrations.AddField(
model_name="service",
name="script_inject",
field=models.TextField(blank=True, default=""),
),
migrations.AlterField(
model_name="user",
name="first_name",
field=models.CharField(
blank=True, max_length=150, verbose_name="first name"
),
),
]

View File

@@ -73,6 +73,7 @@ class Service(models.Model):
hide_referrer_regex = models.TextField(
default="", blank=True, validators=[_validate_regex]
)
script_inject = models.TextField(default="", blank=True)
class Meta:
ordering = ["name", "uuid"]
@@ -128,11 +129,11 @@ class Service(models.Model):
session_count = sessions.count()
hits = Hit.objects.filter(
session__service=self, start_time__lt=end_time, start_time__gt=start_time
service=self, start_time__lt=end_time, start_time__gt=start_time
)
hit_count = hits.count()
bounces = sessions.annotate(hit_count=models.Count("hit")).filter(hit_count=1)
bounces = sessions.filter(is_bounce=True)
bounce_count = bounces.count()
locations = (
@@ -234,7 +235,9 @@ class Service(models.Model):
"session_chart_data": json.dumps(
[
{"x": str(key), "y": value}
for key, value in session_chart_data.items()
for key, value in sorted(
session_chart_data.items(), key=lambda k: k[0]
)
]
),
"online": True,

View File

@@ -1,5 +1,6 @@
from allauth.account.admin import EmailAddress
from django import forms
from django.conf import settings
from django.utils.translation import gettext_lazy as _
from core.models import Service, User
@@ -18,6 +19,7 @@ class ServiceForm(forms.ModelForm):
"hide_referrer_regex",
"origins",
"collaborators",
"script_inject",
]
widgets = {
"name": forms.TextInput(),
@@ -27,35 +29,54 @@ class ServiceForm(forms.ModelForm):
"collect_ips": forms.RadioSelect(choices=[(True, "Yes"), (False, "No")]),
"ignore_robots": forms.RadioSelect(choices=[(True, "Yes"), (False, "No")]),
"hide_referrer_regex": forms.TextInput(),
"script_inject": forms.Textarea(attrs={"class": "font-mono", "rows": 5}),
}
labels = {
"origins": "Allowed Hostnames",
"origins": "Allowed origins",
"respect_dnt": "Respect DNT",
"collect_ips": "Collect IP addresses",
"ignored_ips": "Ignored IP addresses",
"ignore_robots": "Ignore robots",
"hide_referrer_regex": "Hide specific referrers",
"script_inject": "Additional injected JS",
}
help_texts = {
"name": _("What should the service be called?"),
"link": _("What's the service's primary URL?"),
"origins": _(
"At what hostnames does the service operate? This sets CORS headers, so use '*' if you're not sure (or don't care)."
"At what origins does the service operate? Use commas to separate multiple values. This sets CORS headers, so use '*' if you're not sure (or don't care)."
),
"respect_dnt": "Should visitors who have enabled <a href='https://en.wikipedia.org/wiki/Do_Not_Track'>Do Not Track</a> be excluded from all data?",
"collect_ips": "Should individual IP addresses be collected? IP metadata (location, host, etc) will still be collected.",
"ignored_ips": "A comma-separated list of IP addresses or IP ranges (IPv4 and IPv6) to exclude from tracking (e.g., '192.168.0.2, 127.0.0.1/32').",
"ignore_robots": "Should sessions generated by bots be excluded from tracking?",
"hide_referrer_regex": "Any referrers that match this <a href='https://regexr.com/'>RegEx</a> will not be listed in the referrer summary. Sessions will still be tracked normally. No effect if left blank.",
"script_inject": "Optional additional JavaScript to inject at the end of the Shynet script. This code will be injected on every page where this service is installed.",
}
collect_ips = forms.BooleanField(
help_text="IP address collection is disabled globally by your administrator."
if settings.BLOCK_ALL_IPS
else "Should individual IP addresses be collected? IP metadata (location, host, etc) will still be collected.",
widget=forms.RadioSelect(choices=[(True, "Yes"), (False, "No")]),
initial=False if settings.BLOCK_ALL_IPS else True,
required=False,
disabled=settings.BLOCK_ALL_IPS,
)
def clean_collect_ips(self):
collect_ips = self.cleaned_data["collect_ips"]
# Forces collect IPs to be false if it is disabled globally
return False if settings.BLOCK_ALL_IPS else collect_ips
collaborators = forms.CharField(
help_text="Which users should have read-only access to this service? (Comma separated list of emails.)",
help_text="Which users on this Shynet instance should have read-only access to this service? (Comma separated list of emails.)",
required=False,
)
def clean_collaborators(self):
collaborators = []
users_to_emails = (
{}
) # maps users to the email they are listed under as a collaborator
for collaborator_email in self.cleaned_data["collaborators"].split(","):
email = collaborator_email.strip()
if email == "":
@@ -65,6 +86,12 @@ class ServiceForm(forms.ModelForm):
).first()
if collaborator_email_linked is None:
raise forms.ValidationError(f"Email '{email}' is not registered")
user = collaborator_email_linked.user
if user in collaborators:
raise forms.ValidationError(
f"The emails '{email}' and '{users_to_emails[user]}' both correspond to the same user"
)
users_to_emails[user] = email
collaborators.append(collaborator_email_linked.user)
return collaborators

View File

@@ -13,4 +13,18 @@
.rf {
text-align: right !important;
}
:root {
--color-neutral-000: white;
--color-neutral-50: #F8FAFC;
--color-neutral-100: #F1F5F9;
--color-neutral-200: #E2E8F0;
--color-neutral-300: #CBD5E1;
--color-neutral-400: #94A3B8;
--color-neutral-500: #64748B;
--color-neutral-600: #475569;
--color-neutral-700: #334155;
--color-neutral-800: #1E293B;
--color-neutral-900: #0F172A;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

View File

@@ -7,24 +7,25 @@
<title>{% block head_title %}Privacy-oriented analytics{% endblock %} | {{request.site.name}}</title>
<meta name="robots" content="noindex">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
{% include 'a17t/head.html' %}
<script src="https://cdn.jsdelivr.net/npm/litepicker@1.2.0/dist/js/main.js"
integrity="sha256-mOlCEHUNWZPYIrc5OFL4Ab2rsJGzIPld3cy1ok7Cfx0=" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/apexcharts@3.18.1/dist/apexcharts.min.js"
integrity="sha256-RalQXBZdisB04aaBsm+6YZ0b/iRYjX1MZn90m19AnCY=" crossorigin="anonymous"></script>
{% include 'a17t/includes/head.html' %}
<link rel="icon" type="image/png" href="{% static 'dashboard/images/icon.png' %}">
<script src="{% static 'apexcharts/dist/apexcharts.min.js'%}"></script>
<script src="{% static 'litepicker/dist/js/main.js' %}"></script>
<script src="{% static 'dashboard/js/base.js' %}"></script>
<link rel="stylesheet" href="{% static 'dashboard/css/global.css' %}">
{% block extra_head %}
{% endblock %}
</head>
<body class="bg-gray-200 min-h-full">
<body class="bg-neutral-100 min-h-full">
{% block body %}
<section class="max-w-screen-xl mx-auto px-4 py-4 md:py-12 md:flex">
<aside class="mb-8 md:w-2/12 md:pr-6 relative flex flex-wrap md:block justify-between items-center overflow-x-hidden">
<aside
class="mb-8 md:w-2/12 md:pr-6 relative flex flex-wrap md:block justify-between items-center overflow-x-hidden">
<a class="icon ~urge ml-2 md:ml-6 md:mb-8 md:mt-3" href="{% url 'dashboard:dashboard' %}">
<i class="fas fa-binoculars fa-3x text-purple-600 hidden md:block"></i>
<i class="fas fa-binoculars fa-2x text-purple-600 md:hidden"></i>
<i class="fas fa-binoculars fa-3x text-urge-600 hidden md:block"></i>
<i class="fas fa-binoculars fa-2x text-urge-600 md:hidden"></i>
</a>
<button class="button ~neutral !low md:hidden"
@@ -35,13 +36,13 @@
</button>
<hr class="sep h-4 md:h-8 w-full">
<div id="navMenuExpanded"
class="bg-white shadow-lg md:shadow-none p-4 hidden rounded-lg md:block md:bg-transparent md:border-none md:p-0 w-full">
class="bg-neutral-000 shadow-lg md:shadow-none p-4 hidden rounded-lg md:block md:bg-transparent md:border-none md:p-0 w-full">
{% if user.owning_services.all %}
<p class="ml-2 mb-1 supra font-medium text-gray-500 pointer-events-none">Services</p>
{% for service in user.owning_services.all %}
{% url 'dashboard:service' service.uuid as url %}
{% include 'dashboard/includes/sidebar_portal.html' with label=service.name|truncatechars:20 url=url %}
{% include 'dashboard/includes/sidebar_portal.html' with label=service.name|truncatechars:16 url=url icon=service.link|iconify %}
{% endfor %}
{% endif %}
@@ -72,7 +73,7 @@
{% if user.is_superuser %}
{% url 'admin:index' as url %}
{% include 'dashboard/includes/sidebar_portal.html' with label="Admin" url=url %}
{% include 'dashboard/includes/sidebar_portal.html' with label="Admin" disable_turbolinks=True url=url %}
{% endif %}
{% url 'account_email' as url %}

View File

@@ -2,7 +2,7 @@
<input type="hidden" name="startDate" value="{{start_date.isoformat}}" id="startDate">
<input type="hidden" name="endDate" value="{{end_date.isoformat}}" id="endDate">
</form>
<input type="input" id="rangePicker" placeholder="Date range" class="input ~neutral cursor-pointer" readonly>
<input type="input" id="rangePicker" placeholder="Date range" class="input ~neutral bg-neutral-000 cursor-pointer" readonly>
<style>
:root {
--litepickerMonthButtonHover: var(--color-urge);

View File

@@ -13,4 +13,5 @@
{{form.ignore_robots|a17t}}
{{form.hide_referrer_regex|a17t}}
{{form.origins|a17t}}
{{form.script_inject|a17t}}
</details>

View File

@@ -4,8 +4,9 @@
{% with stats=object.stats %}
<div class="p-4 md:flex justify-between">
<div class="flex items-center mb-4 md:mb-0">
<h3 class="heading text-xl md:text-2xl mr-2 mb-1 text-purple-600">
{{object.name}}
<h3 class="heading text-xl md:text-2xl mr-2 mb-1 text-urge-600 flex items-center">
{{object.link|iconify}}
<span>{{object.name}}</span>
</h3>
{% include 'dashboard/includes/stats_status_chip.html' %}
</div>

View File

@@ -13,7 +13,7 @@
<tr>
<td>
<a href="{% url 'dashboard:service_session' object.pk session.pk %}"
class="font-medium text-purple-700">
class="font-medium text-urge-700">
{{session.start_time|date:"M j Y, g:i a"|capfirst}}
{% if session.is_currently_active %}
<span class="badge ~positive">Online</span>

View File

@@ -1,6 +1,6 @@
{% load helpers %}
<div>
<a class="portal !low {% if request.get_full_path|startswith:url %}~urge active bg-gray-100{% endif %}"
href="{{url}}">{{label}}</a>
<a class="portal !low {% if request.get_full_path|startswith:url %}~urge active bg-neutral-100{% endif %} flex items-center"
{% if disable_turbolinks %}data-turbolinks="false"{% endif %} href="{{url}}">{{icon}} {{label}}</a>
</div>

View File

@@ -1,6 +1,6 @@
{% extends "base.html" %}
{% load rules %}
{% load rules pagination %}
{% block content %}
<div class="md:flex justify-between items-center">
@@ -13,14 +13,17 @@
</div>
{% has_perm "core.create_service" user as can_create %}
{% if can_create %}
<a href="{% url 'dashboard:service_create' %}" class="button field w-auto">+ New Service</a>
<a href="{% url 'dashboard:service_create' %}" class="button field bg-neutral-000 w-auto">+ New Service</a>
{% endif %}
</div>
</div>
<hr class="sep">
{% for object in services|dictsortreversed:"stats.session_count" %}
{% for object in object_list|dictsortreversed:"stats.session_count" %}
{% include 'dashboard/includes/service_overview.html' %}
{% empty %}
<p>You don't have any services on {{request.site.name|default:"Shynet"}} yet.</p>
{% endfor %}
{% pagination page_obj request %}
{% endblock %}

View File

@@ -6,29 +6,29 @@
<div class="mr-2">{% include 'dashboard/includes/date_range.html' %}</div>
{% has_perm 'core.change_service' user object as can_update %}
{% if can_update %}
<a href="{% url 'dashboard:service_update' service.uuid %}" class="button field ~neutral w-auto">Manage &rarr;</a>
<a href="{% url 'dashboard:service_update' service.uuid %}" class="button field bg-neutral-000 w-auto">Manage &rarr;</a>
{% endif %}
{% endblock %}
{% block service_content %}
<div class="grid grid-cols-2 gap-6 md:flex justify-between mb-6 card ~neutral !high px-6" id="stats">
{% with classes="text-sm font-semibold" good_classes="text-green-400" bad_classes="text-red-400" neutral_classes="text-gray-400" %}
{% with classes="text-sm font-semibold" good_classes="text-positive-400" bad_classes="text-critical-400" neutral_classes="text-gray-400" %}
<article class="">
<p class="label text-gray-400">Sessions</p>
<p class="heading">
{{stats.session_count|intcomma}}
<div>
{% compare stats.compare.session_count stats.session_count "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.session_count stats.session_count "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
<article class="">
<p class="label text-gray-400">Hits</p>
<p class="heading">
{{stats.hit_count|intcomma}}
<div>
{% compare stats.compare.hit_count stats.hit_count "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.hit_count stats.hit_count "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
<article class="">
@@ -39,9 +39,9 @@
{% else %}
?
{% endif %}
<div>
{% compare stats.compare.avg_load_time stats.avg_load_time "DOWN" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.avg_load_time stats.avg_load_time "DOWN" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
<article class="">
@@ -52,9 +52,9 @@
{% else %}
?
{% endif %}
<div>
{% compare stats.compare.bounce_rate_pct stats.bounce_rate_pct "DOWN" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.bounce_rate_pct stats.bounce_rate_pct "DOWN" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
<article class="">
@@ -65,9 +65,9 @@
{% else %}
?
{% endif %}
<div>
{% compare stats.compare.avg_session_duration stats.avg_session_duration "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.avg_session_duration stats.avg_session_duration "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
<article class="">
@@ -78,9 +78,9 @@
{% else %}
?
{% endif %}
<div>
{% compare stats.compare.avg_hits_per_session stats.avg_hits_per_session "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
<div>
{% compare stats.compare.avg_hits_per_session stats.avg_hits_per_session "UP" classes=classes good_classes=good_classes bad_classes=bad_classes neutral_classes=neutral_classes %}
</div>
</p>
</article>
{% endwith %}
@@ -166,7 +166,7 @@
<tbody>
{% for os in stats.operating_systems %}
<tr>
<td>{{os.os|default:"Unknown"}}</td>
<td class="flex items-center">{{os.os|iconify}}<span>{{os.os|default:"Unknown"}}</span></td>
<td class="rf">{{os.count|intcomma}}</td>
</tr>
{% empty %}
@@ -188,7 +188,8 @@
<tbody>
{% for browser in stats.browsers %}
<tr>
<td>{{browser.browser|default:"Unknown"}}</td>
<td class="flex items-center">
{{browser.browser|iconify}}<span>{{browser.browser|default:"Unknown"}}</span></td>
<td class="rf">{{browser.count|intcomma}}</td>
</tr>
{% empty %}
@@ -225,7 +226,8 @@
<div class="card ~neutral !low limited-height py-2">
{% include 'dashboard/includes/session_list.html' %}
<hr class="sep h-8">
<a href="{% url 'dashboard:service_session_list' service.uuid %}" class="button ~neutral w-auto mb-2">View more sessions
<a href="{% url 'dashboard:service_session_list' service.uuid %}" class="button ~neutral w-auto mb-2">View more
sessions
&rarr;</a>
</div>
{% endblock %}

View File

@@ -5,7 +5,7 @@
{% block head_title %}{{object.name}} Session{% endblock %}
{% block service_actions %}
<a href="{% url 'dashboard:service' object.uuid %}" class="button field ~neutral w-auto">Analytics &rarr;</a>
<a href="{% url 'dashboard:service' object.uuid %}" class="button field bg-neutral-000 w-auto">Analytics &rarr;</a>
{% endblock %}
{% block service_content %}

View File

@@ -6,7 +6,7 @@
{% block service_actions %}
<div class="mr-2">{% include 'dashboard/includes/date_range.html' %}</div>
<a href="{% url 'dashboard:service' object.uuid %}" class="button field ~neutral w-auto">Analytics &rarr;</a>
<a href="{% url 'dashboard:service' object.uuid %}" class="button field ~neutral bg-neutral-000 w-auto">Analytics &rarr;</a>
{% endblock %}
{% block service_content %}

View File

@@ -5,7 +5,7 @@
{% block head_title %}{{object.name}} Management{% endblock %}
{% block service_actions %}
<a href="{% url 'dashboard:service' object.uuid %}" class="button field ~neutral w-auto">View &rarr;</a>
<a href="{% url 'dashboard:service' object.uuid %}" class="button field bg-neutral-000 w-auto">View &rarr;</a>
{% endblock %}
{% block service_content %}
@@ -15,7 +15,7 @@
<div class="card ~neutral !high font-mono text-sm">
{% filter force_escape %}<noscript><img
src="{{script_protocol}}{{request.site.domain}}{% url 'ingress:endpoint_pixel' object.uuid %}"></noscript>
<script src="{{script_protocol}}{{request.site.domain}}{% url 'ingress:endpoint_script' object.uuid %}"></script>
<script defer src="{{script_protocol}}{{request.site.domain}}{% url 'ingress:endpoint_script' object.uuid %}"></script>
{% endfilter %}
</div>
<hr class="sep h-4">
@@ -36,4 +36,4 @@
</div>
</form>
</div>
{% endblock %}
{% endblock %}

View File

@@ -8,6 +8,7 @@
<div class="md:flex justify-between items-center" id="heading">
<a class="flex items-center mb-4 md:mb-0" href="{% url 'dashboard:service' object.uuid %}">
<h3 class="heading leading-none mr-4">
{{object.link|iconify}}
{{object.name}}
</h3>
<div class='text-3xl'>

View File

@@ -1,3 +1,4 @@
from datetime import timedelta
from urllib.parse import urlparse
import flag
@@ -73,7 +74,7 @@ def percent_change_display(start, end):
elif start == 0:
pct_change = "0%"
else:
change = int(round(100 * abs(end - start) / start))
change = int(round(100 * abs(end - start) / max(start, 1)))
if change > 999:
return "> 999%"
else:
@@ -84,7 +85,7 @@ def percent_change_display(start, end):
@register.inclusion_tag("dashboard/includes/sidebar_footer.html")
def sidebar_footer():
return {"version": settings.VERSION}
return {"version": settings.VERSION if settings.SHOW_SHYNET_VERSION else ""}
@register.inclusion_tag("dashboard/includes/stat_comparison.html")
@@ -97,6 +98,12 @@ def compare(
bad_classes=None,
neutral_classes=None,
):
if isinstance(start, timedelta):
start = start.seconds
if isinstance(end, timedelta):
end = end.seconds
return {
"start": start,
"end": end,
@@ -115,12 +122,60 @@ def startswith(text, starts):
return False
@register.filter
def iconify(text):
if not settings.SHOW_THIRD_PARTY_ICONS:
return ""
text = text.lower()
icons = {
"chrome": "chrome.com",
"safari": "www.apple.com",
"windows": "windows.com",
"edge": "microsoft.com",
"firefox": "firefox.com",
"opera": "opera.com",
"unknown": "example.com",
"linux": "kernel.org",
"ios": "www.apple.com",
"mac": "www.apple.com",
"macos": "www.apple.com",
"mac os x": "www.apple.com",
"android": "android.com",
"chrome os": "chrome.com",
"ubuntu": "ubuntu.com",
"fedora": "getfedora.org",
"mobile safari": "www.apple.com",
"chrome mobile ios": "chrome.com",
"chrome mobile": "chrome.com",
"samsung internet": "samsung.com",
"google": "google.com",
"chrome mobile webview": "chrome.com",
"firefox mobile": "firefox.com",
"edge mobile": "microsoft.com",
"chromium": "chromium.org",
}
domain = None
if text.startswith("http"):
domain = urlparse(text).netloc
elif text in icons:
domain = icons[text]
else:
# This fallback works better than you'd think!
domain = text + ".com"
return SafeString(
f'<span class="icon mr-1"><img src="https://icons.duckduckgo.com/ip3/{domain}.ico"></span>'
)
@register.filter
def urldisplay(url):
if url.startswith("http"):
display_url = url.replace("http://", "").replace("https://", "")
return SafeString(
f"<a href='{url}' title='{url}' rel='nofollow'>{escape(display_url if len(display_url) < 40 else display_url[:40] + '...')}</a>"
f"<a href='{url}' title='{url}' rel='nofollow' class='flex items-center'>{iconify(url)} {escape(display_url if len(display_url) < 40 else display_url[:40] + '...')}</a>"
)
else:
return url

View File

@@ -22,16 +22,24 @@ from .forms import ServiceForm
from .mixins import DateRangeMixin
class DashboardView(LoginRequiredMixin, DateRangeMixin, TemplateView):
class DashboardView(
LoginRequiredMixin, DateRangeMixin, ListView
):
model = Service
template_name = "dashboard/pages/dashboard.html"
paginate_by = 5
def get_queryset(self):
return Service.objects.filter(
Q(owner=self.request.user) | Q(collaborators__in=[self.request.user])
).distinct()
def get_context_data(self, **kwargs):
data = super().get_context_data(**kwargs)
data["services"] = Service.objects.filter(
Q(owner=self.request.user) | Q(collaborators__in=[self.request.user])
)
for service in data["services"]:
service.stats = service.get_core_stats(data["start_date"], data["end_date"])
for service in data["object_list"]:
service.stats = service.get_core_stats(self.get_start_date(), self.get_end_date())
return data
@@ -84,6 +92,9 @@ class ServiceUpdateView(
cache.set(
f"service_origins_{self.object.uuid}", self.object.origins, timeout=3600
)
cache.set(
f"script_inject_{self.object.uuid}", self.object.script_inject, timeout=3600
)
return resp
def get_context_data(self, *args, **kwargs):
@@ -136,6 +147,9 @@ class ServiceSessionView(LoginRequiredMixin, PermissionRequiredMixin, DetailView
context_object_name = "session"
permission_required = "core.view_service"
def get_permission_object(self, **kwargs):
return self.get_object().service
def get_context_data(self, **kwargs):
data = super().get_context_data(**kwargs)
data["object"] = get_object_or_404(Service, pk=self.kwargs.get("pk"))

View File

@@ -9,6 +9,7 @@ https://docs.djangoproject.com/en/2.2/ref/settings/
"""
import os
# import module sys to get the type of exception
import sys
import urllib.parse as urlparse
@@ -17,7 +18,7 @@ import urllib.parse as urlparse
from django.contrib.messages import constants as messages
# Increment on new releases
VERSION = "v0.5.2"
VERSION = "v0.8.0"
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -57,6 +58,7 @@ INSTALLED_APPS = [
"allauth",
"allauth.account",
"allauth.socialaccount",
"debug_toolbar"
]
MIDDLEWARE = [
@@ -69,6 +71,7 @@ MIDDLEWARE = [
"django.contrib.sites.middleware.CurrentSiteMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
"debug_toolbar.middleware.DebugToolbarMiddleware",
]
ROOT_URLCONF = "shynet.urls"
@@ -116,24 +119,26 @@ else:
}
# Solution to removal of Heroku DB Injection
if 'DATABASE_URL' in os.environ:
if 'DATABASES' not in locals():
if "DATABASE_URL" in os.environ:
if "DATABASES" not in locals():
DATABASES = {}
url = urlparse.urlparse(os.environ['DATABASE_URL'])
url = urlparse.urlparse(os.environ["DATABASE_URL"])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
DATABASES["default"] = DATABASES.get("default", {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
DATABASES["default"].update(
{
"NAME": url.path[1:],
"USER": url.username,
"PASSWORD": url.password,
"HOST": url.hostname,
"PORT": url.port,
}
)
if url.scheme == "postgres":
DATABASES["default"]["ENGINE"] = "django.db.backends.postgresql_psycopg2"
# Password validation
# https://docs.djangoproject.com/en/2.2/ref/settings/#auth-password-validators
@@ -202,6 +207,11 @@ USE_TZ = True
STATIC_URL = "/static/"
STATIC_ROOT = "compiledstatic/"
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
STATICFILES_FINDERS = [
"npm.finders.NpmFinder",
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
# Redis
if not DEBUG and os.getenv("REDIS_CACHE_LOCATION") is not None:
@@ -239,6 +249,10 @@ LOGIN_REDIRECT_URL = "/"
SITE_ID = 1
INTERNAL_IPS = [
'127.0.0.1',
]
# Celery
CELERY_TASK_ALWAYS_EAGER = os.getenv("CELERY_TASK_ALWAYS_EAGER", "True") == "True"
@@ -271,7 +285,22 @@ else:
EMAIL_PORT = int(os.environ.get("EMAIL_PORT", 465))
EMAIL_HOST_USER = os.environ.get("EMAIL_HOST_USER")
EMAIL_HOST_PASSWORD = os.environ.get("EMAIL_HOST_PASSWORD")
EMAIL_USE_SSL = True
EMAIL_USE_SSL = os.environ.get("EMAIL_USE_SSL")
EMAIL_USE_TLS = os.environ.get("EMAIL_USE_TLS")
# NPM
NPM_ROOT_PATH = "../"
NPM_FILE_PATTERNS = {
"a17t": [os.path.join("dist", "a17t.css"), os.path.join("dist", "tailwind.css")],
"apexcharts": [os.path.join("dist", "apexcharts.min.js")],
"litepicker": [os.path.join("dist", "js", "main.js")],
"turbolinks": [os.path.join("dist", "turbolinks.js")],
"stimulus": [os.path.join("dist", "stimulus.umd.js")],
"inter-ui": [os.path.join("Inter (web)", "*")],
"@fortawesome": [os.path.join("fontawesome-free", "js", "all.min.js")],
}
# Shynet
@@ -292,3 +321,15 @@ SCRIPT_HEARTBEAT_FREQUENCY = int(os.getenv("SCRIPT_HEARTBEAT_FREQUENCY", "5000")
# How much time can elapse between requests from the same user before a new
# session is created, in seconds?
SESSION_MEMORY_TIMEOUT = int(os.getenv("SESSION_MEMORY_TIMEOUT", "1800"))
# Should the Shynet version information be displayed?
SHOW_SHYNET_VERSION = os.getenv("SHOW_SHYNET_VERSION", "True") == "True"
# Should Shynet show third-party icons in the dashboard?
SHOW_THIRD_PARTY_ICONS = os.getenv("SHOW_THIRD_PARTY_ICONS", "True") == "True"
# Should Shynet never collect any IP?
BLOCK_ALL_IPS = os.getenv("BLOCK_ALL_IPS", "False") == "True"
# Include date and service ID in salt?
AGGRESSIVE_HASH_SALTING = os.getenv("AGGRESSIVE_HASH_SALTING", "False") == True

View File

@@ -15,12 +15,14 @@ Including another URLconf
"""
from django.contrib import admin
from django.urls import include, path
import debug_toolbar
urlpatterns = [
path('__debug__/', include(debug_toolbar.urls)),
path("admin/", admin.site.urls),
path("accounts/", include("allauth.urls")),
path("ingress/", include(("analytics.ingress_urls", "ingress")), name="ingress"),
path("dashboard/", include(("dashboard.urls", "dashboard"), namespace="dashboard")),
path("healthz/", include('health_check.urls')),
path("healthz/", include("health_check.urls")),
path("", include(("core.urls", "core"), namespace="core")),
]

View File

@@ -1,8 +1,8 @@
#!/bin/bash
# Check if setup is necessary, do setup as needed
echo "Performing startup checks..."
sanity_results=( $(./manage.py startup_checks) )
if [[ ${sanity_results[0]} == True ]]; then
startup_results=( $(./manage.py startup_checks) )
if [[ ${startup_results[0]} == True ]]; then
echo "Running migrations (setting up DB)..."
{
./manage.py migrate && echo "Migrations complete!"
@@ -12,13 +12,13 @@ if [[ ${sanity_results[0]} == True ]]; then
else
echo "Database is ready to go."
fi
if [[ ${sanity_results[1]} == True ]]; then
if [[ ${startup_results[1]} == True ]]; then
echo "Warning: no admin user available. Consult docs for instructions."
fi
if [[ ${sanity_results[2]} == True ]]; then
if [[ ${startup_results[2]} == True ]]; then
echo "Warning: Shynet's hostname is not set. The script won't work correctly. Consult docs for instructions."
fi
if [[ ${sanity_results[3]} == True ]]; then
if [[ ${startup_results[3]} == True ]]; then
echo "Warning: Shynet's whitelabel is not set. Consult docs for instructions."
fi
echo "Startup checks complete!"

13
tests/js.html Normal file
View File

@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html>
<head>
<title>Pixel test</title>
</head>
<body>
<script src="http://localhost:8000/ingress/9b2c4e2f-8d29-4418-82d4-b68e06795025/script.js"></script>
</body>
</html>

View File

@@ -7,8 +7,7 @@
</head>
<body>
<noscript><img src="//localhost:8000/ingress/test_uuid/pixel.gif"></noscript>
<script src="//localhost:8000/ingress/66015ce4-c69d-40fb-be8f-5535538d795e/script.js"></script>
<img src="http://localhost:8000/ingress/9b2c4e2f-8d29-4418-82d4-b68e06795025/pixel.gif">
</body>
</html>