Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

TeslaMate [megathread]

This site may earn commission on affiliate links.
Just the default home directory from when I ssh in, though I did use sudo docker-compose up when I first ran it, not sure how that would have made a difference though. I still can't get anywhere with the grafana side of things though!
 
Well I know why it's not showing the correct certs now, not quite sure how to fix it yet :) - running docker-compose without the d gave me the below (I just replaced the MYDOMAIN bit for this before anyone wondered if that was my issue:

proxy_1 | time="2020-07-01T13:07:39Z" level=error msg="Unable to obtain ACME certificate for domains \"grafana.MYDOMAIN.com\": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.11:53: read udp 127.0.0.1:56586->127.0.0.11:53: read: connection refused" routerName=grafana@docker rule="Host(`grafana.MYDOMAIN.com`)" providerName=tmhttpchallenge.acmeproxy_1 | time="2020-07-01T13:07:39Z" level=error msg="Unable to obtain ACME certificate for domains \"teslamate.MYDOMAIN.com\": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.11:53: read udp 127.0.0.1:35575->127.0.0.11:53: read: connection refused" providerName=tmhttpchallenge.acme routerName=teslamate@docker rule="Host(`teslamate.MYDOMAIN.com`)"
 
It's vaguely possible that something recent has changed in the pulls for one of the containers which is stopping it working, perhaps?

This is my unedited docker-compose.yml file:
Code:
version: "3"

services:
  teslamate:
    image: teslamate/teslamate:latest
    restart: always
    depends_on:
      - database
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - MQTT_HOST=mosquitto
      - VIRTUAL_HOST=${FQDN_TM}
      - CHECK_ORIGIN=true
      - TZ=${TM_TZ}
    volumes:
      - ./import:/opt/app/import
    labels:
      - "traefik.enable=true"
      - "traefik.port=4000"
      - "traefik.http.middlewares.redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.auth.basicauth.usersfile=/auth/.htpasswd"
      - "traefik.http.routers.teslamate-insecure.rule=Host(`${FQDN_TM}`)"
      - "traefik.http.routers.teslamate-insecure.middlewares=redirect"
      - "traefik.http.routers.teslamate-ws.rule=Host(`${FQDN_TM}`) && Path(`/live/websocket`)"
      - "traefik.http.routers.teslamate-ws.entrypoints=websecure"
      - "traefik.http.routers.teslamate-ws.tls"
      - "traefik.http.routers.teslamate.rule=Host(`${FQDN_TM}`)"
      - "traefik.http.routers.teslamate.middlewares=auth"
      - "traefik.http.routers.teslamate.entrypoints=websecure"
      - "traefik.http.routers.teslamate.tls.certresolver=tmhttpchallenge"
    cap_drop:
      - all

  database:
    image: postgres:12
    restart: always
    environment:
      - POSTGRES_USER=${TM_DB_USER}
      - POSTGRES_PASSWORD=${TM_DB_PASS}
      - POSTGRES_DB=${TM_DB_NAME}
    volumes:
      - teslamate-db:/var/lib/postgresql/data

  grafana:
    image: teslamate/grafana:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - GRAFANA_PASSWD=${GRAFANA_PW}
      - GF_SECURITY_ADMIN_USER=${GRAFANA_USER}
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PW}
      - GF_AUTH_BASIC_ENABLED=true
      - GF_AUTH_ANONYMOUS_ENABLED=false
      - GF_SERVER_ROOT_URL=https://${FQDN_GRAFANA}
    volumes:
      - teslamate-grafana-data:/var/lib/grafana
    labels:
      - "traefik.enable=true"
      - "traefik.port=3000"
      - "traefik.http.middlewares.redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.grafana-insecure.rule=Host(`${FQDN_GRAFANA}`)"
      - "traefik.http.routers.grafana-insecure.middlewares=redirect"
      - "traefik.http.routers.grafana.rule=Host(`${FQDN_GRAFANA}`)"
      - "traefik.http.routers.grafana.entrypoints=websecure"
      - "traefik.http.routers.grafana.tls.certresolver=tmhttpchallenge"

  mosquitto:
    image: eclipse-mosquitto:1.6
    restart: always
    ports:
      - 127.0.0.1:1883:1883
    volumes:
      - mosquitto-conf:/mosquitto/config
      - mosquitto-data:/mosquitto/data

  proxy:
    image: traefik:v2.2
    restart: always
    command:
      - "--global.sendAnonymousUsage=false"
      - "--providers.docker"
      - "--providers.docker.exposedByDefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.tmhttpchallenge.acme.httpchallenge=true"
      - "--certificatesresolvers.tmhttpchallenge.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.tmhttpchallenge.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.tmhttpchallenge.acme.storage=/etc/acme/acme.json"
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./.htpasswd:/auth/.htpasswd
      - ./acme/:/etc/acme/
      - /var/run/docker.sock:/var/run/docker.sock:ro

  teslamateagile:
    image: mattjeanes/teslamateagile:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - TeslaMate__UpdateIntervalSeconds=300
      - TeslaMate__GeofenceId=1
      - TeslaMate__Phases=1
      - Octopus__RegionCode=L

volumes:
  teslamate-db:
  teslamate-grafana-data:
  mosquitto-conf:
  mosquitto-data:

My edited .env file (fields with *** either side are changed):
Code:
TM_DB_USER=teslamate
TM_DB_PASS=***PASSWORD***
TM_DB_NAME=teslamate
TM_TZ=Europe/London

GRAFANA_USER=admin
GRAFANA_PW=***ADIFFERENTPASSWORD***

FQDN_GRAFANA=***grafana.mydomain.net***
FQDN_TM=***teslamate.mydomain.net***

LETSENCRYPT_EMAIL=***[email protected]***

My .htpasswd obviously is just whatever login I want to be able to get to the TeslaMate website in the first place, and doesn't have to be the same password as anything else.

You can delete the teslamateagile stuff from the docker-compose.yml if you are not on Octopus Agile and/or don't want your charges automatically updated with real prices.

That's all I created in terms of files. I have always run docker-compose up as root.
 
  • Like
Reactions: davidmc
Thought I'd try the other advanced option which is to run with apache, but sadly the instructions there aren't complete either, finally managed to get it all installed, but it just points you to a empty www root folder, despite running the a2ensite teslamate command.

huge waste of time so far
 
  • Funny
Reactions: Bongy
Generated a quick visual of the TeslaMate schema that might help some folks when figuring out what data is where and how it links together.

By way of example, you can see a charging_processes entry has to reference an addresses entry, a geo-fences entry, a positions entry and a cars entry in addition to the set of entries in the charges table.

No news for those database-savvy members but probably handy for those who prefer pictures rather than command lines.
teslamate-schema.png
 
Ok well I'm not sure what I did differently this time, but I appear to have it working and have also managed to import my teslafi data in too (well it's currently churning through it anyway - so fingers crossed it still works once it has finished). I'll run through what I think I've done - sorry done this so many different times now I'm struggling to remember what I've done differently each time! I've pretty much followed DaveW's list with a couple of minor adjustments...
  • Create Debian micro instance (tick the http and https boxes): How to set up a free micro VPS on Google Cloud Platform
  • Create two subdomains for your domain (teslamate + grafana) and point the A NAME records to your public IP of the Micro VM
  • Install docker: Install Docker on a Google Cloud virtual machine - slightly amended this part of the guide based on some other guides for Docker on Ubuntu on GCP and included a "sudo apt upgrade" after the first sudo apt update, though I don't think that will have an effect as it only seemed to update a couple of google cloud things. I omitted running the hello-world at the end as I'd done it a hundred times already on previous attempts.
  • Followed by:
    • sudo apt-get install -y libffi-dev libssl-dev
    • sudo apt-get install -y python3 python3-pip
    • sudo apt-get remove python-configparser
    • sudo pip3 install docker-compose
  • Advanced install with Traefik, Let's Encrypt & HTTP Basic Auth | TeslaMate - I used the text from Durzel's post for the docker-compose.yml (minus the agile section) and .env files, though as far as I can see they are virtually identical to those on the Teslamate.org guide.
  • Stop apache: sudo systemctl stop apache2 (don't think this line does anything as I presume it was related to the certbot stuff I had omitted earlier)
  • sudo docker-compose up
  • Once I verified it was up and running I used ctrl-c to exit back out but I found I had to use docker-compose rm -s to stop it properly at that point or else it wouldn't subsequently start up again when I tried sudo docker-compose up -d to run it in background.
  • For the import I just used the ssh browser window to copy my teslafi csv files one at a time to the home directory (luckily only had the car 6 months so not too many files) and then "sudo mv *.csv import" to move them into the correct directory for the import process. To initiate the import had to restart teslamate so did another docker-compose rm -s followed by sudo docker-compose up -d to get back into teslamate.<whateveryourdomain.com> to initiate the import. Once it has finished it should just be a question of one last rm -s and up -d cycle, remembering to delete the csv files from the import directory before restarting it.
 
  • Like
Reactions: davidmc and Durzel
It's vaguely possible that something recent has changed in the pulls for one of the containers which is stopping it working, perhaps?

This is my unedited docker-compose.yml file:
Code:
version: "3"

services:
  teslamate:
    image: teslamate/teslamate:latest
    restart: always
    depends_on:
      - database
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - MQTT_HOST=mosquitto
      - VIRTUAL_HOST=${FQDN_TM}
      - CHECK_ORIGIN=true
      - TZ=${TM_TZ}
    volumes:
      - ./import:/opt/app/import
    labels:
      - "traefik.enable=true"
      - "traefik.port=4000"
      - "traefik.http.middlewares.redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.auth.basicauth.usersfile=/auth/.htpasswd"
      - "traefik.http.routers.teslamate-insecure.rule=Host(`${FQDN_TM}`)"
      - "traefik.http.routers.teslamate-insecure.middlewares=redirect"
      - "traefik.http.routers.teslamate-ws.rule=Host(`${FQDN_TM}`) && Path(`/live/websocket`)"
      - "traefik.http.routers.teslamate-ws.entrypoints=websecure"
      - "traefik.http.routers.teslamate-ws.tls"
      - "traefik.http.routers.teslamate.rule=Host(`${FQDN_TM}`)"
      - "traefik.http.routers.teslamate.middlewares=auth"
      - "traefik.http.routers.teslamate.entrypoints=websecure"
      - "traefik.http.routers.teslamate.tls.certresolver=tmhttpchallenge"
    cap_drop:
      - all

  database:
    image: postgres:12
    restart: always
    environment:
      - POSTGRES_USER=${TM_DB_USER}
      - POSTGRES_PASSWORD=${TM_DB_PASS}
      - POSTGRES_DB=${TM_DB_NAME}
    volumes:
      - teslamate-db:/var/lib/postgresql/data

  grafana:
    image: teslamate/grafana:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - GRAFANA_PASSWD=${GRAFANA_PW}
      - GF_SECURITY_ADMIN_USER=${GRAFANA_USER}
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PW}
      - GF_AUTH_BASIC_ENABLED=true
      - GF_AUTH_ANONYMOUS_ENABLED=false
      - GF_SERVER_ROOT_URL=https://${FQDN_GRAFANA}
    volumes:
      - teslamate-grafana-data:/var/lib/grafana
    labels:
      - "traefik.enable=true"
      - "traefik.port=3000"
      - "traefik.http.middlewares.redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.grafana-insecure.rule=Host(`${FQDN_GRAFANA}`)"
      - "traefik.http.routers.grafana-insecure.middlewares=redirect"
      - "traefik.http.routers.grafana.rule=Host(`${FQDN_GRAFANA}`)"
      - "traefik.http.routers.grafana.entrypoints=websecure"
      - "traefik.http.routers.grafana.tls.certresolver=tmhttpchallenge"

  mosquitto:
    image: eclipse-mosquitto:1.6
    restart: always
    ports:
      - 127.0.0.1:1883:1883
    volumes:
      - mosquitto-conf:/mosquitto/config
      - mosquitto-data:/mosquitto/data

  proxy:
    image: traefik:v2.2
    restart: always
    command:
      - "--global.sendAnonymousUsage=false"
      - "--providers.docker"
      - "--providers.docker.exposedByDefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.tmhttpchallenge.acme.httpchallenge=true"
      - "--certificatesresolvers.tmhttpchallenge.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.tmhttpchallenge.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.tmhttpchallenge.acme.storage=/etc/acme/acme.json"
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./.htpasswd:/auth/.htpasswd
      - ./acme/:/etc/acme/
      - /var/run/docker.sock:/var/run/docker.sock:ro

  teslamateagile:
    image: mattjeanes/teslamateagile:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - TeslaMate__UpdateIntervalSeconds=300
      - TeslaMate__GeofenceId=1
      - TeslaMate__Phases=1
      - Octopus__RegionCode=L

volumes:
  teslamate-db:
  teslamate-grafana-data:
  mosquitto-conf:
  mosquitto-data:

My edited .env file (fields with *** either side are changed):
Code:
TM_DB_USER=teslamate
TM_DB_PASS=***PASSWORD***
TM_DB_NAME=teslamate
TM_TZ=Europe/London

GRAFANA_USER=admin
GRAFANA_PW=***ADIFFERENTPASSWORD***

FQDN_GRAFANA=***grafana.mydomain.net***
FQDN_TM=***teslamate.mydomain.net***

LETSENCRYPT_EMAIL=***[email protected]***

My .htpasswd obviously is just whatever login I want to be able to get to the TeslaMate website in the first place, and doesn't have to be the same password as anything else.

You can delete the teslamateagile stuff from the docker-compose.yml if you are not on Octopus Agile and/or don't want your charges automatically updated with real prices.

That's all I created in terms of files. I have always run docker-compose up as root.

Just did a "find differences' between the contents of your file and the one online and they are exactly the same once you remove the agile bit!
 
Ok well I'm not sure what I did differently this time, but I appear to have it working and have also managed to import my teslafi data in too (well it's currently churning through it anyway - so fingers crossed it still works once it has finished). I'll run through what I think I've done - sorry done this so many different times now I'm struggling to remember what I've done differently each time! I've pretty much followed DaveW's list with a couple of minor adjustments...
  • Create Debian micro instance (tick the http and https boxes): How to set up a free micro VPS on Google Cloud Platform
  • Create two subdomains for your domain (teslamate + grafana) and point the A NAME records to your public IP of the Micro VM
  • Install docker: Install Docker on a Google Cloud virtual machine - slightly amended this part of the guide based on some other guides for Docker on Ubuntu on GCP and included a "sudo apt upgrade" after the first sudo apt update, though I don't think that will have an effect as it only seemed to update a couple of google cloud things. I omitted running the hello-world at the end as I'd done it a hundred times already on previous attempts.
  • Followed by:
    • sudo apt-get install -y libffi-dev libssl-dev
    • sudo apt-get install -y python3 python3-pip
    • sudo apt-get remove python-configparser
    • sudo pip3 install docker-compose
  • Advanced install with Traefik, Let's Encrypt & HTTP Basic Auth | TeslaMate - I used the text from Durzel's post for the docker-compose.yml (minus the agile section) and .env files, though as far as I can see they are virtually identical to those on the Teslamate.org guide.
  • Stop apache: sudo systemctl stop apache2 (don't think this line does anything as I presume it was related to the certbot stuff I had omitted earlier)
  • sudo docker-compose up
  • Once I verified it was up and running I used ctrl-c to exit back out but I found I had to use docker-compose rm -s to stop it properly at that point or else it wouldn't subsequently start up again when I tried sudo docker-compose up -d to run it in background.
  • For the import I just used the ssh browser window to copy my teslafi csv files one at a time to the home directory (luckily only had the car 6 months so not too many files) and then "sudo mv *.csv import" to move them into the correct directory for the import process. To initiate the import had to restart teslamate so did another docker-compose rm -s followed by sudo docker-compose up -d to get back into teslamate.<whateveryourdomain.com> to initiate the import. Once it has finished it should just be a question of one last rm -s and up -d cycle, remembering to delete the csv files from the import directory before restarting it.

Any idea what was different on the docker installation front? That's really the only change.

The stopping apache doesn't seem to be needed, I must have installed apache in all the docking around trying to get this to work :)
 
As far as I can tell the only difference would have been running the docker-compose as sudo, though I did use some different passwords in the .env file this time. It does seem to be very flakey though in terms of when you want to bring the service down and bring it back up again.

I may have spoken too soon though as the import doesn't actually appear to be working lol. Been sat with all six files making no progress for a couple of hours now... I know it took my Synology NAS a while to get through them but I would have thought it would have least managed one or two of the files by now.
 
Well I think I'm officially throwing in the towel - the exact same instance that was working earlier now simply isn't! Now when I try and start it in the exact same way it appears to be running but clearly isn't as there is I just get a 404 at my teslamate address. Think I'll stick with Teslafi for the time being!
 
Are you giving it a little bit of time to all start up? Sounds stupid I know, but I get a “Bad Gateway” error for a short time while it’s starting up.

Also the free F1-micro isn’t the fastest. When I was running it I eventually got an warning in Google Cloud Platform that it was running low in memory. The next level up - F1-small - cost me £10 to run last month, but I actually paid nothing because of the $300 introductory credit. That credit has an expiry date of a year I believe, so I’m not going to be using it up in that time.

Another thing you can try and do is connect to it in a private/incognito web browser, in case anything duff has been cached.

Food for thought maybe.
 
Very close to having cracked this now. Mostly working on Debian 9, but I think it's more around the Docker install instructions I used, certs are working!

Only thing that isn't is the .htpasswd, it's bypassing that for some reason, not entirely sure why.

I'll share instructions once I'm confident it works OK.
 
  • Like
Reactions: davidmc
@Durzel - do you know what version of Debian you are running? I have managed to get this working fine for v9, but for 10 the lets encrypt just doesn't work! -

@davidmc @simon.c - be interested to see if the below works for you?

This is what I did to make it work on v9

Create your free tier GCP instance:
Create Micro instance, enabling http and https (tick boxes) Debian 9, disk size 30gb: How to set up a free micro VPS on Google Cloud Platform

Setting up subdomains:
Create two subdomains for your domain (teslamate + grafana) and point the A NAME records to your public IP of the Micro VM

Wait for the DNS to propagate:
After creating the ANAME records, you will need to wait until the DNS has propagated, you can test this by pinging teslamate.YOURDOMAIN, when it comes back with the GCP IP you've put in, you're good to proceed.

Docker install
Open SSH (browser is fine from the VM screen)
Run the following commands;
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common gnupg2
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] Index of linux/debian/ $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker $USER

Docker compose install
sudo curl -L https://github.com/docker/compose/releases/download/1.26.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Now follow the instructions on TeslaMate: Advanced install with Traefik, Let's Encrypt & HTTP Basic Auth | TeslaMate

Launch with sudo docker-compose up

Once you've confirmed it works without errors, you can Ctrl+C to quit the compose session and restart it with docker-compose up -d
 
  • Like
Reactions: davidmc
@Durzel - do you know what version of Debian you are running? I have managed to get this working fine for v9, but for 10 the lets encrypt just doesn't work! -

@davidmc @simon.c - be interested to see if the below works for you?

This is what I did to make it work on v9

Create your free tier GCP instance:
Create Micro instance, enabling http and https (tick boxes) Debian 9, disk size 30gb: How to set up a free micro VPS on Google Cloud Platform

Setting up subdomains:
Create two subdomains for your domain (teslamate + grafana) and point the A NAME records to your public IP of the Micro VM

Wait for the DNS to propagate:
After creating the ANAME records, you will need to wait until the DNS has propagated, you can test this by pinging teslamate.YOURDOMAIN, when it comes back with the GCP IP you've put in, you're good to proceed.

Docker install
Open SSH (browser is fine from the VM screen)
Run the following commands;
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common gnupg2
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] Index of linux/debian/ $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce
sudo usermod -aG docker $USER

Docker compose install
sudo curl -L https://github.com/docker/compose/releases/download/1.26.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Now follow the instructions on TeslaMate: Advanced install with Traefik, Let's Encrypt & HTTP Basic Auth | TeslaMate

Launch with sudo docker-compose up

Once you've confirmed it works without errors, you can Ctrl+C to quit the compose session and restart it with docker-compose up -d
Many thanks @DaveW will report back once i go through it