[Version 2 Release] RaspberryPints - Digital Taplist Solution

HomeBrewTalk.com - Beer, Wine, Mead, & Cider Brewing Discussion Community.

Help Support Homebrew Talk:

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
@esdill ok, with both temperature probes, I see a significant delay in the page loading, but it does load eventually. I'll run strace on apache and see if I can figure out what is going on.
Mine would "eventually" load as well, the last try it took upwards of 6 hours to finally load, and then the temperatures on the screen did not update once it loaded.

The brand new fresh install seems to be working so far, but only time will tell.
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
@RandR+ I think I know where the issue. After hooking up two temperature probes, the main RPints index.php is taking a very long time to load (47 seconds):
Bash:
time curl http://10.0.1.31/ 1>/dev/null
0.00s user 0.01s system 0% cpu 47.040 total
This delay is caused by the length of time that it takes MariaDB to execute this query:
Code:
MariaDB [raspberrypints]> SELECT temp, tempUnit, COALESCE(NULLIF(p.notes,''), tl.probe) AS probe, takenDate AS takenDate FROM tempProbes p LEFT JOIN tempLog tl ON (tl.probe = p.name AND tl.takenDate = (SELECT MAX(takenDate) FROM tempLog tl2 WHERE tl2.probe = tl.probe))WHERE p.active = 1 ORDER BY tl.probe ASC;

+-------+----------+---------+---------------------+
| temp  | tempUnit | probe   | takenDate           |
+-------+----------+---------+---------------------+
| 21.80 | C        | Probe 1 | 2020-07-14 09:12:41 |
| 21.40 | C        | Probe 2 | 2020-07-14 09:12:42 |
+-------+----------+---------+---------------------+
2 rows in set (49.163 sec)
The join seems to be taking forever. The length of time this query is taking is growing as the number of rows in tempLog grows:
Code:
MariaDB [raspberrypints]> select count(*) from tempLog;
+----------+
| count(*) |
+----------+
|     2999 |
+----------+
1 row in set (0.010 sec)
By doing a fresh install, @esdill has emptied this table but the problem is going to come back as the number of rows grows. I have to go and do my day job now, but I will take a look at constructing a better SQL query tonight, if you don't get to it first.
 
Last edited:

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
I don't have a large dataset for temps
I added over 50000 rows to my local test tempLog table and the query to get the latest time never returns, which explains what the page isn't loading (and no error). I rewrote the query to be a PHP loop to get the latest and it works with large amount of data. Getting latest should fix the loading

I'm going to rework the temp saving to make the original query work which should be the fastest way of doing this
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
Will I still have to do what you posted above? Sounds like the raspberry pints website isn't up to date and will not work as is (with and without flow meters), right?
yes, the raspberry pints website and original code doesn't work on the latest Raspbian OSs so you either need to use an old OS or follow that post.
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
@RandR+ I have an SQL query that works in less than a second:
Code:
MariaDB [raspberrypints]> SELECT tp.temp, tp.tempUnit, tn.notes as probe, tp.takenDate from tempProbes tn LEFT OUTER JOIN (SELECT tt.temp, tt.tempUnit, tt.probe, tt.takenDate FROM tempLog tt INNER JOIN (SELECT probe, MAX(takenDate) AS MaxDateTime     FROM tempLog   GROUP BY probe) groupedtt  ON tt.probe = groupedtt.probe  AND tt.takenDate = groupedtt.MaxDateTime) tp ON tp.probe = tn.name;
+-------+----------+---------+---------------------+
| temp  | tempUnit | probe   | takenDate           |
+-------+----------+---------+---------------------+
| 22.80 | C        | Probe 1 | 2020-07-14 09:57:34 |
| 22.00 | C        | Probe 2 | 2020-07-14 09:57:35 |
+-------+----------+---------+---------------------+
2 rows in set (0.252 sec)
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
I have an SQL query that works in less than a second:
Thanks, but when ran against my dataset and it takes upwards of 2.5 seconds to complete.
if I fix how the data is stored in the database from the get-go I can use a simply query without joins to get the latest and that returns in under .2 seconds for this dataset
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
@esdill @RandR+ that pull request does the trick. Both my temperature probes now work and the page loads in under a second. I added another commit to skip a probe if it's name in NULL to replicate the existing behavior.

Screen Shot 2020-07-14 at 10.27.32 AM.png
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
Thanks, but when ran against my dataset and it takes upwards of 2.5 seconds to complete.
if I fix how the data is stored in the database from the get-go I can use a simply query without joins to get the latest and that returns in under .2 seconds for this dataset
Great, that sounds like a better solution.
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
Talking of temperature probes: I didn't want to use another wire in the cable to my keezer for +3V3. The DS18B20 probes I have can be power by the AlaMode +5V that I use to power my SF800s. I put a 4k7 resistor between my AlaMode board's 3V3 supply and the data line as a pull-up and then connect the data line to the GPIO pin on my Pi. This works fine and only requires one extra wire from my keezer for all the temperature signals. The DB18B20 has an open drain on the data line, so this works fine.
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
Ok got the temp logging updated (again) now it will force each reading iteration to have the same timestamp so I can use my simple query that is fast

Also update the installer two main updates
1. added support for side by side installation with fermentrack.
2. added updating to stash changes before getting latest then after the update is complete, opening a compare so users can see their changes compared to the latest (unfortunately they need to manually apply their changes as the compare happens on temp files)
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
Ok got the temp logging updated (again) now it will force each reading iteration to have the same timestamp so I can use my simple query that is fast

Also update the installer two main updates
1. added support for side by side installation with fermentrack.
2. added updating to stash changes before getting latest then after the update is complete, opening a compare so users can see their changes compared to the latest (unfortunately they need to manually apply their changes as the compare happens on temp files)
I ran the update, my temperature probes show the probe address (28-xxxxxxxxx) instead of the names I have entered in the configuration, and when the screen refreshes sometimes there's one probe shown and sometimes there's two probes shown.

Annotation 2020-07-14 120918.jpg

Annotation 2020-07-14 120920.jpg
 
Last edited:

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
now I cannot upload my brewery logo or my background
probably because it cleared the permissions on the directory with the new way of getting the latest, I will add in the permission fix into the installer.
for now try
sudo chmod +777 /var/www/html -R

my temperature probes show the probe address (28-xxxxxxxxx) instead of the names I have entered in the configuration
I will research and fix, I have a feeling I reverted back too far
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
probably because it cleared the permissions on the directory with the new way of getting the latest, I will add in the permission fix into the installer.
for now try
sudo chmod +777 /var/www/html -R


I will research and fix, I have a feeling I reverted back too
Thanks - I thought of the permissions and reset them and then it worked (I edited my reply as soon as I figured it out). Any idea why the update wipes out the brewery logo and the background each time it runs? (It's not a big deal, I have been re-uploading them each time ... but it would be nice if I did not need to).
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
. Any idea why the update wipes out the brewery logo and the background each time it runs?
Probably because it removes them as conflicts and doesn't add them back in. I will work on a way to get them to stay when you update.

I found I did go back too far with my revert and fixed so now the probes will display their name not their device id.

Can you restart the flowmon service and see if that fixes the issue probes jumping back and forth between 1 and 2. I
Check /python/FlowMonitor.py line 703 to make sure it starts with takenDate and check line 719 to make sure it ends with takenDate)
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
Any idea why the update wipes out the brewery logo and the background each time it runs?
I updated the installer to save off the images and force background and logo to stay as you had it. Also added in fixing the permissions on the directory after the update.

if you rerun
curl -L install.rpints.com | sudo bash
you will have the latest changes
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
Probably because it removes them as conflicts and doesn't add them back in. I will work on a way to get them to stay when you update.

I found I did go back too far with my revert and fixed so now the probes will display their name not their device id.

Can you restart the flowmon service and see if that fixes the issue probes jumping back and forth between 1 and 2. I
Check /python/FlowMonitor.py line 703 to make sure it starts with takenDate and check line 719 to make sure it ends with takenDate)
The temp probe names are now showing correctly. I verified FlowMonitor.py to show what you asked, and it does. I stopped the flowmon service, and then started the flowmon service ... and it still randomly shows one probe or two probes.
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
There's a race condition somewhere:
Code:
MariaDB [raspberrypints]> SELECT tp.temp, tp.tempUnit, tn.notes as probe, tp.takenDate from tempProbes tn LEFT OUTER JOIN (SELECT tt.temp, tt.tempUnit, tt.probe, tt.takenDate FROM tempLog tt INNER JOIN (SELECT probe, MAX(takenDate) AS MaxDateTime     FROM tempLog   GROUP BY probe) groupedtt  ON tt.probe = groupedtt.probe  AND tt.takenDate = groupedtt.MaxDateTime) tp ON tp.probe = tn.name;
+-------+----------+---------+---------------------+
| temp  | tempUnit | probe   | takenDate           |
+-------+----------+---------+---------------------+
| 22.90 | C        | Probe 1 | 2020-07-14 13:11:32 |
| 22.80 | C        | Probe 2 | 2020-07-14 13:11:33 |
+-------+----------+---------+---------------------+
2 rows in set (0.656 sec)
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
and it still randomly shows one probe or two probes.
There's a race condition somewhere:
Ok my guess its when the temp is committed, I did some testing with forcing a delay between reading test probe 1 and test probe 2 and they both had the same time, but if I refreshed the page during the delay only one was shown.

I just need to force the database inserts in one commit to fix, should be later today
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
Success, thanks! I ran the select directly on the database a bunch of times to check and the times are synced.

View attachment 689419
I also now have good temperature readings ... however now my pours are not registering (I hadn't tested the pours on the fresh install prior to this, so I do not know if the recent changes had an impact related to this or not). I have restarted the flowmeter service, and I have reset the Arduino, and no joy.
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
however now my pours are not registering
Did you make sure to reconfigure your taps? You can turn on debugging, try a pour and look at /var/logs/rpints.log

Debugging in /python/Config.py change False#True to True like:
config['flowmon.debug' ] = True
#logging settings for pintdispatch
config['dispatch.debug' ] = True

Then restart the flowmonitor service and pour a beer
sudo /etc/init.d/flowmon restart
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
Did you make sure to reconfigure your taps? You can turn on debugging, try a pour and look at /var/logs/rpints.log

Debugging in /python/Config.py change False#True to True like:
config['flowmon.debug' ] = True
#logging settings for pintdispatch
config['dispatch.debug' ] = True

Then restart the flowmonitor service and pour a beer
sudo /etc/init.d/flowmon restart
1594759254043.png


2020-07-14 16:43:19 RPINTS: valve update: RPU:VALVE:0=0
2020-07-14 16:43:19 RPINTS: valve update: RPU:VALVE:1=0
2020-07-14 16:43:19 RPINTS: valve update: RPU:VALVE:2=0
2020-07-14 16:43:19 RPINTS: valve update: RPU:VALVE:3=0
2020-07-14 16:43:19 RPINTS: Could not import RFID Reader, RFID disabled. Assuming SPI not installed/configured
2020-07-14 16:43:21 RPINTS: starting setup...
2020-07-14 16:43:21 RPINTS: resetting alamode to try to force it to listen to us...
2020-07-14 16:43:22 RPINTS: giving it a short break to wake up again...
2020-07-14 16:43:24 RPINTS: reflashing Arduino via:
/usr/share/arduino/hardware/tools/avrdude -C/usr/share/arduino/hardware/tools/avrdude.conf -patmega328p -calamode -P/dev/ttyS0 -b115200 -D -Uflash:w:/var/www/html//arduino/raspberrypints/raspberrypints.cpp.hex:i
RPINTS: reflashing Arduino failed, moving on anyways, error was: Command '/usr/share/arduino/hardware/tools/avrdude -C/usr/share/arduino/hardware/tools/avrdude.conf -patmega328p -calamode -P/dev/ttyS0 -b115200 -D -Uflash:w:/var/www/html//arduino/raspberrypints/raspberrypints.cpp.hex:i' returned non-zero exit status 1
2020-07-14 16:43:30 RPINTS:
2020-07-14 16:43:30 RPINTS: starting WS server
2020-07-14 16:43:30 RPINTS: starting device monitors...
2020-07-14 16:43:30 RPINTS: starting command server
2020-07-14 16:43:30 RPINTS: resetting Arduino
2020-07-14 16:43:30 RPINTS: starting fan control
2020-07-14 16:43:30 RPINTS: Fan Control fanControl1 is Running
2020-07-14 16:43:30 RPINTS: Not Configured to run Fan
2020-07-14 16:43:32 RPINTS: 1Wire Temperature Thread 1 is Running
2020-07-14 16:43:32 RPINTS: waiting for Arduino to come alive

(I've already had two beers. but for the sake of research I poured myself another one) ... :)

By the way ... THANK YOU for everything you have done so far! If you ever find yourself in Port St Lucie, FL, stop in for a beer. I owe you at LEAST one.
 

esdill

Well-Known Member
Joined
May 16, 2020
Messages
51
Reaction score
13
Are you using Uno or Alamode?

The board isn't talking back to the Pi and I can see that you have it configured for serial communication which is alamode
It's a UNO and I selected USB when I ran the installer.

Arrrrhg! I'll run the installer and reconfigure and see if it fixes it. Of course that means I'll need a fourth beer, you know, just to make sure it's working.
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
@garzlok asked me how I manage my fork of the code on GitHub, so here's the details. This assumes that you're somewhat familiar with the GitHub Fork and Pull Model and want to make some local stylistic changes to your RPints install. If you want to contribute to the development, you probably already know how to do this so I am not including instructions for that. This also assumes that you have forked the the repository at rtlindne/RaspberryPints (do not fork the original repo as that is defunct) and that your original install was performed using the RandR+ version by the install script.

When you run the install script, it clones the GitHub repository rtlindne/RaspberryPints into /var/www/html/ and this becomes the active code. After you're done with the install, you can inspect this by running the commands
Bash:
cd /var/www/html
git remote -vvv
This should return
Code:
origin    https://github.com/rtlindne/RaspberryPints.git (fetch)
origin    https://github.com/rtlindne/RaspberryPints.git (push)
which says that any git push or pull commands will be run against the RandR+ repository. You can't push to this repo, and updates are done by the install script triggering at git pull to refresh the master branch of the repo you have checked out.

Now you need to point the version on your Pi to your fork of the code. Run the commands below with username replaced by your GitHub username
Bash:
cd /var/www/html
sudo git remote add fork https://github.com/username/RaspberryPints.git
sudo git fetch --all
You can now run the following command to checkout the master branch of your repository
Bash:
cd /var/www/html
sudo git checkout -b fork fork/master
You will see the message
Code:
Branch 'fork' set up to track remote branch 'master' from 'fork'.
Switched to a new branch 'fork'
so your active code is now using your own version of the code. When you do this, you may see messages like
Code:
M    admin/img/logo.png
M    img/logo.png
which indicates that you have locally modified these files.

To update the running version of your code, you can run the command
Code:
cd /var/www/html
sudo git pull
which will update the code against the master branch of your fork.

Note that by adding the remote with https, we have create a read only git repo on the Pi. This is my preference, as I do the actual development work (or style changes) in a repo I have checked out on my desktop machine and then when I'm happy, commit the changes to the master branch on my fork and update the read-only branch on the Pi.

If you want to make your changes in the active code, you can check out your GitHub repo using the SSH protocol, e.g.
[email protected]:RaspberryPints/RaspberryPints.git. You can then push your changes back up to GitHub.
 

troyp42

Well-Known Member
Joined
May 12, 2012
Messages
436
Reaction score
43
Location
Melbourne
Ok got the temp logging updated (again) now it will force each reading iteration to have the same timestamp so I can use my simple query that is fast

Also update the installer two main updates
1. added support for side by side installation with fermentrack.
2. added updating to stash changes before getting latest then after the update is complete, opening a compare so users can see their changes compared to the latest (unfortunately they need to manually apply their changes as the compare happens on temp files)
Mate that is awesome you've made these changes. Updating was a PITA before. So with the compare is it just a matter of copying the changed lines of code from the existing files into the new ones that were made after the update?
 

garzlok

Well-Known Member
Joined
Aug 19, 2012
Messages
262
Reaction score
134
Location
Dacula
My Gitkwondo Is not very strong. Followed the directions and still ended up with the commit or stash when trying to pull....leads me to believe, I did something wrong, I am not merged when I thought I was, or I’m doing things backwards.

After I point my Pi to my Git Fork, and then run checkout, can I checkout a specific branch in my fork by changing fork/master to fork/branch_x?

When I run the git pull, does it always pull from Master or does it pull to the branch I specified during checkout?
 

duncan.brown

Supporting Member
HBT Supporter
Joined
Apr 8, 2020
Messages
204
Reaction score
142
Location
Syracuse
The best way to inspect what is going on is to run git status in the html directory, e.g.
Code:
[email protected]:/var/www/html $ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   admin/img/logo.png
    modified:   img/logo.png
    modified:   includes/refresh.sh
    modified:   python/Config.py

no changes added to commit (use "git add" and/or "git commit -a")
I have some locally modified but uncommitted files, but other than that I am on the right branch. For me, origin/master is my fork, which I can check with
Code:
[email protected]:/var/www/html $ git remote -vvv
origin    https://github.com/duncan-brown/RaspberryPints.git (fetch)
origin    https://github.com/duncan-brown/RaspberryPints.git (push)
Note that origin is my repo.

After I point my Pi to my Git Fork, and then run checkout, can I checkout a specific branch in my fork by changing fork/master to fork/branch_x?
Yes, but you need to check it out as a remote tracking branch with
Code:
sudo git checkout -b fork fork/branch_x
When I run the git pull, does it always pull from Master or does it pull to the branch I specified during checkout?
git pull without any arguments does a pull to the currently active branch. You can check this with git status or git branch -vvv
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
So with the compare is it just a matter of copying the changed lines of code from the existing files into the new ones that were made after the update?
Yes, you need to look through the differences for the ones you want to keep (i.e the ones you made) then manually copy and paste them into the same file under your install path (for example you added text to /var/www/html/index.php, the compare will show the difference and you will need to manually copy it back to /var/www/html/index.php)
 

Ferdaze

New Member
Joined
Mar 30, 2020
Messages
4
Reaction score
0
Anyone else getting this when trying to update with the script?

Code:
[email protected]:~ $ curl -L install.rpints.com | sudo bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   299  100   299    0     0   1738      0 --:--:-- --:--:-- --:--:--  1748
100 54276  100 54276    0     0   115k      0 --:--:-- --:--:-- --:--:--  115k

***Script rpints_install.sh starting.***
RPints Install Detected. Do you wish to Update RPints? [Y/n]:
Stopping Rpints...
No local changes to save
Already up to date.

This message is displayed because 'diff.tool' is not configured.
See 'git difftool --tool-help' or 'git help config' for more details.
'git difftool' will now attempt to use one of the following tools:
kompare emerge vimdiff
Vim: Warning: Output is not to a terminal
Vim: Warning: Input is not from a terminal
" ======================================|" =====================================
" Netrw Directory Listing               |" Netrw Directory Listing
"   /tmp/git-difftool.MnZmkp/left       |"   /tmp/git-difftool.MnZmkp/right
"   Sorted by      name                 |"   Sorted by      name
"   Sort sequence: [\/]$,\<core\%(\.\d\+|"   Sort sequence: [\/]$,\<core\%(\.\d\
"   Quick Help: <F1>:help  -:go up dir  |"   Quick Help: <F1>:help  -:go up dir
" ======================================|" =====================================
../                                     |../
./                                      |./
admin/                                  |admin/
img/                                    |img/
includes/                               |includes/
maintenance_scripts/                    |maintenance_scripts/
python/                                 |python/
util/                                   |util/
bootstrap.sh*                           |[email protected]                       --> /
style.css*                              |[email protected]                        --> /
LICENSE.md                              |[email protected]                     --> /
README.md                               |[email protected]                        --> /
~                                       |~
~                                       |~
~                                       |~
<ool.MnZmkp/left [RO] 8,1            All <l.MnZmkp/right [RO] 8,1            All
"/tmp/git-difftool.MnZmkp/right/" Illegal file name
 

RandR+

Well-Known Member
Joined
Oct 5, 2015
Messages
529
Reaction score
162
Anyone else getting this when trying to update with the script?
That's now normal, you can just close the diff (control+c) if you don't have any manual changes you made locally.

you can install kompare (sudo apt-get install kompare) it will display the differences graphically making it easier to see
 

Ferdaze

New Member
Joined
Mar 30, 2020
Messages
4
Reaction score
0
That's now normal, you can just close the diff (control+c) if you don't have any manual changes you made locally.

you can install kompare (sudo apt-get install kompare) it will display the differences graphically making it easier to see
That works better. Thanks!
 
Top