Run as Service in Raspbian/Debian?

[LBussy]

I am starting a new thread here since it would probably only confuse someone coming to the Mega-Thread trying to get actionable info. I'm also mentioning the folks I remember working with before on some BrewPi-related things. If I've forgotten to add you, it's not because I don't want your opinion, this is just a kickstarter:

@Thorrak @pocketmon @day_trippr @kaljade @gromitdj @CadiBrewer @Bigdaddyale

I've read a few threads at the BrewPi Community pages, as well as discussions on GitHub where the BrewPi contributors tried to hash out the age old question: How should we run BrewPi scripts? I think that some of the concerns they have are valid, some are frankly not for me to worry about.

Since I've put in (I'm not saying how many) hours to resurrect the Legacy BrewPi for new users on contemporary Pi's, I'm now staring at the need to address the problem of starting BrewPi. This is not a unique issue, some of you have projects with similar needs, so I am really seeking your ideas on how to best handle this. It would be easy to leave it as-is with cron.d scripts, but those really start failing when you add multi-chamber support.

I'll throw out the first pitch, here are what I think are the absolute must-haves:

• BrewPi needs to be able to be started automatically
• BrewPi needs to start after a reboot
• BrewPi needs a way to be controlled from the command line

There are some "requirements" that I'm not sure are not artificial that create some challenges:

• Start/stop from web page
• ... others?

Is starting and stopping from the web page REALLY needed? What drives that need? If a person needed to stop the script, aren't they going to be in and able to use the CLI?

As to what solution to use, I think that the following is a good start for requirements:

• It needs to provide command-line access/options to control the script
• It should be able to be modified by the installation scripts (thinking adding chambers in multi-chamber mode for instance)
• It ideally would be part of base Debian (trying to avoid bloat)
• (does it need to be controlled via the web page?)

Reviewing this, I can't think of a reason not to use systemd. It's available, it's part of Debian distros now, it's lightweight, it's well-supported.

About the only things I can think of that are strikes against it are:

• It requires Debian (I think it is supported in other distros but not sure to what extent)
• It absolutely requires Linux - no Windows support that I know of
• It's somewhat more convoluted to create support for systemd, but that's a one-time investment

What do y'all think? There are no stupid questions or answers, I'd love to have a good conversation about this. I've added a poll, but I'm probably not capturing all the choices.

 

[RPh_Guy]

It requires Debian (I think it is supported in other distros but not sure to what extent)

I don't know anything about this specific software, but I would pretty much guarantee it will fully run on Ubuntu and Linux Mint at minimum. I use the latter.
I've been using Linux exclusively since 2005.

Edit: Looks like you have a handle on things. I'm going to unsubscribe. I'd think any distro should have the services mentioned.

 

[day_trippr]

From my perspective:
- need hands-off deterministic startup on boot - meaning all ports are correctly assigned and services (or scripts) started without user intervention. I have that now and would not give it up.
- need to maintain the ability to start/stop scripts via the web gui/without resorting to CLI which may not be as easily available (think remote access).
- indeed CLI control of a BrewPi instance isn't that important to me, but moving from a cron script to a service actually makes CLI control incrementally easier.
- don't know why Windows pops its head in this conversation
- I'd be surprised if any RPi compatible Linux distro doesn't share a system service method with the rest of the versions...

Cheers!

 

[shoengine]

I voted systemd, but that was before I saw the requirement to have control from a webpage. I still think systemd is the way to do from an IT standpoint, but not sure a safe way to manage access via a webpage. You can script almost any action I think from a webpage, but how to keep the box safe from ne'erdowells is another thing.

 

[LBussy]

From my perspective:
- need hands-off deterministic startup on boot - meaning all ports are correctly assigned and services (or scripts) started without user intervention. I have that now and would not give it up.

I agree - anything else would be a step backwards.

- need to maintain the ability to start/stop scripts via the web gui/without resorting to CLI which may not be as easily available (think remote access).

I am thinking remote access. SSH is remote access. Why would you need to stop/reset the script remotely I guess is what I am getting at? What makes you need to do that from the web page rather than the (password protected and therefore more secure) CLI? (That of course brings up another feature path to explore, but let's let that lie for a while.) Putting system administrative functions on an application administration screen seems like a horrible mix to me. What application administrative functions need to happen that require the script is stopped?

- indeed CLI control of a BrewPi instance isn't that important to me, but moving from a cron script to a service actually makes CLI control incrementally easier.

Cron is just UGLY when it comes to considering how to add (and remove) chamber configurations.

- don't know why Windows pops its head in this conversation

Because as I scan over old posts, people want to do the dumbest things. "I have this UltraSPARC sitting under my desk I want to use for BrewPi." Initially, I actually got BrewPi working under Windows because I had that and not a RPi sitting around. Sure it was a matter of a couple days wait but I wanted to mess with it NOW. Just trying to identify what would change with such a configuration change so that it can be documented.

- I'd be surprised if any RPi compatible Linux distro doesn't share a system service method with the rest of the versions...

I'd be mildly surprised, but not dumbstruck.

I voted systemd, but that was before I saw the requirement to have control from a webpage. I still think systemd is the way to do from an IT standpoint, but not sure a safe way to manage access via a webpage. You can script almost any action I think from a webpage, but how to keep the box safe from ne'erdowells is another thing.

To that point, I think having the script control in the web page to begin with is a bad idea. I need more information about why/how people use it I guess.

 

[gromitdj]

I'm glad you included me in this conversation, but I must be nothing more than a monkey with a soldering iron. 90% of this went right over my head. I had to google "CLI"!

But, I'll be watching.

 

[LBussy]

Well, now you’ve learned something Donnie!

A “strong user” is more than welcome to chime in here. Anyone who has used BrewPi for any length of time will be familiar with the balancing act between the crontab, the do_not_run file and the web page. Opinions
On how that should work as a user story are just as important.

 

[gromitdj]

I've only used the start script button when it says "script not running". Then I mash it like crazy until it works again! I never understood it's purpose to be honest.

 

[LBussy]

That’s part of the issue I think. The button removes the do_not_run file and cron runs every minute and tries to start the process. Seeing no file, it continues to run.

To say it is a kludge gives a bad name to real kludges everywhere.

 

[LBussy]

Three votes for systemd, none for anything else. Would still love to hear why the control via the web page is really needed.

Played with systemd a little today, have a multi-chamber setup running with that so that was fairly quick. Logging is another thing all together. I think I;m going to have to hack and slash all the python scripts to un-ass the logging on them. It's really quite convoluted.

The alternative is to put it all in the syslog. Not a bad solution really, but the stdin and stdout in the web interface would have to be re-written somehow. The only thing that functionality is absolutely needed for is flashing the Arduino as near as I can tell and that can be worked around. The only other time I remember going to the logs there is to debug why things were not working simply BECAUSE things are so convoluted.

I dunno, takes looking at it all differently I think.

 

[Thorrak]

The way this is solved in Fermentrack is through the use of Circus and polling the database. Circus launches/manages each BrewPi script instance with pidfiles and database calls replacing the do-not-run concept.

 

[LBussy]

That sounds like a 10-pound sledge approach for BrewPi. I like it though. I thought about something similar but much simpler where the "root" of /home/brewpi" would contain the subs for the chambers, as well as management files for the "farm." Would be a simple concept, I'm just afraid it kicks the can down the road with all the other things it hides.

 

[LBussy]

So, looks like systemd as of 236 will allow writing stdout and stderr to file:

https://github.com/systemd/systemd/pull/7198

Problem there is, current version from apt for Raspbian Stretch is 232. I could always download and build a newer version but there I'm getting WAY away from what should be a collection of scripts.

So, a "drop in" using systemd is not possible right now - at least using all the current "functionality". It bears the question, does BrewPi need stderr and stdout in the web UI? On the face of it, I think a lot of folks would point out that they use them all the time. Digging down, I think we've all done that because "functionality" demanded it, not because it should have been necessary. In other words, if things worked better, maybe looking at those raw logs would not be necessary?

The more I play with systemd though, the more I like the built-in functionality. For instance by default there's seven default log levels available which are automatically colorized and timestamped:

So, current thinking (and seriously, y'all please argue with me if I'm going off the rails) is to harden the way BrewPi (Remix) runs so that logs are not needed all the time. I also think there's some functionality in python which will allow me to load the logs as needed. Finally, the only real-time log watching necessary should be when flashing the Arduino. Or am I all wet?

The good news is, I am having fun with all this. I can see why a couple of you want different directions completely though.

ETA: So I did a little survey. I figure any python file doing a print will be a candidate to review if I go the systemd route. There's 17 files with 421 lines to review.

 

[day_trippr]

Jeeze, you're not contemplating removing the log and error panes in the gui, are you?

Cheers!

 

[LBussy]

Well, I guess the question is: if it's all working, what do you need that for in the web UI? And I still don't understand really what purpose stopping the script in the web UI serves.

If it all just worked better, would those things be needed?

 

[day_trippr]

Things fail.

Cheers!

 

[shoengine]

I get that having the ability to restart the process via a webpage would be handy, but that seems like it would open up a can of worms. I for one am never excited about web access to files outside of the webroot unless we are talking about sanitized input to a db. With that said, I am not a programmer, just a Linux admin, so not exactly knowledgeable about this side of the program.

With that said, I have never touched brewpi, but now I want to. On my DEC alpha. Can I have a port for that?

 

[LBussy]

I'm actually not all that bad with DEC Alpha ... In my formative years I spent a LOT of time on an old VAX. Not too long ago I was still scouring eBay for 9-track parts and reels to keep some of our systems running. Somehow I see that as being a little outside of the mainstream user's realm though. i.e. if you can drive that, you can figure it out yourself.

To Dave's point - yes, things break. In my experience however if people know enough to make sense of the logs, they know enough to 'tail /home/brewpi/logs/stderr.txt'. Not to mention FIXING what goes wrong is almost always going to take CLI access. My first goal however is to remove the initial issue being that things break. THEN I can worry about what's available in the UI. I've cobbled together some pretty cool bits and pieces that I might use to improve all that. For right now, I've made a new release (0.5.0) which has all that I think most users will want, and serves as a jumping off point for finishing multi-chamber work. The bones are there.

I'm not actually thinking about removing 100% of that functionality, I'd just like to refine the information that's available there and not rely on "raw data" so much.

 

[khannon]

On the pi I have found supervisor to be pretty easy/friendly with logging support, starting/restarting scripts automatically etc..

 

[LBussy]

Ended up going with systemd - seems a pretty solid way to run things. And, Dave, I kept the log files. The control from the web page still works, and an added benefit is the script will start up within a few seconds of pressing the button in the web UI.

My issue right now is an internal brewpi.py bug with multi-chamber so I need to figure that part out. My bug list is shrinking and my "might be cool" list is growing ...

 

[day_trippr]

"Go Lee! GO!"

 

[LBussy]

Figured I'd update here since only real geeks are interested in this thread and would care what I found:

Multi-chamber bug was pretty interesting. You can run brewpi.py without the --config argument and if there's a config.cfg present it will load it. BrewPi does conflict checking however, looking for another BrewPi process which may be using the same config file or the same port. That's a great plan in theory, but the problem comes when the two meet: If you have let's say "chamber1" running with no arguments and then try to run "chamber2" it will not be able to find a config file in the command line for chamber1 when it iterates the processes. In that case, BrewPi's other routines take over doing the whole "default load" thing, populating the configuration for chamber1. The problem is, it populates chamber1's object with the config files out of the chamber 2 directory.

If you're with me so far ....

So, there's no context for checking the configuration of chamber1 from the context of chamber2. What I ended up doing, and this is probably a horrible kludge but it works; is get the execution path for brewpi.py from chamber1's pocess, munge it to point to chamber1's local config files, then allow chamber2's brewpi process to compare the real chamber1's config with chamber2.

Details here if you are interested. I'd be more than happy to hear ideas about how I could have done this better. Python's not really my jam:

https://github.com/lbussy/brewpi-script-rmx/commit/403d83b7ba71725fb318ddbf672ad663d5139da8

Honestly, all credit to Elco and the others, it's amazing that this is all that was fubared internally when going to a multi-chamber setup (so far anyway).