It seems all the water spreadsheets require tedious manual addition of salts and acid. Why can't this be automated easily to reach target? It is very time consuming and bothersome to constantly enter value and guess, though it gets better with time/experience, still automation would be beneficial.
Shouldn't spreadsheet be automatic
- Thread starter esahlong
- Start date
Help Support Homebrew Talk - Beer, Wine, Mead, & Cider Brewing Discussion Forum:
cyanmonkey
Well-Known Member
- Joined
- Nov 9, 2014
- Messages
- 1,074
- Reaction score
- 153
Beersmith will do this.
I agree there is a good deal of manual entry that you must do; however, if you know of a way to automate these things then I would certainly be pleased to check it out. Identifying an issue without proposing a solution just kind of makes this thread a complaint thread. 
Perhaps the better way to put this out there would be to ask anyone if they're aware of a spreadsheet that automates some of the need to enter things manually. In truth, I cannot see how this would be automated though.
Edit: I had no idea BeerSmith did it. I'm going to check this out today.
Perhaps the better way to put this out there would be to ask anyone if they're aware of a spreadsheet that automates some of the need to enter things manually. In truth, I cannot see how this would be automated though.
Edit: I had no idea BeerSmith did it. I'm going to check this out today.
cgrivois
Well-Known Member
Would be nice to enter your water profile, enter target profile, enter grist and water amounts then mash the enter button.
GotPushrods
Well-Known Member
Sometimes solving a multivariable equation isn't as straight forward as you think. If there is a solution at all. You could use the Solver tool of Excel, but it's all really an academic exercise anyway.
The way I see it, the first 10 to 20 seconds of work get it close enough. After that, nobody has a palette that will discern the differences. A few ppm is irrelevant to the end perception (which is the whole point, right?) and is another good example of why chasing an exact profile is pretty futile.
The way I see it, the first 10 to 20 seconds of work get it close enough. After that, nobody has a palette that will discern the differences. A few ppm is irrelevant to the end perception (which is the whole point, right?) and is another good example of why chasing an exact profile is pretty futile.
While I agree that it is nice to push a button and have an answer pop out, I also find that it only takes a few seconds to work through a few potential additions to produce a profile that is 'close enough'. It is foolish to try and get additions down to the gnat's ass. Brewing water chemistry doesn't have to be that exacting and there is no way that anyone would be able to tell that the chloride ion was at 45 ppm vs. 40 ppm.
Another thing that the users of Bru'n Water have been pushing for is to get rid of any sort of macro features (drop down boxes and sliders), so that the program will work on more computer platforms. Adding Solver to Bru'n Water is exactly the wrong direction according to their comments. (By the way, a user did create a Solver and it works well, but that isn't something that the majority want).
philipCT
Brewniversity student
- Joined
- May 10, 2013
- Messages
- 771
- Reaction score
- 171
I do agree that some kind of rough fit should be possible. Maybe not using spreadsheet tools, but certainly in an app.
I know that there are whole disciplines of mathematics dedicated to solving multivariate equations, but if you left the tolerances fairly loose, it seems like some of the work could be taken out of the front-end of matching a desired profile. Maybe we have some AI programmers on the team?
On another note, if Beersmith has water chemistry, I'll be horn-swaggled, but PLEASE let me know if I'm wrong.
Edit/Postscript: I do agree with Martin that once you get used to Bru'nwater, it's pretty quick and easy. I did find that it took me some time to really understand how to use it though, but I am kind of dull
I know that there are whole disciplines of mathematics dedicated to solving multivariate equations, but if you left the tolerances fairly loose, it seems like some of the work could be taken out of the front-end of matching a desired profile. Maybe we have some AI programmers on the team?
On another note, if Beersmith has water chemistry, I'll be horn-swaggled, but PLEASE let me know if I'm wrong.
Edit/Postscript: I do agree with Martin that once you get used to Bru'nwater, it's pretty quick and easy. I did find that it took me some time to really understand how to use it though, but I am kind of dull
Last edited:
- Joined
- Jan 23, 2008
- Messages
- 65,431
- Reaction score
- 10,793
"New Feature: BeerSmith now includes a Calculate Best Additions button that automatically finds the best water additions to match your base and target water profile. It does this using a "best fit" linear programming model."
http://www.beersmith.com/help2/index.html?water_profiler.htm
FWIW I agree with OP and vote for an auto calc button.
My feeling is if your going to use a computer to crunch some of the numbers you might as well let the computer crunch all of the numbers be as exact as possible in a fraction of the time it would take to do it by hand. Now you (theoretically) know exactly what you need to achieve exactly what you want, you can then make judgement calls and be as precise or imprecise as you want.
My feeling is if your going to use a computer to crunch some of the numbers you might as well let the computer crunch all of the numbers be as exact as possible in a fraction of the time it would take to do it by hand. Now you (theoretically) know exactly what you need to achieve exactly what you want, you can then make judgement calls and be as precise or imprecise as you want.
Where can this solver be downloaded?
Does Beersmith's water feature compare better or equal to the other spreadsheets?
Solver comes with Excel but you have to do something minor to 'install' it and make it available. Or you did. At one time Microsoft took it out and you had to get it from its manufacturer. Then they put it back in. It appears under the 'Tools' menu when it is ready to go. If it is there you can start using it. If it isn't then do a search for it in Excel help.
It is very powerful. I use it all the time to predict mash pH (it iteratively tries trial pH's until it finds the one that zeroes the sum of all the mash component proton deficits). I can also use it for synthesis of ion profiles but I seldom do that anymore unless someone has a question here. It systematically tries sets of salts searching for the set that minimizes the rms error between desired and achieved ion profiles. And it does all kinds of other stuff too like computing the amount of acid needed for a particular mash pH given a given set of grains or how much carbon dioxide needs to be dissolved to bring a given water to saturation WRT the atmosphere.
I find that in general people are afraid of it for some reason. But some of the problems brewers are interested in can't be conveniently solved (other than by groping as discussed here) without it. I asked at an MBAA talk how many in the room used it. Only a few hands, e.g. Karl Siebert's, went up.
It is very powerful. I use it all the time to predict mash pH (it iteratively tries trial pH's until it finds the one that zeroes the sum of all the mash component proton deficits). I can also use it for synthesis of ion profiles but I seldom do that anymore unless someone has a question here. It systematically tries sets of salts searching for the set that minimizes the rms error between desired and achieved ion profiles. And it does all kinds of other stuff too like computing the amount of acid needed for a particular mash pH given a given set of grains or how much carbon dioxide needs to be dissolved to bring a given water to saturation WRT the atmosphere.
I find that in general people are afraid of it for some reason. But some of the problems brewers are interested in can't be conveniently solved (other than by groping as discussed here) without it. I asked at an MBAA talk how many in the room used it. Only a few hands, e.g. Karl Siebert's, went up.
I agree, it would easily allow one to play around with the variables. (As opposed to manually "starting over" if you will.)
cyanmonkey
Well-Known Member
- Joined
- Nov 9, 2014
- Messages
- 1,074
- Reaction score
- 153
It compares, but not perfect with spreadsheets as much as they don't compare with each other. It is very close to Bru 'n Water, however.
I don't see where this would save a lot of time in the end, or make it any easier. You would still need to enter your grains, sparge and mash volumes, preference of which additions you would be using..... gypsum, CaCl, Baking soda, Lactic or phosphoric acid. You would have to designate dilution % with RO and pick a style guideline..... By that point, the only thing that is left is to put a couple numbers in the gypsum or CaCl box....... that is about the least tedious aspect of the entire process in my opinion.
I always save my spreadsheets for my beers and then just reopen older ones to tweak anything that might be new with the latest version of a batch.
I always save my spreadsheets for my beers and then just reopen older ones to tweak anything that might be new with the latest version of a batch.
cyanmonkey
Well-Known Member
- Joined
- Nov 9, 2014
- Messages
- 1,074
- Reaction score
- 153
Another note, from what I can tell Beersmith doesn't predict mash ph or acid additions, just salts.
yes, but.................it's a 'dumb' best addition calculator. What I mean is, it will match up the sulfate level, etc- but the flavors won't be optimized and neither will the mash pH. In other words, to match the calcium level, it may ask for calcium carbonate (chalk) which you generally never want to use and this will have a flavor impact.
There really isn't any substitute for doing it yourself- but we're talking about maybe three minor additions total for each beer.
It's worse, from what I've seen. Ridiculous additions that won't give the beer the correct flavor although the ions sort of would mash up.
Mash pH is really the most crucial part of the whole thing. It really doesn't matter if calcium is 48 ppm vs 98 ppm, or if sulfate is 137 vs 155, for example, if the mash pH is 5.9.
A lot of mis-understanding and mis-information here.
A good automatic spreadsheet would target what the user enters and use that which the user selects.
It would make it very easy to play-around with different target profiles and mash pHs.
The different platform argument seems a little absurd as what platforms don't support spreadsheets? Bru N Water seems to run fine on Mac, Windows, Linux...
The arguments of "it only takes a couple minutes" and "you don't have to be precise" are invalid as if a computer is doing the work, then let the computer be as precise as the user wants. The length of the time the computer works on a problem should be selectable also.
I really don't see the problem, except that no-one has pursued this. (Well almost no-one according to Martin.)
Most people here are being negative about a positive improvement to the water spreadsheets. So there's a button on the spreadsheet, click it if you value the improvement or ignore it if you don't.
A good automatic spreadsheet would target what the user enters and use that which the user selects.
It would make it very easy to play-around with different target profiles and mash pHs.
The different platform argument seems a little absurd as what platforms don't support spreadsheets? Bru N Water seems to run fine on Mac, Windows, Linux...
The arguments of "it only takes a couple minutes" and "you don't have to be precise" are invalid as if a computer is doing the work, then let the computer be as precise as the user wants. The length of the time the computer works on a problem should be selectable also.
I really don't see the problem, except that no-one has pursued this. (Well almost no-one according to Martin.)
Most people here are being negative about a positive improvement to the water spreadsheets. So there's a button on the spreadsheet, click it if you value the improvement or ignore it if you don't.
I am going to lean the direction of the feedback of the Bru'n users. I want absolute control of what I am putting into my beer and have found that there maybe several directions to take in achieving a given goal - all with slightly different compromises... an example might be using pickling lime v baking soda (more calcium v more sodium) or using Epsom v gypsum (adding magnesium v calcium) in achieving my target sulfate and chloride levels.
The solver might be a solution for some people... but you lose some creativity in the process.
The BeerSmith calculator does not help to manage mash pH, and that is the critical missing element. Now - I would get behind Bru'n Water capability licensed into BeerSmith. That might be an interesting twist.
EDIT: One more comment. If everyone donated to use Bru'n Water, then the arguments would be much stronger. The reality is that the vast majority use the free spreadsheet - which limits the impetus to make massive (and effectively programming and debugging Excel across those platforms is a challenge) interactive elements to the program.
The solver might be a solution for some people... but you lose some creativity in the process.
The BeerSmith calculator does not help to manage mash pH, and that is the critical missing element. Now - I would get behind Bru'n Water capability licensed into BeerSmith. That might be an interesting twist.
EDIT: One more comment. If everyone donated to use Bru'n Water, then the arguments would be much stronger. The reality is that the vast majority use the free spreadsheet - which limits the impetus to make massive (and effectively programming and debugging Excel across those platforms is a challenge) interactive elements to the program.
Last edited:
That makes absolutely no sense at all. Why wouldn't you be able to choose the salts and profiles and pH to target.
It's pretty clear if we are trying to figure out how much calcium chloride and how much calcium sulfate and how much magnesium sulfate and how much sodium bicarbonate to add to even DI water (simplest case) to get desired concentrations of sodium ion, calcium ion, chloride ion etc. given that calcium and chloride are in fixed proportion in CaCl2 and similarly for other salts and that the desired ion concentration may not be balanced that there will be no solution. Put another way if x is the vector of salt amounts and y the vector of desired ion concentrations and A a matrix with entries relating to the amount of chloride in a gram of calcium chloride, the amount of calcium in a gram of calcium chloride, the amount of chloride in a gram of sodium chloride etc. the system of equations we'd need to solve is A*x = y. As noted there may be (in fact, probably isn't) any solution y = (1/A)*x and so we seek the best solution i.e. the one that minimizes |A*x -y| or one that minimizes some other cost function calculated from the matching errors. This is where Excel excels. Add the nuance of allowing the pH to shift in order to get a better match and you have a problem that only Excel (or similar linear programming software) can solve.
I'll make bold and say that if you aren't using Excel you might as well guess as without it you aren't going to get very good solutions. And, BTW, that's what I usually do. I only fiddle with spreadsheets because I'm interested in finding out whether it is possible to build a good one (I am doubtful at this point).
What makes no sense? BeerSmith only manages to solve ion concentrations and has no consideration for mash pH, other than the ability to notate the value.
Relative to the spreadsheet, anyone is welcome to create their own solver that rides on top of Martin's work. What doesn't make sense is the argument that this should be done in a market where the vast majority of users are not paying for the functionality. I would also submit that such interactivity is better served in an app (like a website) to avoid cross-platform issues and cross-application issues (Office v Freeware solutions).
I guess I am happy that for once we are discussion functionality rather than accuracy of estimates.
Relative to the spreadsheet, anyone is welcome to create their own solver that rides on top of Martin's work. What doesn't make sense is the argument that this should be done in a market where the vast majority of users are not paying for the functionality. I would also submit that such interactivity is better served in an app (like a website) to avoid cross-platform issues and cross-application issues (Office v Freeware solutions).
I guess I am happy that for once we are discussion functionality rather than accuracy of estimates.
philipCT
Brewniversity student
- Joined
- May 10, 2013
- Messages
- 771
- Reaction score
- 171
Oh, yeah. Can't believe I forgot a about that. I'm now a religiously fanatic Brun'water user, so I don't know if I can adopt the BeerSmith tool for that - even though I use BeerSmith for nearly everything else, but maybe I'll take another look at it.
cyanmonkey
Well-Known Member
- Joined
- Nov 9, 2014
- Messages
- 1,074
- Reaction score
- 153
I'll use Beersmith to calculate and then import into Bru 'n Water to fine tune. It's nice for me to hit a button and there it is as opposed to starting from zero. Beersmith just gives me a head start.
So - that is different than an "Easy" button. To be clear, I have no issue with auto-generating suggested mineral additions, but would not likely use the feature unless I was out some specific mineral salt. Perhaps the curmudgeon is coming out in me - I have been using BWS for a couple of years and am pretty accurate as to what concentration of something will yield in ions... takes me just a few minutes at best to produce my first round and I might revisit it in planning should I want something perhaps more hop forward or more malty.
My guess is that if the tool were available you'd use it. A few minutes vs. a few seconds ... You would then be using the tool to enhance and tweak your profiles.
What is BWS?
Martin commented that someone did make a solver for his spreadsheet. How much time/work was put in? I'm sure that person wasn't paid and just did it for fun.
The problem with charging for something like this is that it's not an application. It's a spreadsheet with a bunch of formulas.
Also, how mature is the science behind it?
The Brewers Friend calculator seems to be in a position where it would benefit from something like a solver/addition calculator.
No argument about application vs spreadsheet. Unless the application is web based you'll also get into platforms.
People seem happy to just have "something". That doesn't mean that the something that people have is correct, as long it gets them within range, it's all they have to go on.
None. Solver is part of excel. All you do is tell it what you want it to do to a target cell (e.g. zero proton deficit) and which cell or cells you want it to adjust (say amounts of hydrochloric acid and sulfuric acid) subject to any conditions you want (e.g. 2 times as much hydrochloric as sulfuric).
You already paid Bill Gates for it when you bought Office.
Very. People have been working optimization problems like this for as long as there have been computers. For pathological problems Solver cannot find a solution or finds a local minimum (maximum) instead of a global one. So whenever someone tells you that Microsoft never produced any software that wasn't a POS, tell them about Solver (except that they didn't create it).
cyanmonkey
Well-Known Member
- Joined
- Nov 9, 2014
- Messages
- 1,074
- Reaction score
- 153
That's actually another Excel tool (also found in the Tools) menu which allows you to set one cell to a desired value by changing one other cell. It is thus an extremely simplified version of Solver which allows you to set a cell to min, max or specified value by varying dozens (not sure what the limit is) of other cells subject to dozens of constraints.
Martin said that a user created a solver for bru'n water, implying something separate from the standard excel Solver add-in.
The math behind solving optimization problems is way more advanced than what is needed for making automated salt/acid addition calculations. Things like getting as 'close' to a desired ion concentration profile are fairly simple (even taking into account things like requiring all the additions need to be positive) and can be solved in milliseconds.
More sophisticated approaches like trying to match a predicted pH to a target pH are also quite feasible since the number of variables in these problems is relatively small by modern standards (maybe a dozen if you include a bunch of different salts and acids), but how worthwhile solving for those will be depends largely on how accurate the pH prediction is, or how well the errors in the pH prediction can be modeled.
Let's say that there are optimization algorithms far more sophisticated than the simple matrix Newton's method (Moore-Penrose iteration) needed to solve the profile matching problem and that the Solver that comes with Excel is capable of doing several of these. Solver can also be expanded to solve the huge hairy problems. If the point is that Solver is, algorithmically, overkill, it is. It's just much easier to invoke it than to write the VBasic code to do Moore-Penrose. Someone producing a product for which he is charging money might want to build in the algorithms.
Actually this problem (predicting mash pH, assuming that's what you meant) is much simpler than the profile matching problem as no matrix algebra is required. One simply computes the proton deficit for each mash component as a function of pH, adds them up and then finds the pH that causes the sum to be 0. Total proton deficit is monotonic in pH so the pH which zeroes it is easily found by root bisection, Newton's method or any of the Solver algorithms.
The worth of the method depends on the proton deficit models. The model is very robust for the water's alkalinity and for any added acids or bases. A simple Taylor series expansion about the DI water mash pH for a given malt is robust if the coefficients were obtained from the titration of a sample of the malt one is going to brew with. Using coefficients from Maltster A's Pilsner malt when you are actually mashing Maltster's B Pilsner malt is less robust and using coefficients from flaked barley for flaked oats even less so.
Because the models' proton deficits are simple error propagation is also quite simple. In other threads from a week or so ago I published some histograms of predicted pH based on assumptions about the errors in individual mash components proton deficits.
Yes, I agree calling the Excel solver is much easier than writing one's own optimization code (assuming the Excel solver can handle things such as non-negativity constraints and the like - I haven't played around with it much). My point was more along the lines that by modern optimization standards optimizing over a dozen or so variables is very simple and there are multiple 'off-the-shelf' techniques that could be used for most any optimization objective function one might be interested in (many with publicly available source code).
For example, Newton's method works fine if you have a twice differentiable objective function without any constraints, but even something like requiring that the salt/acid additions are non-negative puts you outside the realm of problems that can be solved with Newton's method. That being said, however, there are multiple 'mature' optimization techniques which can handle the constrained version of the problem with nearly identical computational cost.
More generally, when measuring how close Ax-b is to zero, the sum of squared errors is just one option. Really, any distance function, such as the sum of the absolute values of the errors or the maximum absolute error of any one ion concentration could be used in its place. Each choice of distance function will give a different solution (unless there exists an exact solution Ax=b) and each has pros and cons depending on what sort of solution we're looking for. For example, minimizing the maximum absolute error of any one ion is insensitive to small errors away from the target but ensures that all ions will be within a certain range of the target. These other problems are efficiently solved by standard optimization techniques but can't be solved by with something like Newton's method since the distance functions are no longer differentiable.
Well, I guess this is semantics, but 'more sophisticated' was referring to an optimization problem where your salt/acid additions are optimized to 1) put the final ion concentrations within some predefined range or make them 'close' to a desired concentration profile and 2) trying to make the predicted mash pH as close as possible to a target pH. The 'worthwhile' comment simple alludes to the fact that we can always make a mathematical model and solve an optimization problem to 1e-12 precision or whatever, but if the measurements that we're basing the model on are only accurate to within x% then we might not be gaining much over what we'd get from 2 minutes of manually twiddling salt/acid addition values in a spreadsheet.
I haven't been reading the forum as regularly as I used to, but I will check out the histograms. If we can make a good estimate of the proton deficit distribution, then it should be possible to calculate "optimal" salt/acid additions which take into account sensitivities to potential errors and produce solutions which will "most likely" give a target pH in a desired range.
Yes, it can. There is a check box which constrains any cells being varied to the +ive domain. One can impose more involved constraints (cell = constant, cell < constant, cell > another cell etc.)
Certainly the functions of interest here are differentiable and continuous and the non zero constraint is easily implemented. If your system is Ax = b with A the matrix of partials at x0, your initial guess at a solution, then your correction to x0 is ∆x = A#(Ax0-b) (A# is Moore-Penrose pseudo inverse of A). If ∆x carries any element of x below 0 you limit that element to 0 and carry on. I won't say that there aren't situations where that might back you into a corner distant from the best solution but it's always worked for me on these problems (and actually any other I've applied it to) anyway. Maybe I shouldn't call this Newton's method but Newton's method is what you get if you do it in 1 dimension.
In general, for ion concentration matching, I prefer the error function to be the rms of p[Ci]/wi in other words, to minimize the weighted geometric error (Ci is the concentration of the ion, wi the weight assigned to it and p the -log operator. As this function is clearly differentiable WRT each element of x Moore-Pensrose is suited. More to the point, Solver can certainly handle it. If I want to minimize the error with respect to one of the ions I just put a bigger weight on it.
Solver is capable of doing problems like this. You can, for example, ask it to minimize proton deficit (predict mash pH) while getting the calcium ion concentration as close to 100 as possible trimming any unneutralized alkalinity with sauermalz.
The beauty of Moore-Penrose (Roger P is on my mind as I just saw 'Theory of Everything') is that the error analysis is built in. A# = GDOP*A_tranpose where GDOP is the dilution of precision matrix so you have to compute it anuyway to get to A# (you don't really, of course if you used SVD on A but it's trivial to do so). If you have estimates for errors, even for consider parameters, it is possible to get covariance estimates for the things you are solving for.
I
They are no big deal but represent a sort of first step in this direction. The bottom line was that if you knew your base malt parameters well it didn't matter if you were a bit squishy on your specialty malts for no more sophisticated a reason than that they are used in relatively small quantity.
https://gist.github.com/jcipar/16c5663f49d9b4824f3d
This is a GIST script that was written to solve for salt requirements, works independent of Bru'n although he is loading a Bru'n profile target. It ignores mash pH. Command Line and no UI.
This is a GIST script that was written to solve for salt requirements, works independent of Bru'n although he is loading a Bru'n profile target. It ignores mash pH. Command Line and no UI.
I guess I don't know why anyone would want to go to the trouble to develop something like this when it is built into Excel except for the joy of exploration. Or is it impossible to use Solver with Bru'n because of the locked cells?
I honestly haven't tried. That might be the case... will see what happens here.
I talked with the guy who wrote that script - he is looking for a solution similar to the OP here in that a simple way to compute the so-called ideal mineral additions to achieve a profile. The interesting part of the discussion is the several ways to achieve that profile and prioritization of specific ions and specific mineral additions to achieve that, while reserving any acid/alkali from the scenario for a later mash pH correction... It is very much the BeerSmith approach.
I talked with the guy who wrote that script - he is looking for a solution similar to the OP here in that a simple way to compute the so-called ideal mineral additions to achieve a profile. The interesting part of the discussion is the several ways to achieve that profile and prioritization of specific ions and specific mineral additions to achieve that, while reserving any acid/alkali from the scenario for a later mash pH correction... It is very much the BeerSmith approach.
Yeah to confirm, the Solver add-on will not function if the worksheet is protected or the acting cells are locked. This is with a new Office 360 installation on Windows.
I'd argue the function of interest here being something like the sum of squared errors is more out of mathematical convenience and historical precedent than having much physical meaning for the problem. The squared error is making an inherent statistical assumption that the noise in the model is gaussian. For something like least squares regression where you're finding one set of parameters to fit a large number of data points to a model the gaussian assumption is certainly reasonable due to central limit theorem arguments. When you only have a single 'data point' such as a target ion concentration you're trying to match this isn't necessarily the case.
What you're describing will sometimes work and sometimes won’t depending on the geometry of the constraint set. For minimizing a quadratic objective function, such as |Ax-b|^2, this certainly can be solved in closed form using a Moore-Penrose pseudo inverse if the system is unconstrained. Because the objective function is quadratic the Newton direction will always point from the current value of x to the unconstrained solution. As a result, when you add constraints you’re going to move from the initial point towards the unconstrained solution until you hit a constraint boundary (or arrive at the unconstrained solution if it lies in the feasible set). Assuming you hit a constraint boundary, you now need to proceed along the constraint boundary; for simple constraints like x>=0 you can remove variables from the ‘active set’ of variables we’re optimizing over once they hit the boundary and continue with a reduced dimensionality problem, but for more complicated constraints this won’t necessarily work. For example, if we want to limit the maximum concentrations of some ions/acids then you’ll have a constraint of the form Mx <= y. Once this is combined with the constraint x>=0, you now have ‘corners’ in the feasible set where you can get stuck, so you’ll potentially need to return variables to the active set and move along a new facet of the constraint set. But, at that point, you’re basically just doing a bizarre form of quadratic programming.
I’m not totally sure what you’re defining as your error function, but assuming it’s sum_i (p[Ci]/wi – p[Ti]/wi)^2 where Ci is the concentration we’re trying to fit and Ti is the target, then you’ll have to be a bit careful here if you try to apply Newton’s method as this is no longer convex wrt the Ci variables. As a result, the Hessian may not be positive definite and the Newton direction may not give a descent direction. The Excel solver is likely using a version of gradient descent, which should at least give a local minimum, but not necessarily a global minimum.
edit: Rereading this, I'm assuming you mean you do a change of variables yi = p[Ci], solve an unconstrained weighted least squares problem for the yi variables and then just take Ci = exp{-yi}. If so, then sure this can obviously be done in closed form and an optimization method isn't needed.
Right, the fact that second order methods like Newton’ or interior point methods (which is basically an augmentation of Newton’s method to handle non-smooth constraints/functions) include the Hessian information (your GDOP is the inverse of the Hessian) is why they will typically be orders of magnitude faster than a first order method like gradient descent. The only downside to them is that the Hessian grows by the number of variables squared, so for large scale problems second order methods aren’t feasible.
Similar threads
