Talk:Stiff systems
--Byrne 17:20, 25 February 2007 (EST)<review>In the description of the combustion example, it is implied that \(\epsilon\) is a small positive number. I may have missed the statement that is so, but think that should be explicitly stated.
I really like the real time solutions with the stiff and non-stiff solvers. The two graphs are very instructive. Will the uninitiated reader understand that the circle centers depict the discrete approximate solutions and that the gaps between consecutive circles represent the step lengths? The authors do say that they were real time solutions and that may be enough.
Just above the test equation I reread the section beginning "The eigenvalues of the coefficient matrix...." a couple of times. I think that the authors mean:
... none of the eigenvalues \(\lambda_i\) has a positive real part that is at all large....
As I said earlier, I really like the real time graphics of the solutions of the combustion example. I also think that the article treats a very complex issue, namely stiffness, in a comprehensible way.</review>
reviewer B
We have modified the article extensively in order to address the various concerns you raise. Thank you once again for your careful reading of the article.
To authors:
The definition that \[ L = \max |Re(\lambda_i)|. \] be large has a questionable value. Indeed, take the system \[ y_i' = \lambda_i y_i \] and rescale the time \(t\rightarrow a t\). Then the system becomes \[ y_i' = a \lambda_i y_i \] and \[ L = a \max |Re(\lambda_i)|. \] In particular, \(L\) could be made as large as you wish or as small as you wish just by rescaling time. That is, time rescaling could make a stiff system non-stiff and vice-versa, which is not true.
In contrast, the definition that involves ratios of the two eigenvalues is invariant with respect to the rescaling of time and measures the true stiffness of the system.
To reviewer B:
Thank you very much for your constructive comments. We have modified the article in response to them. Please let know if they address your concerns adequately. Once again thanks for your careful review.
You need to start the article with the exact mathematical definition of the stiffness of a system. Instead, you start with an example and with explaining why stiff systems are important, concentrating only on 1-d systems.
Start with something like this (copy and paste to the article):
A system
\[
x'=f(x)
\], \(x \in \R^n\)
is stiff at an equilibrium \(x=x_0\) if
- The equilibrium is stable, i.e., all eigenvalues, \(\lambda_k\), of the Jacobian matrix \(J=Df(x_0)\) have negative real parts (\(\mbox{Re }\lambda_k<0\)).
- The ratio
\[ L = \frac{\max_k |\mbox{Re }\lambda_k |}{\min_k |\mbox{Re }\lambda_k |} \],
- called the stiffness index, is sufficiently large.
This notion applies to non-autonomous systems \(x'=f(x, t)\) via the time-dependent Jacobian \(J(t)\) and stiff index \(L(t)\). Here, a system may be stiff on some time intervals, but non-stiff on others.
Then you say that stiff systems are important in IVP, give your example and motivation, describe problems encountered by ODE solvers, etc.
reviewer C
Thanks for these helpful comments. In most cases they led to straightforward changes, but some comments are embedded below. The authors.
Overall, this article is very informative, useful, and well written. The moving plots of numerical solutions are especially nice. However, some improvements can be made, notably in the aspect of defining stiffness.
1. In the first section, on mathematical definitions of stiffness, all of the definitions given suffer from difficulties that are cited either here or later. All of those difficulties can be avoided by using the following definition of the stiffness ratio, which is similar to \(L(t_f - t_0)\), but better. For a system with eigenvalues \(\lambda_i\), at least one of which is in the left half-plane, and at a given point \((t,y(t))\), define \[ R = max [-Re(\lambda_i)] \tau_r, \] where \(\tau_r\) is the time step size sufficient to resolve the solution (at that point) with acceptable accuracy. This is similar to \(L(t_f - t_0)\), but replaces the total time interval with the local resolution step size. Admittedly, the latter is subject to interpretation, as it depends on the desired level of accuracy. But it is more relevent, it is scale-independent, and it applies both to nonlinear systems and to systems with eigenvalues in both half-planes. Now one can simply say that a system is stiff at \((t,y(t))\) if \(R\) is large, and this means that the smallest decay time constant, \(\tau_m = min [-1/Re(\lambda_i)]\), is much smaller than the step size appropriate for an acceptable resolution.
You make a good point, but as you note, it is subject to interpretation and the interpretation fits better in the Computational Stiffness section. We have added a sentence about the matter that replaces the sentence you reference in #4 below.
2. That first section should point out that some authors use a definition of stiffness ratio based on \(max|\lambda_i|\), without any restrictions, allowing for pure imaginary \(\lambda_i\). This is a useless definition, because methods for problems with high frequency oscillations require a completely different approach.
3. The Computational Stiffness section almost gets to another definition that would be useful: If the per-step costs of a stiff and nonstiff method are \(C_s\) and \(C_n\) respectively, then a system is computationally stiff if \(R C_n / C_s\) is large (with \(R\) defined as above). This is based on the idea that the total costs for the two methods of an integration over an interval of length \(T\) (surrounding the given time \(t\)) can be estimated as \((T/\tau_r)C_s\) and \((T/\tau_m) C_n\), respectively, and the problem is computationally stiff if the cost is much lower for the stiff method.
We agree. Indeed, in LFS, Measuring Stiffness, Appl. Numer. Math. 1 (1985) 107-119 reference methods are used to quantify this approach. However, in this brief article for the encyclopedia we felt that we could not devote any more space to this matter.
4. In the Computational Stiffness section, first paragraph, what does the last sentence ("This assumption ...") mean? Is this about the imprecision of "smooth, slowly varying" or something else?
5. In the Example section, I see two minor errors. In the third sentence after the equation, use either \(F\) or \(f\) in both places. In the 7th sentence after the equation, I think "increases rapidly to a limit value near 1" should be changed to "increases towards a limit value of 1", since the limit value is 1, not just near 1.
6. In the Stability Issues section, in the 6th sentence, what is the point of saying "the solution quickly becomes easy to approximate in the sense of absolute error"? Since this is about the scalar test equation, the solution quickly goes to zero; why not say that?
7. The use of numbers brackets ([1] etc.) at references that are not in the References list is confusing. I suggest removing them.
Additional comments, 19 March 2007.
As it stands, the roles of time constants and local solution time scale are severely underplayed. May I suggest a compromise -- adding the following short paragraph after the one beginning "Either measure ..."
In terms of damping time constants, \(-1/Re(\lambda_i),\) \(S\) is the ratio of the largest to the smallest of these. An alternative definition is the ratio of the local solution time scale (or resolution step size) to the smallest time constant. This can be more useful when some \(Re(\lambda_i) = 0\) or when some other driving term sets the local solution time scale.
In then next paragraph, I would also suggest that the notion \(S(t_f - t_0) \gg 1\) be dropped. There is no justification whatever for that (being scale-dependent) as a definition of stiffness.
The definitions here still almost completely ignore the crucial role of per-step costs. Again how about a compromise -- the following two sentences added to the last paragraph of the Computational Stiffness section:
More specifically, the choice of methods must include costs of taking a step with a suitable stiff method vs that of a nonstiff method. The gain in step size achieved with a stiff method is offset by its higher per-step cost, and so for a mildly stiff problem a nonstiff method may be cheaper.
3/19/07 To Reviewer C: We are once again grateful for your comments. We have incorporated each of them in the article.
3/27/07 Comment by Reviewer C: The definition of minimum time constant was left out, so I inserted that at the end of the added sentence in the first section. In the same sentence, there is an inappropriate link to an article on "Mind" which should be eliminated.
