Home » Analysis and Reconstruction » Tracking & Vertexing » Questions about fitting in MarlinTPC
Questions about fitting in MarlinTPC [message #1975] 
Fri, 07 May 2010 01:28 
killenberg Messages: 125 Registered: July 2005 Location: CERN 



Hello,
Christoph had a few questions about the track fitting in MarlinTPC and we both
think they are of general interest. So I answer them here.
Quote:  1.) Is there a "simple" chi^2 fit processor for straight lines?

You can use the TrackFitterSimpleChiSquareProcessor and fix the curvature to 0. Use the parameters OmegaStart=0 and FixOmega=true.
An other alternative is the LinearRegressionProcessor. The linear regression is equivalent to a chi^2 minimisation with all errors being 1.
Quote:  2.) A chi^2 helix fitter seems to be implemented in TrackFitterSimpleChiSquareProcessor. Why are defocussing and diffusion parameters of the fit? To fit a function one only needs the functions, its parameters and errors. Why are there other parameters, which are not part of the fit?

The SimpleChiSquare fitter does not use the errors of the hits (for instance to be able to run with the current hit finders which do not set the errors. At least the TopoFinder does not).
In a first version all errors were set to 1, but it turned out that the errors in xy and z can be very different, and they both depend on z.
The SimpleChiSquare fitter estimates the errors as
sigma_xy = sqrt( TransDefocussing^2 + TransDiffusionCoef^2 * z )
sigma_z = sqrt( LongDefocussing^2 + LongDiffusionCoef^2 * z )
where z is the drift distance. The names are missleading. Originally the idea was to use defocussing and diffusion coefficients from the conditions data. But it turned out that calculating the required values from them is not straight forward.
So "Defocussing" is the intrinsic detector resoution at zero drift, and "Diffusion" is the drift distance (diffusion) dependent part of the resolution.
The documentation is definitely wrong, and we probably should also change the names to be clear.
Quote: 
3.) Is it corrent that neither dEdx (and its error) nor the covariance matrix are stored, or not even calculated?

Unfortunately yes. dEdx is not calculated at all. The covariance matrix is available in the Minuit fit, but it is not stored. The fitter is still alpha and got stuck in the phase when I was struggling that the fit converged at all. In the end it turned out that there was a problem with the start parameters and the pattern recognition (track finding). So the fit should converge, but I did not have the time to finish it yet.
Quote: 
4.) The base class / base fitter is TrackFitterSimpleChiSquare. Is it correct that some things are not cleanly implemented, i. e. hardcoded?

The actual base class is TrackFitterBase. This imlements a generic version of calculateResiduals() (residual = distance perpendicular to the helix).
TrackFitterSimpleChiSquare inherits from this and implements the fitting function.
TrackFitterSimpleChiSquarePads inherits the fitter function from TrackFitterSimpleChiSquare, but reimplements calculateResiduals() to return the residuals along the pad row.
Minuit needs limits for the track parameters to converge reliably. These are currently hardcoded:
 omega=[1 .. 1] (only tracks with radius larger 1 mm)
 D0=[100 .. 100] (impact parameter + 100 mm)
 phi=[2 pi .. 2 pi] (two full circles, no real limit)
 tanLambda=[100 .. 100] (only tracks with less than 100 cm z variation on 1
cm on the xy projection)
 D0=[1000 .. 1000] (only tracks with Z0 + 1 m)
The only problematic value is D0, since this is a sort of "vertex constraint". In prototype geometry D0 can be large, depending on the orientation of the readout module in global coordinates.
+ 1 m in z is fine for the LP, but still hard coded is not good.
How about using + 100 mm around the start value from the seed track, for both D0 and Z0?
The other values should be no real limitation, although it should be in the documentation in case someone has problems in exotic cases.
Cheers
Martin
Martin Killenberg
CERN
martin.killenberg@cern.ch



Re: Questions about fitting in MarlinTPC [message #1977 is a reply to message #1975] 
Fri, 07 May 2010 08:48 
rosemann Messages: 41 Registered: March 2009 Location: hamburg.de 



Hi Martin,
several things come to my mind.
killenberg wrote on Fri, 07 May 2010 01:28 
Quote:  1.) Is there a "simple" chi^2 fit processor for straight lines?

You can use the TrackFitterSimpleChiSquareProcessor and fix the curvature to 0. Use the parameters OmegaStart=0 and FixOmega=true.
An other alternative is the LinearRegressionProcessor. The linear regression is equivalent to a chi^2 minimisation with all errors being 1.

I think we should better do this again. Of course it's possible to write the fitter in a way that it can handle both kinds of model (straight line and helix). But it is probably more suggestive to use the processor name to specify it's type. And then use the (still to be defined) track type of the candidate to determine, whether the fitter can handle this kind of track.
Moving on to errors:
killenberg 
Quote:  2.) A chi^2 helix fitter seems to be implemented in TrackFitterSimpleChiSquareProcessor. Why are defocussing and diffusion parameters of the fit? To fit a function one only needs the functions, its parameters and errors. Why are there other parameters, which are not part of the fit?

The SimpleChiSquare fitter does not use the errors of the hits (for instance to be able to run with the current hit finders which do not set the errors. At least the TopoFinder does not).
In a first version all errors were set to 1, but it turned out that the errors in xy and z can be very different, and they both depend on z.
The SimpleChiSquare fitter estimates the errors as
sigma_xy = sqrt( TransDefocussing^2 + TransDiffusionCoef^2 * z )
sigma_z = sqrt( LongDefocussing^2 + LongDiffusionCoef^2 * z )
where z is the drift distance. The names are missleading. Originally the idea was to use defocussing and diffusion coefficients from the conditions data. But it turned out that calculating the required values from them is not straight forward.
So "Defocussing" is the intrinsic detector resoution at zero drift, and "Diffusion" is the drift distance (diffusion) dependent part of the resolution.
The documentation is definitely wrong, and we probably should also change the names to be clear.

The "RowBasedHitFinder" calculates the right covariance matrix for the hits [actually I'm implementing this right now, since the z error was missing information from the pulse level  which is committed to the LCIO head already].
Actually I can't really follow your idea of parametrising x and y errors in terms of drift distance z.
The uncertainty of the position measurement (x,y,z) is determined by the reconstruction algorithm, since these values are computed by it.
killenberg 
Quote: 
3.) Is it corrent that neither dEdx (and its error) nor the covariance matrix are stored, or not even calculated?

Unfortunately yes. dEdx is not calculated at all. The covariance matrix is available in the Minuit fit, but it is not stored. The fitter is still alpha and got stuck in the phase when I was struggling that the fit converged at all. In the end it turned out that there was a problem with the start parameters and the pattern recognition (track finding). So the fit should converge, but I did not have the time to finish it yet.

OK. Seems to me, that we possibly should rework the fitting scheme a little. This might turn into a much larger thread, to sort this out  but I think we should get this right. I'm still a bit puzzled about the approach to the "fitting problem", and if it is a good way like it is now.
I'm treading in dangerous waters here, since you obviously invested both time, thought and work into the existing scheme. But maybe we should start with a more pragmatic approach, from which we could draw conclusions about a more generic ansatz.
Concerning the parameter error calculation  this directly comes out of the minimising program, so we would need an interface to it. I don't see right away if there is a simple solution. It might be possible, but thinking straight has become a little harder recently
I hope, I've got this right  but the calculation of the dx should not be too complicated, even for a helix.
In my opinion, the only really unfortunate part comes in, when we associate hits (to identify the charge) with dE. In this case we would need geometrical information about the readout plane layout again; at least for row/pad based schemes. (Maybe I shouldn't try to write this at Friday evening, hopefully my idea comes across)
killenberg 
Quote: 
4.) The base class / base fitter is TrackFitterSimpleChiSquare. Is it correct that some things are not cleanly implemented, i. e. hardcoded?

The actual base class is TrackFitterBase. This imlements a generic version of calculateResiduals() (residual = distance perpendicular to the helix).
TrackFitterSimpleChiSquare inherits from this and implements the fitting function.
TrackFitterSimpleChiSquarePads inherits the fitter function from TrackFitterSimpleChiSquare, but reimplements calculateResiduals() to return the residuals along the pad row.

Correct me, if I'm wrong. In principle the whole base class idea is to have this as a generic interface, to provide a common name for the "residual" and "distance" access (I use the naming of Ralf and Matthias, hopefully it is known beyond our borders...). Eventually the current naming scheme for this "track with/without hit"  thing is not so good. I don't know if there is any convention concerning this. But I think using the same name, but different arguments to distinguish is not a good idea:
[code title=TrackFitterBase.h]
/** Return the distance of the hit \c testHitNumber of the testTrack
* to the point of closest approach on the referenceTrack wrt.\ the
* hit.
[...]
*/
virtual EVENT::DoubleVec calculateResiduals(EVENT::Track *testTrack,
unsigned int testHitNumber,
EVENT::Track *referenceTrack = NULL);
/** Return the distance of the given hit to the test track
* where this hit has been removed.
* The residual is calculated as the distance
* to the point of closest approach on the referenceTrack wrt.\ the hit.
*/
virtual EVENT::DoubleVec calculateResiduals(EVENT::TrackerHit *testHit,
EVENT::Track *trackWithoutTestHit);
[/code]
And besides the names  has there ever been a discussion about the definition? I would think, that this is one of the common topics, that must have arisen in result discussions before. So I would somehow think, that we should have the standard definition in  and everything else is up to the "user". Having different meanings by processor name seems not a very good idea to me.
And finally the hardcoded stuff:
killenberg 
Minuit needs limits for the track parameters to converge reliably. These are currently hardcoded:
 omega=[1 .. 1] (only tracks with radius larger 1 mm)
 D0=[100 .. 100] (impact parameter + 100 mm)
 phi=[2 pi .. 2 pi] (two full circles, no real limit)
 tanLambda=[100 .. 100] (only tracks with less than 100 cm z variation on 1
cm on the xy projection)
 D0=[1000 .. 1000] (only tracks with Z0 + 1 m)
The only problematic value is D0, since this is a sort of "vertex constraint". In prototype geometry D0 can be large, depending on the orientation of the readout module in global coordinates.
+ 1 m in z is fine for the LP, but still hard coded is not good.
How about using + 100 mm around the start value from the seed track, for both D0 and Z0?
The other values should be no real limitation, although it should be in the documentation in case someone has problems in exotic cases.

I somehow still have a little bit of trouble with Minuit working this way. If both the track finding works reliably and the hit errors are reasonably well described the fit should converge. We could think at some point about outlier rejection, downweighing, blabla.
But eventually Chi^2 fitting is not a heavily complicated problem. In principle it comes down to matrix inversions and a few matrix multiplications. I can have a closer look into that part.
Cheers,
Christoph
When you have eliminated the impossible, whatever remains, however improbable, must be the truth. (Sir A.C. Doyle in Sign of Four)



Re: Questions about fitting in MarlinTPC [message #1979 is a reply to message #1977] 
Mon, 10 May 2010 08:18 
killenberg Messages: 125 Registered: July 2005 Location: CERN 



Hello Christoph,
Quote: 
I think we should better do this again. Of course it's possible to write the fitter in a way that it can handle both kinds of model (straight line and helix).

The chi^2 fitter just happens to be able to do straight lines. For some studies it is necessary to fix the curvature and set it to a specific value, so I implemented this. If one sets it to 0 it's a straight line. I totally agree that we should have a dedicated straight line processor.
Quote: 
The "RowBasedHitFinder" calculates the right covariance matrix for

This is what I expected. So it's time we get a proper implementation of the chi^2 fitter.
Quote: 
Actually I can't really follow your idea of parametrising x and y errors in terms of drift distance z.
The uncertainty of the position measurement (x,y,z) is determined by the reconstruction algorithm, since these values are computed by it.

If you plot the residuals in xy and z against z this is the parametrisation you get. So it's an a posteriori estimate for the mean error (if the individual errors are not available). since the individual errors are there now we definitely should use them.
Quote: 
Seems to me, that we possibly should rework the fitting scheme a little.

Yes, we should to that. I am also not happy with it. I think it's too complicated and tangled. I discussed with several people but could not come up with something simpler that provided the functionality I wanted. We should discuss it in a separate thread.
Quote: 
Concerning the parameter error calculation

It should only be a few lines to read the covariance matrix from Minuit and fill it into the track. I admit I was just lazy looking it up when I still had problems with the convergence.
The same for dEdx: One just has to write a few lines, no principle problem. Maybe one can put a generic calculation into the base class so one does not have to copy it for all the implementations.
Quote: 
I somehow still have a little bit of trouble with Minuit working this way. If both the track finding works reliably and the hit errors are reasonably well described the fit should converge. We could think at some point about outlier rejection, downweighing, blabla.

Yes, once the pattern recognition works Minuit converges fine. And we definitely should have an outlier rejection. Maybe just do a double pass fit?
Cheers
Martin
Martin Killenberg
CERN
martin.killenberg@cern.ch



Re: Questions about fitting in MarlinTPC [message #1980 is a reply to message #1977] 
Mon, 10 May 2010 23:57 
killenberg Messages: 125 Registered: July 2005 Location: CERN 



Hello Christoph,
Quote: 
I think we should better do this again. Of course it's possible to write the fitter in a way that it can handle both kinds of model (straight line and helix).

The chi^2 fitter just happens to be able to do straight lines. For some studies it is necessary to fix the curvature and set it to a specific value, so I implemented this. If one sets it to 0 it's a straight line. I totally agree that we should have a dedicated straight line processor.
Quote: 
The "RowBasedHitFinder" calculates the right covariance matrix for

This is what I expected. So it's time we get a proper implementation of the chi^2 fitter.
Quote: 
Actually I can't really follow your idea of parametrising x and y errors in terms of drift distance z.
The uncertainty of the position measurement (x,y,z) is determined by the reconstruction algorithm, since these values are computed by it.

If you plot the residuals in xy and z against z this is the parametrisation you get. So it's an a posteriori estimate for the mean error (if the individual errors are not available). since the individual errors are there now we definitely should use them.
Quote: 
Seems to me, that we possibly should rework the fitting scheme a little.

Yes, we should to that. I am also not happy with it. I think it's too complicated and tangled. I discussed with several people but could not come up with something simpler that provided the functionality I wanted. We should discuss it in a separate thread.
Quote: 
Concerning the parameter error calculation

It should only be a few lines to read the covariance matrix from Minuit and fill it into the track. I admit I was just lazy looking it up when I still had problems with the convergence.
The same for dEdx: One just has to write a few lines, no principle problem. Maybe one can put a generic calculation into the base class so one does not have to copy it for all the implementations.
Quote: 
I somehow still have a little bit of trouble with Minuit working this way. If both the track finding works reliably and the hit errors are reasonably well described the fit should converge. We could think at some point about outlier rejection, downweighing, blabla.

Yes, once the pattern recognition works Minuit converges fine. And we definitely should have an outlier rejection. Maybe just do a double pass fit?
Cheers
Martin
Martin Killenberg
CERN
martin.killenberg@cern.ch



Goto Forum:
[ ]
Current Time: Tue Nov 19 04:21:15 Pacific Standard Time 2019
