Add docs for recognition, user interface and exports

docs/source/recognition/multicolor.md

 # Multicolor Marker
 
-Here are the docs on how to use the multicolor marker and what its options are
+The multicolor marker is a colored hat that pedestrians wear. For a quick rundown on when to use which type of marker, see **TODO ref to Maiks section**. But in a nutshell, the multicolor marker is easy to prepare and works even with low resolution cameras. It can be combined with an code-marker as well, if some cameras are high-resolution and some are not. Then, low-resolution cams fall back to the multicolor marker in case the code cannot be read.
\ No newline at end of file
+
+There are two places for settings parameters of the multicolor marker. The parameter window as well as the recognition tab.
+
+## Multicolor Parameter Window
+
+:::{figure-md} multicolor-param-win
+![parameter window of multicolor marker](images/multicolor_param_win.png)
+
+Parameter window of the multicolor marker
+:::
+
+In the parameter window, we have a plethora of options:
+
+The first options concern themselves with combinations of the multicolor marker with other markers, namely the hermes marker and code marker. When the multicolor marker is combined with another marker, first the multicolor detection is run. Afterwards, on the area around the detected head, the other marker is searched. The idea is to combine the robust, but slightly inaccurate multicolor marker with a more accurate and "structured" marker.
+
+use black dot
+: The multicolor marker can be used with a black dot added, essentially creating a combination between the hermes marker and the multicolor marker. **TODO: add picture** This gives a bit more structure for tracking and a smaller specific point in the image to locate. If one does use this, this checkbox should be enabled and the size set to the size of the point in cm. Else, this option can be turned off.
+
+use code marker
+: The multicolor marker can be combined with the [code marker](aruco), which is called code marker here. A click onto the `parameter`-button opens the parameter window for the code marker. There the options for the detection of the code marker can be changed. Please refer to the [documentation of the code marker](aruco) for further information on this.
+
+The following three options are all for combinations with either the hermes or the code marker
+
+ignore head without black dot or code marker
+: If this is enabled, all detections where only the colored hat, without the extra feature, was detected will be discarded. If this is not checked, detections on color only are used as well. If e.g. the code cannot always be recognized due to the camera resolution, this should be off. However, in case the extra feature should be recognizable, this could reduce false positives or at least mediocre detections.
+
+use color marker for big error
+: If both, the color point and the feature point (black dot, code marker) are detected and subsequent tracking of the feature point has a high error, enabling this options lets PeTrack redo the tracking, using the color point instead of the feature point.
+
+restrict position of black dot
+: When this option is enabled, the black dot is searched in a smaller area, determined by the expected position of the dot given the angle at which the camera looks at the pedestrian.
+
+From here on, the other options refer to the color marker itself.
+
+auto correct perspective view
+: The multicolor marker always takes the point in the middle of the colored blob in the image. However, since the camera looks at the pedestrian from a certain angle, this is not the same as the middle of the head. Enabling this option activates a correction for this error.
+
+only for export
+: Activating this option means, the correction for perspective view is during the export and not during tracking and recognition.
+
+:::{note}
+If you enable `auto correct perspective view`, it is recommended to also enable `only for export`. Otherwise the correction of the perspective view during tracking can lead to the point going to the edge of the visible head, worsening tracking performance.
+:::
+
+show mask
+: When enabled, we show a mask visualizing the color selection **TODO: true, what about the other mask?**
+
+close radius
+: Closing is a morphological operation to remove small holes. A larger value means, larger holes will be filled. The filter is only applied if `use` is checked.
+
+open radius
+: Opening is a morphological operation to remove small dots or specks. The larger the value, the larger the specks which are removed. The filter is only applied if `use` is checked.
+
+The head detection detects blobs of a certain color. It then filters out some of these blobs on a variety of criteria. Here are some options on those:
+
+area
+: This is the minimum and maximum area in pixels a head is allowed to have. Each blob that has fewer pixels than the minimum or more pixels than the maximum does not count as a detection. When `head size` is checked, PeTrack tries to determine sensible values from the camera position and average head size of pedestrians.
+
+max ratio
+: This is the maximum ration between the long and short semi-axis of the ellipse fitting to the blob. If the ratio is larger, the detection is discarded. This filters out very long very thin blobs.
+
+Finally, another option on color mask selection
+
+opacity
+: This sets the opacity of the mask which shows the current color selection. If `mask` is checked, the selected parts of the mask are translucent. Otherwise, they are white.
+
+## Color Selection
+
+:::{figure-md} color-selection
+![Color Selection part of recognition tab](images/color_selection.png)
+
+Part of recognition tab for color and height selection
+:::
+
+The second part of the settings for the multicolor marker are part of the recognition tab. There we can see the color selection. PeTrack uses the color to encode the height of pedestrians. By giving pedestrians of similar height the same hat color and encoding the average height per group in this color, the accuracy of the coordinates can be enhanced. Especially if the height of the pedestrians has large variations (e.g. kids and adults, or adults and wheel-chair users).
+
+The mappings from color to height are called `map` in PeTrack. This mapping is used for multiple markers, including e.g. the casern marker. Therefore, some of the options here are not relevant for us.
+
+:::{figure-md} color-plot
+![alt text](imageS/color_plot.png)
+
+Color Plot: The rectangles are color regions corresponding to a certain color map. The circles are colors corresponding to a detected point. The `X` shows where the color of the pixel under the cursor lies in the plot.
+:::
+
+The most eye-catching element of this part of the UI is the [color plot](#color-plot). This plots visualizes the existing maps as well as the colors of existing detections and of the pixel under the cursor. The visualization of the colors of the detections can be disabled with the checkbox at the top of the `size and color` area titled `show recognition color`.
+
+One quickly realizes that all detections lie directly in the middle of their respective map. This is by design. The multicolor marker searches e.g. for an orange hat. When it finds such a hat, it just assigns it the color orange. Therefore, all detections from the same map have the same color. This is not the case if you use an older marker, like the casern marker. There, the color needed to be sampled from the image. The `optimize` button includes some functionality to improve this sampling by removing outliers. Accordingly, it has no functionality for multicolor markers.
+
+The other options above the color plot are for changing its visualization. However, the multicolor marker has other visualization means, which are better suited. So no more detail for these options are given.
+
+:::{figure-md} maps-section
+![maps section in recognition tab](images/maps2.png)
+
+The maps section of the recognition tab
+:::
+
+Below the color plot, we see the maps. We see the currently selected map (1 in the [figure](#maps-section)) and the corresponding height (2 in the [figure](#maps-section)). We can change both, what map is selected and what height we want to assign to this map. If we want to add or delete maps, we need to use the according button (3 and 4 in the [figure](#maps-section)).
+
+The different sliders are for an old way of color selection. They are not suited for multicolor markers and can safely be ignored. Similarly, the `color` checkbox can be ignored. Instead, we use **either** the `color range` or the `color picker` button to select a color for the multicolor marker (5 and 6 in the [figure](#maps-section)).
+
+:::{note}
+`color picker` and `color range` do not play well together. That is, a color obtained with one cannot be fine tuned with the other.
+:::
+
+Some general remarks hold for either method:
+
+:::{tip}
+Deactivate `open` and `close` while selecting a color. First try to select the color best you can and then activate the options to improve upon this selection. Using them while selecting the color means you are optimizing 3 parameters at once.
+:::
+
+:::{tip}
+Activate the `mask` from the multicolor parameters and set the opacity to ca. 50%. This enables you to see how well your color selection works (see screenshots later)
+:::
+
+:::{note}
+You need to activate `perform` for the color mask to update!
+:::
+
+### Color Picker
+
+When pressing the `color picker` button, the color picker modus is activated. In this modus, one can `shift + left mouse click` on a pixel to start a color selection and then expand this selection with a normal `left click`. 
+
+:::{figure-md}
+![color picker usage example](images/color_picker_example.webp)
+
+Example usage of color picker. With the mask enabled and `perform` checked, we can interactively select the color, looking at different parts of the image to find still uncovered parts. Note that colors which are "close" to each other may not be easy to differentiate with the color picker (here green and yellow). Then the `color range` dialog might be a better option
+:::
+
+### Color Range
+
+When you click on `color range`, a color selection dialog appears:
+
+:::{figure-md} color-range-dialog
+![color range dialog](images/color_range_dialog.png)
+
+Color range dialog: The dialogue allows selecting the lower bound and upper bound color via an hsv color picker. 
+:::
+
+The color detection works in the HSV (Hue, Saturation, Value) color space. Hue is the "color", i.e. redness or greenness of the color. Saturation how saturated it is. And value, how bright it is. The dialog allows you to select to colors from this space. All colors that lie "in between" your selected colors are then part of your selection. It is noteworthy that the hue part is a value in degrees from 0° to 360°, with red being around zero degree. To allow the selection of red-ish colors as well, there is an option `inverse hue`, which changes the direction of this difference. That is we select colors with hue $h$ such that $h_{min} < h < h_{max}$ if `inverse hue` is not selected and colors with $h < h_{min} \lor h > h_{max}$ if `inverse hue` is selected.
+
+:::{figure-md} color-range-example
+![usage example of color range dialog](images/color_range_example.webp)
+
+Example usage of color range dialog. With the mask enabled and perform
+activated, one can interactively see which colors in the frame are selected and
+which are not.
+:::
+

docs/source/recognition/others.md

-# Other Markers
-
-TODO: Short list of other markers, with capabilities (table from Maik); what additional info they can give, thats it; no usage guide, since we don't actively use any of these
-
-| marker           | robustness | precision | high density | additional info | needed resolution | effort in creation |
-| ---------------- | ---------- | --------- | ------------ | --------------- | ----------------- | ------------------ |
-| casern           | ++         | ++        | o            | +               | o                 | o                  |
-| hermes           | o          | ++        | +            | -               | +                 | +                  |
-| japan            | +          | ++        | -            | o               | +                 | o                  |
-| color/multicolor | +          | o         | -            | o               | ++                | ++                 |
-| aruco            | TODO       |           |              |                 |                   |                    |
-

docs/source/recognition/recognition.md

 # Recognition
 
-Hier wird einmal grob erklaert, was recognition eigentlich so ist. Die genaue Doku passiert pro Marker im folgenden:
+It is possible to use PeTrack to do semi-automatic tracking. That means, one does detect the pedestrians manually and then tracks these automatically. However, it is faster, potentially more accurate and more convenient to let PeTrack handle the detection of pedestrians automatically. For this purpose, the pedestrians usually get a _marker_. That is some kind of special hat, which then gets detected from PeTrack.
+
+While there are a plethora of options available, we suggest using the multicolor or code markers. Please see their respective sites for more information.
 
 ## Contents
 
 :::{toctree}
 :maxdepth: 1
+setting-up
 multicolor
 aruco
-others
 :::
\ No newline at end of file

docs/source/recognition/setting-up.md

+# Set Up Recognition
+
+
+## General Settings
+
+:::{figure-md} general-settings-reco
+![general settings](images/general_settings.png)
+
+The general settings at the top of the recognition tab
+:::
+
+At the top of the recognition tab, you can find the [general settings](#general-settings-reco) of recognition:
+
+perform
+: When checked, this enables the automatic recognition of pedestrians
+
+step
+: This number decides on which frames to run the detection, i.e. if step is set to 2, only every second frame a detection of pedestrians is done. The gaps are filled with tracking.
+
+marker
+: Where the image shows "multicolor marker", we can select the specific marker we want to use
+
+parameter
+: The button parameter opens a pop-up window where we can set the parameters of the selected marker
+
+The `step`-option exists, because recognition is more computationally expensive than tracking. On modern computers this should not be a problem. Since recognition is more accurate than tracking, a step of 1 is recommended, i.e. running recognition on every frame.
+
+## Region Of Interest
+
+The region of interest (ROI) is the part of the image where a detection is
+performed. For the ROI we see the options
+
+show
+: Shows the ROI as a blue rectangle on the image
+
+fix
+: "Fixes" the rectangle, such that it cannot be moved with the mouse
+
+full image
+: Sets the ROI to the full image
+
+adjust automatically
+: Sets the ROI as one head size smaller than the tracking ROI
+
+The ROI can be adjusted in the image, by resizing and moving with the mouse. **TODO: Image or GIF of manipulation of ROI**
+
+:::{warning}
+The tracking ROI must be larger than the recognition ROI. Any detection outside the tracking ROI will create an entirely new, one point trajectory. We recommend setting the tracking ROI first and then using `adjust automatically` for the recognition ROI
+:::
+
+## Height-based Options
+
+At the bottom of the recognition tab there are a few options regarding the height of pedestrians.
+
+:::{figure-md} height-options
+![height options](images/height_options.png)
+
+Height options at the bottom of the recognition tab
+:::
+
+def. height
+: This is the default height. It is used when a pedestrian does not get a height by other means (i.e. from their marker) and for some calculations where the height of a pedestrian is not known already. Ideally, this is set to the average height of pedestrians in your experiment.
+
+dist.
+: prints the height distribution to the console (currently **only works with multicolor marker**)
+
+  ```
+  [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 146.00 - number   1 (1.14%)
+  [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 158.00 - number   9 (10.23%)
+  [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 168.00 - number  31 (35.23%)
+  [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 179.00 - number  39 (44.32%)
+  [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 192.00 - number   8 (9.09%)
+  ```
+
+res. height
+: Resets the height of all pedestrians that was obtained through either a stereo camera or marker ids
+
+res. pos.
+: Resets the position of all points calculated via a stereo camera
+
+read heights from file
+: This button enables you to load a file with the following format. This file gives assigns a height to each marker id (that is the id from the code marker). That individual height is more accurate than an average height assigned via the color of a hat.
+
+  ```
+  # id z/cm
+  987 184
+  988 179
+  993 175.5
+  ```
+
+import markerIDs
+: This button enables you to load a file to assign a marker id (or code id) to a pedestrian with a given petrack-internal id. That can be used if a select few people in the video have a code marker which is never recognized, but they can be identified by the experimenter visually.
+
+  ```
+  # id markerID
+  1 995
+  2 999
+  3 998
+  ```
\ No newline at end of file

docs/source/section/MeinTest.md

@@ -14,10 +14,9 @@ text
 
 With some text asdfjnasöfhaöshf asjföds föijsa följsad kfjsdljflkdsajfjlks jfksa jlkds jösadj föjdsaf ölksj flkajs flksdj fölsaj fölksadj fölkjdsa fölkjdsa fölkdsa fölkadsj fölksaj 
 
-::::{figure-md}
+::::{figure-md} abb
 
 :::{image} ../_static/logo.png
-:name: abb
 :width: 50%
 :::
 
@@ -65,6 +64,8 @@ Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie co
 
 Test fuer ein paar Referenzen, z.B. auf eine [Abbildung](#abb) oder eine [Notiz](#notiz). Dafuer muss `:name: NAME` mit angegeben werden. Fuer eine Formel sollte man `:label:` verwenden und kriegt eine Referenz der Form [](#formel), [](eqn:test). Fuer Ueberschriften (und Absaetze) kann man mit `(heading-target)=` einen Namen definieren. Dabei kommt diese Markierung **vor** dem Target. Die Referenz sieht so aus: [Ueberschrift](#subheading-target), [Absatz](#absatz-target). Links zu anderen Dokumenten (z.B. [](/getting_started.md), [](/getting_started.md#calibration)) gehen natuerlich auch.
 
+Einen DOI-Link kann man Einbinden mit `<doi:10.1016/j.neucom.2012.01.036>`, das ergibt: <doi:10.1016/j.neucom.2012.01.036>
+
 ::::{grid} 2
 :::{grid-item-card}  Title 1
 :class-card: intro-card

docs/source/tracking/export.md

-# Export
-
-- Explaining different export options and what marker and/or camera (stereo) is needed to use them
\ No newline at end of file

docs/source/tracking/tracking.md

@@ -8,5 +8,4 @@ Here is the tracking docs.
 :maxdepth: 1
 setting-up
 parameter-tuning
-export
 :::
\ No newline at end of file

docs/source/user_interface.md

-# User Interface
-
-Screenshot von PeTrack und einfache Erklaerung, was in welchem Bildbereich zu sehen ist (Video, Seitenleiste mit versch. Tabs, ...)
\ No newline at end of file

docs/source/user_interface/user_interface.md

+# User Interface
+
+:::{figure-md} ui-annotated
+![annotated user interface](images/user_interface_empty.png)
+
+PeTrack after opening without project
+:::
+
+:::{figure-md} ui-with-vid
+![ui after loading video](images/petrack_ui_with_video.png)
+
+PeTrack after opening a video file
+:::
+
+PeTrack's UI is made out of a few parts.
+
+## Video View and Video Side Bar
+
+The video view is where the video and possibly visualizations on top of the video are displayed. One can zoom in via the scroll wheel or the lower half of the video side bar. When using the scroll wheel, one scrolls towards the pixel under the mouse cursor. If only parts of the image remain visible due to zooming in, one can navigate in the image by pressing down the left mouse button inside the image and dragging it. One can fit the entire image in view with `Ctrl + 0` or under `View > Fit in window`. Alternatively, one can fit the recognition region of interest to the view with `Ctrl + 1` or `View > Fit in region of interest`.
+
+Furthermore, one can rotate the image using the upper half of the video side bar.
+
+## Player Control
+
+The player control has the following buttons from left to right
+
+play backwards
+: This button starts a backward playback of the video. 
+
+:::{note}
+Backward playback is *a lot* slower than forward playback, because video codecs are optimized for forward playback.
+:::
+
+single frame backward
+: This button jumps one frame backwards
+
+pause
+: Pauses the video
+
+single frame forward
+: This button jumps one single frame forward
+
+play
+: Plays the video (forward direction)
+
+record
+: This button starts a recording. 
+
+:::{note}
+We recommend using the `Export Video` or `Export Video View` options from the `File` menu in the menu bar instead of the recording button.
+:::
+
+Next to these buttons are the boxes labeled `In` and `Out`. If the video is longer than the experiment, one can use these to specify a different starting frame (`In`) or ending frame (`Out`) for PeTrack. PeTrack then treats the video as if all frames before `In` and after `Out` do not exist. The default values are 0 for `In` and the last frame of the video file for `Out`.
+
+Then there is a slider which can be used to scrub through the video.
+
+Last on the line are two boxes `_ @ _ fps`. The first of these boxes shows the current frame and the last shows the framerate at which PeTrack tries to play the video. Both values can be edited by the user, either to jump to a certain frame by changing the current frame or to change th playback speed by changing the fps.
+
+:::{note}
+For changing the fps relative to the original speed, there are options under `View > Playback speed`. The desired playback speed only changes the behavior of PeTrack if `Fix playback speed` or `Limit playback speed` are selected (both under the `View` menu).
+`Fix playback speed` plays at the desired speed, skipping frames if necessary. `Limit playback speed` only ever slows down a video, never skips frames to speed it up.
+:::
+
+
+## Tab View
+
+The tab view is where most settings can be set. Its content depends on the tab selected and is described in the documentation using these. The widths of the tab view can be changed by dragging the border of the tab view with the mouse. With the green arrow in the video side bar, the tab view can be collapsed entirely as well.
+
+## Status Bar
+
+The status bar is mostly empty on an empty project, but once a video is loaded, one can see different data in it, see the following figure:
+
+:::{figure-md}
+![status bar](images/status_bar2.png)
+
+The right side of the status bar shows a lot of information. All information
+shown right from the current framerate is dependent on the current position of
+the mouse cursor in the image.
+:::
+
+Timecode
+: This part of the status bar displays the timecode corresponding to the current frame. Since the timecode is taken from the video file, this does not necessarily correspond to either the real world time nor does it need to start at zero. In stereo recordings, this can also show the time given by a time file.
+
+Current Framerate
+: This shows the framerate at which the player is currently playing. Note that there is sime *intertia* to how fast this display updates. The color indicated whether we play slower than realtime (red) or at realtime or faster (green). (**TODO: we can also click on it to fix speeds... do we want to document that? There now is a more intuitive entry in the menu bar for it**)
+
+Probing Height
+: The following data fields contain some information about the position of the pixel under the mouse cursor. Due to the perspective projection onto the camera, this information can only be given when assuming a certain height, i.e. the position of the pixel under the cursor is different, if the object under the cursor is 180cm or 150cm high. This box is for changing the height used for the next few fields
+
+X, Y, Angle to Ground
+: These fields show the x and y position of the pixel under the cursor under the assumption it has a height equal to the probing height. The angle to ground is the angle between the line from the camera to the point and the ground. An angle of 90° means the point is perpendicularly under the camera.
+
+Color
+: This shows the color of the pixel under the cursor
+
+## Menu Bar
+
+The menu bar includes a plethora of menus. Feel free to explore the different options.
+
+
+```{toctree}
+:hidden:
+visualization
+```
\ No newline at end of file

docs/source/user_interface/visualization.md

+# Visualization of Points and Trajectories
+
+Options for the visualization of points and trajectories are included in the tracking tab.
+
+**TODO: image of viz options, followed by explanation (or explanation without image? Could open PeTrack to the side instead as user...)**
+