diff --git a/docs/source/conf.py b/docs/source/conf.py index b9033047f42dd078d6d83b59279678b5bbfa715f..cf358eab60583f71f437a35943f45038c40dd505 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -44,7 +44,8 @@ myst_enable_extensions = [ "html_image", "attrs_inline", "attrs_block", - "dollarmath" + "dollarmath", + "deflist" ] myst_url_schemes = { diff --git a/docs/source/export/export.md b/docs/source/export/export.md new file mode 100644 index 0000000000000000000000000000000000000000..2d15d1cef90d8daa1d63cc80f1f7ee617ed16127 --- /dev/null +++ b/docs/source/export/export.md @@ -0,0 +1,11 @@ +# Input and Output + +PeTrack works with a variety of files in different formats. This section of the documentation shows what different kind of files PeTrack consumes and what kind of files PeTrack can export itself. + +## Contents + +```{toctree} +:maxdepth: 1 +trajectory +video +``` \ No newline at end of file diff --git a/docs/source/export/trajectory.md b/docs/source/export/trajectory.md new file mode 100644 index 0000000000000000000000000000000000000000..4dcfa822c48175b71837abc117809804c98c8c07 --- /dev/null +++ b/docs/source/export/trajectory.md @@ -0,0 +1,85 @@ +# Trajectory Import and Export + +PeTrack supports a few custom trajectory formats. These formats can be exported from PeTrack as well as re-imported into PeTrack. + +## TXT-File + +### File Format + +The `.txt` file is the current main export target for external consumption of the trajectories (e.g. for analysis). The basic format of a `.txt` file is the following: + +``` +# PeTrack project: multicolor.pet +# raw trajectory file: multicolor_truth_selbst.trc +# framerate: 25 fps +# z: can be 3d position or height of person (alternating or not) +# id frame x/m y/m z/m +1 0 13.7271 1.02799 1.58 +1 1 13.7288 1.03453 1.58 +1 2 13.7288 1.03453 1.58 +1 3 13.7293 1.03445 1.58 +1 4 13.7242 1.02797 1.58 +... +``` + +The file starts with a header of comments, all of which start with an `#`. These contain the used PeTrack project file, the `.trc` file containing the raw trajectories, the fps of the video or image sequence and a small explanation what variables are listed in what order. + +That is followed by all values, each data point being in one line and containing space separated values. + +The information you see in the example above is part of any exported `.txt` file. Depending on the used marker, you can enable even more data to be added to the `.txt` file. + +### Export Options + +PeTrack has a lot of export options. They either add output to the exported file or perform some additional calculation on export. + +insert miss. frames +: When this option is enabled, PeTrack uses an algorithm to detect frame drops. These dropped frames are then inserted into the export. The position on these dropped frames is linearly interpolated. + +:::{note} +The calculation of missing/dropped frames is expensive. If you know there are no frame drops in your video, let this option disabled. +::: + +recalc. height +: This option is only relevant for project using a **stereo** camera. This recalculates the height of the pedestrian using the median height of the measured 3D points. + +alt. height +: This options allows `z` to alternate. This is relevant when alternating height information is available (e.g. when using stereo) and the pedestrians are not at the same height for the whole recording. This could be the case e.g. at staircases. If disabled all `z` coordinates are just the height of the pedestrian. + +elim. tp. wo. hgt. +: This option discards points for which no height was measured. This option only takes effect if `alt. height` is also selected. Note that PeTrack prints a warning for each discarded point, so remember to save the output of PeTrack during the export into a log file. + +elim. trj. wo. hgt. +: This option discards all trajectories which include a point for which no height was measured. This option only takes effect if `alt. height` is also selected. Note that PeTrack prints a warning for each discarded trajectory, so remember to save the output of PeTrack during the export into a log file. + +smooth +: This option smooths the `z` component of trajectories. This has no real effect if `alt. height` is not enabled. + +:::{warning} +The `smooth` option changes the trajectories in-place! That is, when trajectories are exported multiple times with this option enabled, they are also smoothed multiple times! +::: + +add head direction +: Adds the head direction to the `.txt` file. Head directions can be extracted for different markers. From the currently recommended markers, the code marker as well as the combined multicolor and code marker support this export option. The head direction is saved as a vector with the entires: `viewDirX` and `viewDirY` + +add angle of view +: Adds the angle between the line from the camera to the point and the line from the camera perpendicularly to the ground in radians + +use meter +: Use meter in export instead of centimeter. (**Recommended**) + +add comment +: PeTrack allows to attach comments to pedestrians (`right mouse button > Edit comment`). Enabling this option exports these comments as well + +add marker ID +: Adds the ID of code markers to the export as well. Note that the ID of the person will be put at every point in the trajectory, not only at the points at which the marker was recognized + + +## TRC-File + +The `.trc` files are the internal trajectory file format from PeTrack. They are not meant for consumption by any other program but PeTrack and no guarantees on the stability of the format are made. + +The `.trc` files can be exported and imported. They are the only files that contain **all** information PeTrack has about a trajectory and therefore the only file which recreates the original trajectories perfectly in import. + +When a `.trc` file is exported, its project associated itself with it. If PeTrack find the file when the project is opened again, it is imported automatically. For finding the file PeTrack uses both, the absolute path to the file as well as the relative path of the `.trc` file to the `.pet` project file. + +Export options do not apply to `.trc` files. \ No newline at end of file diff --git a/docs/source/export/video.md b/docs/source/export/video.md new file mode 100644 index 0000000000000000000000000000000000000000..d0b0989d2bd034e8c7f88ad04fc8e72286912f6e --- /dev/null +++ b/docs/source/export/video.md @@ -0,0 +1,35 @@ +# Video View Import and Export + +## Import + +PeTrack internally uses [ffmpeg](https://ffmpeg.org/) for reading video files. This should support virtually any video file format of practical relevance. PeTrack also supports reading in image series. In this case, the images need to be named with the same stem, followed by an increasing number. In the simplest case, the images are named by numbers only, e.g. + +``` +000100000.jpg +000100001.jpg +000100002.jpg +000100003.jpg +000100004.jpg +``` + +## Export + +PeTrack allows you to export the video. All of the options related to video export are in the `File` menu from the menu bar. + +Export Video +: Export as video as is, i.e. without undistortion or overlays. Can be used to get video out of image sequence. + +Export Video View +: Export as video as is shown in PeTrack. That is undistortion is applied and all visualizations (e.g. trajectories) are part of the exported video + +Export Image +: Export currently shown frame with undistortion and border, but without any overlays + +Export Image Sequence +: Export as image sequence with undistortion and border, but without any overlays. Can be used to get an image sequence out of a video. + +Export View +: Export currently shown frame with undistortion and overlays + +Export View Sequence +: Export as image sequence with undistortion and overlays. \ No newline at end of file diff --git a/docs/source/index.md b/docs/source/index.md index 60ab1daebebbce4f2086d4e32cda1fea2bec0b48..2fde7975f4143604de6f0dd1c1ddabeef0255c32 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -9,15 +9,36 @@ Different kinds of markers (e.g. with height information, head direction, indivi An executable of `PeTrack` is available [here](https://go.fzj.de/petrack-download). +::::{grid} 2 +:::{grid-item-card} Getting Started +:class-card: intro-card +:link: getting_started +:link-type: doc +:link-alt: To getting started +A guide through a small demo project +::: +:::{grid-item-card} Planning of Experiments +:class-card: intro-card +:link: planning/planning +:link-type: doc +:link-alt: To planning experiments +A guide to all things to consider **before** using PeTrack or doing the experiments themselves. Recommended reading during planning to get good quality data. +::: +:::: + + +## Contents + :::{toctree} :maxdepth: 1 getting_started -user_interface planning/planning +user_interface/user_interface calibration/calibration recognition/recognition tracking/tracking correction/correction +export/export misc/misc contribute/contribute section/MeinTest diff --git a/docs/source/misc/FAQ.md b/docs/source/misc/FAQ.md index d06fb1d5a38c17a231f9208918046e80b50bc4b3..8a2954e37b39d9b4b84b407d5a5b390ef7569ac8 100644 --- a/docs/source/misc/FAQ.md +++ b/docs/source/misc/FAQ.md @@ -2,5 +2,11 @@ Frequently Asked Questions -**Where can I get help when I encounter a bug?** -Please [open an issue](https://jugit.fz-juelich.de/ped-dyn-emp/petrack/-/issues/new) in our GitLab repository. Mention the expected behaviour, the bug and don't forget to mention the version of PeTrack you used as well as your operaating system. \ No newline at end of file +### Where can I get help when I encounter a bug? +Please [open an issue](https://jugit.fz-juelich.de/ped-dyn-emp/petrack/-/issues/new) in our GitLab repository. Mention the expected behavior, the bug and don't forget to mention the version of PeTrack you used as well as your operating system. + +<!-- +### How can I contact the authors? + +We can be contacted via the email-address `petrack (at) fz-juelich.de`. Feel free to contact us for advice on experiment setups and similar. However, if your enquiry is about a bug, please [open an issue](https://jugit.fz-juelich.de/ped-dyn-emp/petrack/-/issues/new) instead. +--> \ No newline at end of file diff --git a/docs/source/misc/combine.md b/docs/source/misc/combine.md deleted file mode 100644 index 6fba36acd00c7184289cc01974ea2636f6181377..0000000000000000000000000000000000000000 --- a/docs/source/misc/combine.md +++ /dev/null @@ -1,3 +0,0 @@ -# Combine Trajectories From Different Videos - -Sth. about combining trajectories from different cameras. Combine-Script currently is not public, how do we want to deal with that? \ No newline at end of file diff --git a/docs/source/misc/misc.md b/docs/source/misc/misc.md index a976adcee1e89e3d89d311bf492e1d6eb232460d..f066b358dd3aec9d8ddbf697a10d42b56ac16f81 100644 --- a/docs/source/misc/misc.md +++ b/docs/source/misc/misc.md @@ -9,5 +9,4 @@ Title page of misc topics, probably just a table of contents FAQ citation batch_processing -combine ::: \ No newline at end of file diff --git a/docs/source/recognition/aruco.md b/docs/source/recognition/aruco.md index e25310fd0ce729dd2d8e4818a8ac1d640406c951..d7f74b3f7f5a754c9a323d9cc5d49d618cc1d851 100644 --- a/docs/source/recognition/aruco.md +++ b/docs/source/recognition/aruco.md @@ -1,3 +1,37 @@ -# Aruco Marker +# Code Marker -Here are the docs on how to use the aruco marker and what its options are \ No newline at end of file +The code marker is an aruco marker, a special fiducial tag. This means that not only is the marker designed to be easy and robust to detect, it also contains a code number. Therefore the code marker enables an association of trajectories with other data, e.g. questionnaires. However, the code marker needs a comparatively high resolution (**TODO: reference to Maiks part**). + +## Marker Options + +The options for the code marker are entirely contained within the parameter dialog of the marker. + +:::{figure-md} aruco-options + + +The option dialog for the code marker. +::: + +The code marker has a lot of options and it is beyond the scope of this documentation to explain all of them. We refer to the [documentation of OpenCV](https://docs.opencv.org/4.5.3/d1/dcd/structcv_1_1aruco_1_1DetectorParameters.html#details) for a more detailed explanation of the parameters (also accessible via the `more infos...` button in the parameter dialog). + +:::{tip} +In our experience the `adaptive thresholding window` parameters are very influential +::: + +The only option that is unique to PeTrack is `show detected candidates`. When this checkbox is enabled, all candidates are drawn onto the screen. + +:::{figure-md} aruco-candidates + + + +Visualization of code marker candidates. Successful detections are green, mark the upper left corner and show the id of the marker. Candidates which did not result in a detection, because of an unreadable code, are shown in red. Accordingly, without id. +::: + +## Extra export options + +The usage of the code marker enables two new export options to be enabled: + +- `add marker id` + Adds the id of the detected code marker to the txt export +- `add head direction` + The code marker has a fixed orientation. Therefore, its orientation can be detected and exported. If care is given that the orientation of the marker aligns with the orientation of the pedestrian head, this is a measurement of the head view direction. \ No newline at end of file diff --git a/docs/source/recognition/images/aruco_viz.png b/docs/source/recognition/images/aruco_viz.png new file mode 100644 index 0000000000000000000000000000000000000000..152ce3c6aa76b1821577081d9caa1788b950b9a8 Binary files /dev/null and b/docs/source/recognition/images/aruco_viz.png differ diff --git a/docs/source/recognition/images/aurco_options.png b/docs/source/recognition/images/aurco_options.png new file mode 100644 index 0000000000000000000000000000000000000000..9062857415c4e99156bfb9061bfb638fe76d4817 Binary files /dev/null and b/docs/source/recognition/images/aurco_options.png differ diff --git a/docs/source/recognition/images/color_picker_example.webp b/docs/source/recognition/images/color_picker_example.webp new file mode 100644 index 0000000000000000000000000000000000000000..0ee2c29d0ba866cd43618c35afe12f661c3a37b9 Binary files /dev/null and b/docs/source/recognition/images/color_picker_example.webp differ diff --git a/docs/source/recognition/images/color_plot.png b/docs/source/recognition/images/color_plot.png new file mode 100644 index 0000000000000000000000000000000000000000..7ca536d3d2349952db3d2650713a65e5a3cdc21e Binary files /dev/null and b/docs/source/recognition/images/color_plot.png differ diff --git a/docs/source/recognition/images/color_range_dialog.png b/docs/source/recognition/images/color_range_dialog.png new file mode 100644 index 0000000000000000000000000000000000000000..f50ead3b1b800a517a2bea3a08179a0eb9e50d16 Binary files /dev/null and b/docs/source/recognition/images/color_range_dialog.png differ diff --git a/docs/source/recognition/images/color_range_example.webp b/docs/source/recognition/images/color_range_example.webp new file mode 100644 index 0000000000000000000000000000000000000000..2799c27bed060f2da4246f540a0a7670d6c31e31 Binary files /dev/null and b/docs/source/recognition/images/color_range_example.webp differ diff --git a/docs/source/recognition/images/color_selection.png b/docs/source/recognition/images/color_selection.png new file mode 100644 index 0000000000000000000000000000000000000000..39ed44f06daaf8a4c06d7f37f2f4ac1a8ebb6123 Binary files /dev/null and b/docs/source/recognition/images/color_selection.png differ diff --git a/docs/source/recognition/images/general_settings.png b/docs/source/recognition/images/general_settings.png new file mode 100644 index 0000000000000000000000000000000000000000..e3360a129103cbdaa718d5d7e7c2066b940bd367 Binary files /dev/null and b/docs/source/recognition/images/general_settings.png differ diff --git a/docs/source/recognition/images/height_options.png b/docs/source/recognition/images/height_options.png new file mode 100644 index 0000000000000000000000000000000000000000..8cd403a81cd98206ce23b848753e6bd6bc6cee66 Binary files /dev/null and b/docs/source/recognition/images/height_options.png differ diff --git a/docs/source/recognition/images/maps2.png b/docs/source/recognition/images/maps2.png new file mode 100644 index 0000000000000000000000000000000000000000..5ecb7b977de55e08bb3044cb3ee8fd1d400c008f Binary files /dev/null and b/docs/source/recognition/images/maps2.png differ diff --git a/docs/source/recognition/images/multicolor_param_win.png b/docs/source/recognition/images/multicolor_param_win.png new file mode 100644 index 0000000000000000000000000000000000000000..8be611f975ed49c3395de6a4655aee893df78a6d Binary files /dev/null and b/docs/source/recognition/images/multicolor_param_win.png differ diff --git a/docs/source/recognition/multicolor.md b/docs/source/recognition/multicolor.md index de2478f3319454af687a8b452b4f91f42735af49..608b27e74d5ef448d85b4091f2f68f5c886f6573 100644 --- a/docs/source/recognition/multicolor.md +++ b/docs/source/recognition/multicolor.md @@ -1,3 +1,151 @@ # Multicolor Marker -Here are the docs on how to use the multicolor marker and what its options are \ No newline at end of file +The multicolor marker is a colored hat that pedestrians wear. For a quick rundown on when to use which type of marker, see **TODO ref to Maiks section**. But in a nutshell, the multicolor marker is easy to prepare and works even with low resolution cameras. It can be combined with an code-marker as well, if some cameras are high-resolution and some are not. Then, low-resolution cams fall back to the multicolor marker in case the code cannot be read. + +There are two places for settings parameters of the multicolor marker. The parameter window as well as the recognition tab. + +## Multicolor Parameter Window + +:::{figure-md} multicolor-param-win + + +Parameter window of the multicolor marker +::: + +In the parameter window, we have a plethora of options: + +The first options concern themselves with combinations of the multicolor marker with other markers, namely the hermes marker and code marker. When the multicolor marker is combined with another marker, first the multicolor detection is run. Afterwards, on the area around the detected head, the other marker is searched. The idea is to combine the robust, but slightly inaccurate multicolor marker with a more accurate and "structured" marker. + +use black dot +: The multicolor marker can be used with a black dot added, essentially creating a combination between the hermes marker and the multicolor marker. **TODO: add picture** This gives a bit more structure for tracking and a smaller specific point in the image to locate. If one does use this, this checkbox should be enabled and the size set to the size of the point in cm. Else, this option can be turned off. + +use code marker +: The multicolor marker can be combined with the [code marker](aruco), which is called code marker here. A click onto the `parameter`-button opens the parameter window for the code marker. There the options for the detection of the code marker can be changed. Please refer to the [documentation of the code marker](aruco) for further information on this. + +The following three options are all for combinations with either the hermes or the code marker + +ignore head without black dot or code marker +: If this is enabled, all detections where only the colored hat, without the extra feature, was detected will be discarded. If this is not checked, detections on color only are used as well. If e.g. the code cannot always be recognized due to the camera resolution, this should be off. However, in case the extra feature should be recognizable, this could reduce false positives or at least mediocre detections. + +use color marker for big error +: If both, the color point and the feature point (black dot, code marker) are detected and subsequent tracking of the feature point has a high error, enabling this options lets PeTrack redo the tracking, using the color point instead of the feature point. + +restrict position of black dot +: When this option is enabled, the black dot is searched in a smaller area, determined by the expected position of the dot given the angle at which the camera looks at the pedestrian. + +From here on, the other options refer to the color marker itself. + +auto correct perspective view +: The multicolor marker always takes the point in the middle of the colored blob in the image. However, since the camera looks at the pedestrian from a certain angle, this is not the same as the middle of the head. Enabling this option activates a correction for this error. + +only for export +: Activating this option means, the correction for perspective view is during the export and not during tracking and recognition. + +:::{note} +If you enable `auto correct perspective view`, it is recommended to also enable `only for export`. Otherwise the correction of the perspective view during tracking can lead to the point going to the edge of the visible head, worsening tracking performance. +::: + +show mask +: When enabled, we show a mask visualizing the color selection **TODO: true, what about the other mask?** + +close radius +: Closing is a morphological operation to remove small holes. A larger value means, larger holes will be filled. The filter is only applied if `use` is checked. + +open radius +: Opening is a morphological operation to remove small dots or specks. The larger the value, the larger the specks which are removed. The filter is only applied if `use` is checked. + +The head detection detects blobs of a certain color. It then filters out some of these blobs on a variety of criteria. Here are some options on those: + +area +: This is the minimum and maximum area in pixels a head is allowed to have. Each blob that has fewer pixels than the minimum or more pixels than the maximum does not count as a detection. When `head size` is checked, PeTrack tries to determine sensible values from the camera position and average head size of pedestrians. + +max ratio +: This is the maximum ration between the long and short semi-axis of the ellipse fitting to the blob. If the ratio is larger, the detection is discarded. This filters out very long very thin blobs. + +Finally, another option on color mask selection + +opacity +: This sets the opacity of the mask which shows the current color selection. If `mask` is checked, the selected parts of the mask are translucent. Otherwise, they are white. + +## Color Selection + +:::{figure-md} color-selection + + +Part of recognition tab for color and height selection +::: + +The second part of the settings for the multicolor marker are part of the recognition tab. There we can see the color selection. PeTrack uses the color to encode the height of pedestrians. By giving pedestrians of similar height the same hat color and encoding the average height per group in this color, the accuracy of the coordinates can be enhanced. Especially if the height of the pedestrians has large variations (e.g. kids and adults, or adults and wheel-chair users). + +The mappings from color to height are called `map` in PeTrack. This mapping is used for multiple markers, including e.g. the casern marker. Therefore, some of the options here are not relevant for us. + +:::{figure-md} color-plot + + +Color Plot: The rectangles are color regions corresponding to a certain color map. The circles are colors corresponding to a detected point. The `X` shows where the color of the pixel under the cursor lies in the plot. +::: + +The most eye-catching element of this part of the UI is the [color plot](#color-plot). This plots visualizes the existing maps as well as the colors of existing detections and of the pixel under the cursor. The visualization of the colors of the detections can be disabled with the checkbox at the top of the `size and color` area titled `show recognition color`. + +One quickly realizes that all detections lie directly in the middle of their respective map. This is by design. The multicolor marker searches e.g. for an orange hat. When it finds such a hat, it just assigns it the color orange. Therefore, all detections from the same map have the same color. This is not the case if you use an older marker, like the casern marker. There, the color needed to be sampled from the image. The `optimize` button includes some functionality to improve this sampling by removing outliers. Accordingly, it has no functionality for multicolor markers. + +The other options above the color plot are for changing its visualization. However, the multicolor marker has other visualization means, which are better suited. So no more detail for these options are given. + +:::{figure-md} maps-section + + +The maps section of the recognition tab +::: + +Below the color plot, we see the maps. We see the currently selected map (1 in the [figure](#maps-section)) and the corresponding height (2 in the [figure](#maps-section)). We can change both, what map is selected and what height we want to assign to this map. If we want to add or delete maps, we need to use the according button (3 and 4 in the [figure](#maps-section)). + +The different sliders are for an old way of color selection. They are not suited for multicolor markers and can safely be ignored. Similarly, the `color` checkbox can be ignored. Instead, we use **either** the `color range` or the `color picker` button to select a color for the multicolor marker (5 and 6 in the [figure](#maps-section)). + +:::{note} +`color picker` and `color range` do not play well together. That is, a color obtained with one cannot be fine tuned with the other. +::: + +Some general remarks hold for either method: + +:::{tip} +Deactivate `open` and `close` while selecting a color. First try to select the color best you can and then activate the options to improve upon this selection. Using them while selecting the color means you are optimizing 3 parameters at once. +::: + +:::{tip} +Activate the `mask` from the multicolor parameters and set the opacity to ca. 50%. This enables you to see how well your color selection works (see screenshots later) +::: + +:::{note} +You need to activate `perform` for the color mask to update! +::: + +### Color Picker + +When pressing the `color picker` button, the color picker modus is activated. In this modus, one can `shift + left mouse click` on a pixel to start a color selection and then expand this selection with a normal `left click`. + +:::{figure-md} + + +Example usage of color picker. With the mask enabled and `perform` checked, we can interactively select the color, looking at different parts of the image to find still uncovered parts. Note that colors which are "close" to each other may not be easy to differentiate with the color picker (here green and yellow). Then the `color range` dialog might be a better option +::: + +### Color Range + +When you click on `color range`, a color selection dialog appears: + +:::{figure-md} color-range-dialog + + +Color range dialog: The dialogue allows selecting the lower bound and upper bound color via an hsv color picker. +::: + +The color detection works in the HSV (Hue, Saturation, Value) color space. Hue is the "color", i.e. redness or greenness of the color. Saturation how saturated it is. And value, how bright it is. The dialog allows you to select to colors from this space. All colors that lie "in between" your selected colors are then part of your selection. It is noteworthy that the hue part is a value in degrees from 0° to 360°, with red being around zero degree. To allow the selection of red-ish colors as well, there is an option `inverse hue`, which changes the direction of this difference. That is we select colors with hue $h$ such that $h_{min} < h < h_{max}$ if `inverse hue` is not selected and colors with $h < h_{min} \lor h > h_{max}$ if `inverse hue` is selected. + +:::{figure-md} color-range-example + + +Example usage of color range dialog. With the mask enabled and perform +activated, one can interactively see which colors in the frame are selected and +which are not. +::: + diff --git a/docs/source/recognition/others.md b/docs/source/recognition/others.md deleted file mode 100644 index 23359e5ff4075198c86ecbee84c2d41eb881b8f5..0000000000000000000000000000000000000000 --- a/docs/source/recognition/others.md +++ /dev/null @@ -1,12 +0,0 @@ -# Other Markers - -TODO: Short list of other markers, with capabilities (table from Maik); what additional info they can give, thats it; no usage guide, since we don't actively use any of these - -| marker | robustness | precision | high density | additional info | needed resolution | effort in creation | -| ---------------- | ---------- | --------- | ------------ | --------------- | ----------------- | ------------------ | -| casern | ++ | ++ | o | + | o | o | -| hermes | o | ++ | + | - | + | + | -| japan | + | ++ | - | o | + | o | -| color/multicolor | + | o | - | o | ++ | ++ | -| aruco | TODO | | | | | | - diff --git a/docs/source/recognition/recognition.md b/docs/source/recognition/recognition.md index fbff520388167e5ea054d1aa20c55d48a01a1484..ac8714567c7e83c839bf0e64ca7dd27ab6ea9e5b 100644 --- a/docs/source/recognition/recognition.md +++ b/docs/source/recognition/recognition.md @@ -1,12 +1,14 @@ # Recognition -Hier wird einmal grob erklaert, was recognition eigentlich so ist. Die genaue Doku passiert pro Marker im folgenden: +It is possible to use PeTrack to do semi-automatic tracking. That means, one does detect the pedestrians manually and then tracks these automatically. However, it is faster, potentially more accurate and more convenient to let PeTrack handle the detection of pedestrians automatically. For this purpose, the pedestrians usually get a _marker_. That is some kind of special hat, which then gets detected from PeTrack. + +While there are a plethora of options available, we suggest using the multicolor or code markers. Please see their respective sites for more information. ## Contents :::{toctree} :maxdepth: 1 +setting-up multicolor aruco -others ::: \ No newline at end of file diff --git a/docs/source/recognition/setting-up.md b/docs/source/recognition/setting-up.md new file mode 100644 index 0000000000000000000000000000000000000000..07923063dec09d0cae844b504beb2552e8f837b8 --- /dev/null +++ b/docs/source/recognition/setting-up.md @@ -0,0 +1,99 @@ +# Set Up Recognition + + +## General Settings + +:::{figure-md} general-settings-reco + + +The general settings at the top of the recognition tab +::: + +At the top of the recognition tab, you can find the [general settings](#general-settings-reco) of recognition: + +perform +: When checked, this enables the automatic recognition of pedestrians + +step +: This number decides on which frames to run the detection, i.e. if step is set to 2, only every second frame a detection of pedestrians is done. The gaps are filled with tracking. + +marker +: Where the image shows "multicolor marker", we can select the specific marker we want to use + +parameter +: The button parameter opens a pop-up window where we can set the parameters of the selected marker + +The `step`-option exists, because recognition is more computationally expensive than tracking. On modern computers this should not be a problem. Since recognition is more accurate than tracking, a step of 1 is recommended, i.e. running recognition on every frame. + +## Region Of Interest + +The region of interest (ROI) is the part of the image where a detection is +performed. For the ROI we see the options + +show +: Shows the ROI as a blue rectangle on the image + +fix +: "Fixes" the rectangle, such that it cannot be moved with the mouse + +full image +: Sets the ROI to the full image + +adjust automatically +: Sets the ROI as one head size smaller than the tracking ROI + +The ROI can be adjusted in the image, by resizing and moving with the mouse. **TODO: Image or GIF of manipulation of ROI** + +:::{warning} +The tracking ROI must be larger than the recognition ROI. Any detection outside the tracking ROI will create an entirely new, one point trajectory. We recommend setting the tracking ROI first and then using `adjust automatically` for the recognition ROI +::: + +## Height-based Options + +At the bottom of the recognition tab there are a few options regarding the height of pedestrians. + +:::{figure-md} height-options + + +Height options at the bottom of the recognition tab +::: + +def. height +: This is the default height. It is used when a pedestrian does not get a height by other means (i.e. from their marker) and for some calculations where the height of a pedestrian is not known already. Ideally, this is set to the average height of pedestrians in your experiment. + +dist. +: prints the height distribution to the console (currently **only works with multicolor marker**) + + ``` + [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 146.00 - number 1 (1.14%) + [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 158.00 - number 9 (10.23%) + [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 168.00 - number 31 (35.23%) + [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 179.00 - number 39 (44.32%) + [colorPlot.cpp:683:ColorPlot::printDistribution][info] height: 192.00 - number 8 (9.09%) + ``` + +res. height +: Resets the height of all pedestrians that was obtained through either a stereo camera or marker ids + +res. pos. +: Resets the position of all points calculated via a stereo camera + +read heights from file +: This button enables you to load a file with the following format. This file gives assigns a height to each marker id (that is the id from the code marker). That individual height is more accurate than an average height assigned via the color of a hat. + + ``` + # id z/cm + 987 184 + 988 179 + 993 175.5 + ``` + +import markerIDs +: This button enables you to load a file to assign a marker id (or code id) to a pedestrian with a given petrack-internal id. That can be used if a select few people in the video have a code marker which is never recognized, but they can be identified by the experimenter visually. + + ``` + # id markerID + 1 995 + 2 999 + 3 998 + ``` \ No newline at end of file diff --git a/docs/source/section/MeinTest.md b/docs/source/section/MeinTest.md index 6ea960e5c1bbe909de43c84b759f3ec0c634235f..7afb2630e2d59c033b3da6886a81658b64b341af 100644 --- a/docs/source/section/MeinTest.md +++ b/docs/source/section/MeinTest.md @@ -14,10 +14,9 @@ text With some text asdfjnasöfhaöshf asjföds föijsa följsad kfjsdljflkdsajfjlks jfksa jlkds jösadj föjdsaf ölksj flkajs flksdj fölsaj fölksadj fölkjdsa fölkjdsa fölkdsa fölkadsj fölksaj -::::{figure-md} +::::{figure-md} abb :::{image} ../_static/logo.png -:name: abb :width: 50% ::: @@ -65,6 +64,8 @@ Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie co Test fuer ein paar Referenzen, z.B. auf eine [Abbildung](#abb) oder eine [Notiz](#notiz). Dafuer muss `:name: NAME` mit angegeben werden. Fuer eine Formel sollte man `:label:` verwenden und kriegt eine Referenz der Form [](#formel), [](eqn:test). Fuer Ueberschriften (und Absaetze) kann man mit `(heading-target)=` einen Namen definieren. Dabei kommt diese Markierung **vor** dem Target. Die Referenz sieht so aus: [Ueberschrift](#subheading-target), [Absatz](#absatz-target). Links zu anderen Dokumenten (z.B. [](/getting_started.md), [](/getting_started.md#calibration)) gehen natuerlich auch. +Einen DOI-Link kann man Einbinden mit `<doi:10.1016/j.neucom.2012.01.036>`, das ergibt: <doi:10.1016/j.neucom.2012.01.036> + ::::{grid} 2 :::{grid-item-card} Title 1 :class-card: intro-card diff --git a/docs/source/tracking/export.md b/docs/source/tracking/export.md deleted file mode 100644 index 44f3ca666e71e890992288ede86d4e57d1912815..0000000000000000000000000000000000000000 --- a/docs/source/tracking/export.md +++ /dev/null @@ -1,3 +0,0 @@ -# Export - -- Explaining different export options and what marker and/or camera (stereo) is needed to use them \ No newline at end of file diff --git a/docs/source/tracking/tracking.md b/docs/source/tracking/tracking.md index c6df0f56dfef57751ca7c8f6cedee78b9d285561..535f68e7fa51694de6772e65b7b4c2b6969ca701 100644 --- a/docs/source/tracking/tracking.md +++ b/docs/source/tracking/tracking.md @@ -8,5 +8,4 @@ Here is the tracking docs. :maxdepth: 1 setting-up parameter-tuning -export ::: \ No newline at end of file diff --git a/docs/source/user_interface.md b/docs/source/user_interface.md deleted file mode 100644 index 86708777a7c1a521e9fb63118d90b13b0bdd2569..0000000000000000000000000000000000000000 --- a/docs/source/user_interface.md +++ /dev/null @@ -1,3 +0,0 @@ -# User Interface - -Screenshot von PeTrack und einfache Erklaerung, was in welchem Bildbereich zu sehen ist (Video, Seitenleiste mit versch. Tabs, ...) \ No newline at end of file diff --git a/docs/source/user_interface/images/petrack_ui_with_video.png b/docs/source/user_interface/images/petrack_ui_with_video.png new file mode 100644 index 0000000000000000000000000000000000000000..e0124cd518370bc8f14befc3defcb9c579056c8b Binary files /dev/null and b/docs/source/user_interface/images/petrack_ui_with_video.png differ diff --git a/docs/source/user_interface/images/status_bar2.png b/docs/source/user_interface/images/status_bar2.png new file mode 100644 index 0000000000000000000000000000000000000000..af269497e26d14eca027f57dfa902e8bc427b453 Binary files /dev/null and b/docs/source/user_interface/images/status_bar2.png differ diff --git a/docs/source/user_interface/images/user_interface_empty.png b/docs/source/user_interface/images/user_interface_empty.png new file mode 100644 index 0000000000000000000000000000000000000000..f8656734045f8f048492e8ad01131c7ff5ac8b7b Binary files /dev/null and b/docs/source/user_interface/images/user_interface_empty.png differ diff --git a/docs/source/user_interface/user_interface.md b/docs/source/user_interface/user_interface.md new file mode 100644 index 0000000000000000000000000000000000000000..6f4abd18caa3a3a723cd49a4cf2ecc93dfc0a985 --- /dev/null +++ b/docs/source/user_interface/user_interface.md @@ -0,0 +1,104 @@ +# User Interface + +:::{figure-md} ui-annotated + + +PeTrack after opening without project +::: + +:::{figure-md} ui-with-vid + + +PeTrack after opening a video file +::: + +PeTrack's UI is made out of a few parts. + +## Video View and Video Side Bar + +The video view is where the video and possibly visualizations on top of the video are displayed. One can zoom in via the scroll wheel or the lower half of the video side bar. When using the scroll wheel, one scrolls towards the pixel under the mouse cursor. If only parts of the image remain visible due to zooming in, one can navigate in the image by pressing down the left mouse button inside the image and dragging it. One can fit the entire image in view with `Ctrl + 0` or under `View > Fit in window`. Alternatively, one can fit the recognition region of interest to the view with `Ctrl + 1` or `View > Fit in region of interest`. + +Furthermore, one can rotate the image using the upper half of the video side bar. + +## Player Control + +The player control has the following buttons from left to right + +play backwards +: This button starts a backward playback of the video. + +:::{note} +Backward playback is *a lot* slower than forward playback, because video codecs are optimized for forward playback. +::: + +single frame backward +: This button jumps one frame backwards + +pause +: Pauses the video + +single frame forward +: This button jumps one single frame forward + +play +: Plays the video (forward direction) + +record +: This button starts a recording. + +:::{note} +We recommend using the `Export Video` or `Export Video View` options from the `File` menu in the menu bar instead of the recording button. +::: + +Next to these buttons are the boxes labeled `In` and `Out`. If the video is longer than the experiment, one can use these to specify a different starting frame (`In`) or ending frame (`Out`) for PeTrack. PeTrack then treats the video as if all frames before `In` and after `Out` do not exist. The default values are 0 for `In` and the last frame of the video file for `Out`. + +Then there is a slider which can be used to scrub through the video. + +Last on the line are two boxes `_ @ _ fps`. The first of these boxes shows the current frame and the last shows the framerate at which PeTrack tries to play the video. Both values can be edited by the user, either to jump to a certain frame by changing the current frame or to change th playback speed by changing the fps. + +:::{note} +For changing the fps relative to the original speed, there are options under `View > Playback speed`. The desired playback speed only changes the behavior of PeTrack if `Fix playback speed` or `Limit playback speed` are selected (both under the `View` menu). +`Fix playback speed` plays at the desired speed, skipping frames if necessary. `Limit playback speed` only ever slows down a video, never skips frames to speed it up. +::: + + +## Tab View + +The tab view is where most settings can be set. Its content depends on the tab selected and is described in the documentation using these. The widths of the tab view can be changed by dragging the border of the tab view with the mouse. With the green arrow in the video side bar, the tab view can be collapsed entirely as well. + +## Status Bar + +The status bar is mostly empty on an empty project, but once a video is loaded, one can see different data in it, see the following figure: + +:::{figure-md} + + +The right side of the status bar shows a lot of information. All information +shown right from the current framerate is dependent on the current position of +the mouse cursor in the image. +::: + +Timecode +: This part of the status bar displays the timecode corresponding to the current frame. Since the timecode is taken from the video file, this does not necessarily correspond to either the real world time nor does it need to start at zero. In stereo recordings, this can also show the time given by a time file. + +Current Framerate +: This shows the framerate at which the player is currently playing. Note that there is sime *intertia* to how fast this display updates. The color indicated whether we play slower than realtime (red) or at realtime or faster (green). (**TODO: we can also click on it to fix speeds... do we want to document that? There now is a more intuitive entry in the menu bar for it**) + +Probing Height +: The following data fields contain some information about the position of the pixel under the mouse cursor. Due to the perspective projection onto the camera, this information can only be given when assuming a certain height, i.e. the position of the pixel under the cursor is different, if the object under the cursor is 180cm or 150cm high. This box is for changing the height used for the next few fields + +X, Y, Angle to Ground +: These fields show the x and y position of the pixel under the cursor under the assumption it has a height equal to the probing height. The angle to ground is the angle between the line from the camera to the point and the ground. An angle of 90° means the point is perpendicularly under the camera. + +Color +: This shows the color of the pixel under the cursor + +## Menu Bar + +The menu bar includes a plethora of menus. Feel free to explore the different options. + + +```{toctree} +:hidden: +visualization +``` \ No newline at end of file diff --git a/docs/source/user_interface/visualization.md b/docs/source/user_interface/visualization.md new file mode 100644 index 0000000000000000000000000000000000000000..6b15e617d79fde722fc98fd5311fb58c6f86474e --- /dev/null +++ b/docs/source/user_interface/visualization.md @@ -0,0 +1,6 @@ +# Visualization of Points and Trajectories + +Options for the visualization of points and trajectories are included in the tracking tab. + +**TODO: image of viz options, followed by explanation (or explanation without image? Could open PeTrack to the side instead as user...)** +