OpenHKL issueshttps://jugit.fz-juelich.de/mlz/openhkl/-/issues2022-09-26T11:14:47+02:00https://jugit.fz-juelich.de/mlz/openhkl/-/issues/543Core: Pixel statistics do not respect masks2022-09-26T11:14:47+02:00Raza, ZamaanCore: Pixel statistics do not respect masksMasked pixels should be excluded from histograms.Masked pixels should be excluded from histograms.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/539core: Peaks generated by Predictor are stored on the heap, possible memory leak2022-09-14T15:36:30+02:00Raza, Zamaancore: Peaks generated by Predictor are stored on the heap, possible memory leakFrom `Predictor.cpp`
```
std::vector<Peak3D*> peaks;
for (const auto& [hkl, event] : events) {
Peak3D* peak(new Peak3D(data, hkl)); // here
Eigen::Vector3d center = {event.px, event.py, event.frame};
// s...From `Predictor.cpp`
```
std::vector<Peak3D*> peaks;
for (const auto& [hkl, event] : events) {
Peak3D* peak(new Peak3D(data, hkl)); // here
Eigen::Vector3d center = {event.px, event.py, event.frame};
// setShape may disable the peak if the centre is out of bounds
peak->setShape(Ellipsoid(center, 1.0));
if (peak->selected()) {
peak->setUnitCell(unit_cell);
peaks.push_back(peak);
}
}
```
Prefer smart pointers instead, see `PeakFinder` for implementation.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/536GUI: DetectorScene intensity scale should be linked to image bit depth2022-09-09T09:09:00+02:00Raza, ZamaanGUI: DetectorScene intensity scale should be linked to image bit depth`SubframeExperiment.cpp`:
```
_npoints_intensity->setMaximumWidth(100);
_npoints_intensity->setMaximum(65535);
_npoints_intensity->setMinimum(100);
_npoints_intensity->setValue(100);
```
The maximum intensity value is ...`SubframeExperiment.cpp`:
```
_npoints_intensity->setMaximumWidth(100);
_npoints_intensity->setMaximum(65535);
_npoints_intensity->setMinimum(100);
_npoints_intensity->setValue(100);
```
The maximum intensity value is dependent on the bit depth of the detector image, and the maximum should reflect that. 65535 is only valid in the case of 16 bit images.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/531GUI: Multiple DataSets: reset controls when switching data set/peak collections2022-08-10T13:18:29+02:00Raza, ZamaanGUI: Multiple DataSets: reset controls when switching data set/peak collectionsThe "Remove overlaps" and "Remove masked peaks" update immediately when clicked, but do not reset when the `DataSet` or `PeakCollection` is changed.The "Remove overlaps" and "Remove masked peaks" update immediately when clicked, but do not reset when the `DataSet` or `PeakCollection` is changed.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/529"File small_cell_low_intensity.tar.gz corrupted, downloading..."2022-08-10T10:57:20+02:00Wuttke, Joachimj.wuttke@fz-juelich.de"File small_cell_low_intensity.tar.gz corrupted, downloading..."When repeating `cmake .. -DOHKL_FULL_WORKFLOW_TEST=ON`, each time I get the message
```
...
-- Checking 16 test data files in directory OHKL_TESTDATA_DIR=test/data:
-- Full workflow tests enabled, checking for data files...
-- - File sma...When repeating `cmake .. -DOHKL_FULL_WORKFLOW_TEST=ON`, each time I get the message
```
...
-- Checking 16 test data files in directory OHKL_TESTDATA_DIR=test/data:
-- Full workflow tests enabled, checking for data files...
-- - File small_cell_low_intensity.tar.gz corrupted, downloading...
-- Installer name: OpenHKL-0.1.0-linux
...
```https://jugit.fz-juelich.de/mlz/openhkl/-/issues/528GUI: DetectorScene should be cleared when changing DataSet2022-08-10T11:36:49+02:00Raza, ZamaanGUI: DetectorScene should be cleared when changing DataSetCurrently, the old peak collection remains, and having a peak collection from a different data set makes not sense.Currently, the old peak collection remains, and having a peak collection from a different data set makes not sense.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/527Core: batch unit cells should have different names based on their data set2022-08-09T09:10:38+02:00Raza, ZamaanCore: batch unit cells should have different names based on their data setFor a data reduction with multiple data sets, it should be possible to distinguish the unit cells based on which data set they belong to.For a data reduction with multiple data sets, it should be possible to distinguish the unit cells based on which data set they belong to.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/525GUI: multiple data sets: refiner fails to update peaks for second data set2022-08-08T17:18:28+02:00Raza, ZamaanGUI: multiple data sets: refiner fails to update peaks for second data setThe first run can be refined, and the peaks updated as normal; however, after refining the second data set, the predicted peaks fail to update (or at least, a very small number seem to update).The first run can be refined, and the peaks updated as normal; however, after refining the second data set, the predicted peaks fail to update (or at least, a very small number seem to update).Raza, ZamaanRaza, Zamaanhttps://jugit.fz-juelich.de/mlz/openhkl/-/issues/523GUI: Reimplement legend in PlotPanel2022-08-08T12:55:11+02:00Raza, ZamaanGUI: Reimplement legend in PlotPanelIn !497, legends were rmoved from `PlotPanel` as a result of a bad implementation for ploting intensity profiles. Now the legend is missing from the plots in `SubframeRefine`.In !497, legends were rmoved from `PlotPanel` as a result of a bad implementation for ploting intensity profiles. Now the legend is missing from the plots in `SubframeRefine`.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/486Cannot make docs under Debian/testing2022-07-25T17:41:40+02:00Wuttke, Joachimj.wuttke@fz-juelich.deCannot make docs under Debian/testing`make docs` fails with
```
Generate XML output for dir /G/sw/ohkl/gui/widgets/
Running plantuml with JAVA...
Running dot...
lookup cache used 11594/65536 hits=155169 misses=12202
finished...
Exception occurred:
File "/usr/local/lib/py...`make docs` fails with
```
Generate XML output for dir /G/sw/ohkl/gui/widgets/
Running plantuml with JAVA...
Running dot...
lookup cache used 11594/65536 hits=155169 misses=12202
finished...
Exception occurred:
File "/usr/local/lib/python3.9/dist-packages/sphinx/theming.py", line 199, in load_external_theme
theme_entry_points = entry_points(group='sphinx.html_themes')
TypeError: entry_points() got an unexpected keyword argument 'group'
The full traceback has been saved in /tmp/sphinx-err-qyhl6vio.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[3]: *** [doc/CMakeFiles/docs.dir/build.make:72: doc/CMakeFiles/docs] Error 2
make[2]: *** [CMakeFiles/Makefile2:3924: doc/CMakeFiles/docs.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:3931: doc/CMakeFiles/docs.dir/rule] Error 2
make: *** [Makefile:1824: docs] Error 2
```
File `/tmp/sphinx-err-qyhl6vio.log`:
```
# Sphinx version: 5.1.0
# Python version: 3.9.2 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.11.3
# Last messages:
# Loaded extensions:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/sphinx/cmd/build.py", line 272, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/usr/local/lib/python3.9/dist-packages/sphinx/application.py", line 261, in __init__
self._init_builder()
File "/usr/local/lib/python3.9/dist-packages/sphinx/application.py", line 333, in _init_builder
self.builder.init()
File "/usr/local/lib/python3.9/dist-packages/sphinx/builders/html/__init__.py", line 248, in init
self.init_templates()
File "/usr/local/lib/python3.9/dist-packages/sphinx/builders/html/__init__.py", line 299, in init_templates
self.theme = theme_factory.create(themename)
File "/usr/local/lib/python3.9/dist-packages/sphinx/theming.py", line 232, in create
self.load_extra_theme(name)
File "/usr/local/lib/python3.9/dist-packages/sphinx/theming.py", line 177, in load_extra_theme
self.load_external_theme(name)
File "/usr/local/lib/python3.9/dist-packages/sphinx/theming.py", line 199, in load_external_theme
theme_entry_points = entry_points(group='sphinx.html_themes')
TypeError: entry_points() got an unexpected keyword argument 'group'
```https://jugit.fz-juelich.de/mlz/openhkl/-/issues/470GUI: Peak filter allows filtering by "predicted"2022-07-15T10:00:51+02:00Raza, ZamaanGUI: Peak filter allows filtering by "predicted"In `SubframeFilterPeaks`, under `Peak type`, there is a checkbox for filtering predicted peaks. I do not think there will ever be an instance where there is a collection containing both found and predicted peaks, so this should be removed.In `SubframeFilterPeaks`, under `Peak type`, there is a checkbox for filtering predicted peaks. I do not think there will ever be an instance where there is a collection containing both found and predicted peaks, so this should be removed.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/458TestPeakFinder.py fails2022-07-26T11:24:36+02:00Wuttke, Joachimj.wuttke@fz-juelich.deTestPeakFinder.py failsNewly cloned git repository on my Debian/testing machine. Ran cmake, make, ctest. TestPeakFinder.py fails:
```
$ uname -a
Linux ho6 5.18.0-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 5.18.5-1 (2022-06-16) x86_64 GNU/Linux
$ ctest --rerun-faile...Newly cloned git repository on my Debian/testing machine. Ran cmake, make, ctest. TestPeakFinder.py fails:
```
$ uname -a
Linux ho6 5.18.0-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 5.18.5-1 (2022-06-16) x86_64 GNU/Linux
$ ctest --rerun-failed --output-on-failure
Test project /G/sw/ohkl/build
Start 51: TestPeakFinder.py
1/1 Test #51: TestPeakFinder.py ................***Failed 12.85 sec
F
======================================================================
FAIL: test (__main__.TestPeakFinder)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/G/sw/ohkl/test/python/TestPeakFinder.py", line 57, in test
self.assertTrue(n_peaks >= 465, f"found only {n_peaks} peaks")
AssertionError: False is not true : found only 426 peaks
----------------------------------------------------------------------
Ran 1 test in 12.676s
FAILED (failures=1)
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 12.86 sec
The following tests FAILED:
51 - TestPeakFinder.py (Failed)
Errors while running CTest
```https://jugit.fz-juelich.de/mlz/openhkl/-/issues/457GUI: default file extension not appended on Ubuntu 22.042022-07-14T12:38:02+02:00Raza, ZamaanGUI: default file extension not appended on Ubuntu 22.04The file extension `.nsx` is added by default when saving the Experiment state on Arch or MacOS. There is something in the system file dialogue for Ubuntu that "swallows" the file extension.The file extension `.nsx` is added by default when saving the Experiment state on Arch or MacOS. There is something in the system file dialogue for Ubuntu that "swallows" the file extension.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/439Core: generating shapes from *predicted* peaks2022-07-06T09:31:34+02:00Raza, ZamaanCore: generating shapes from *predicted* peaksIn order to generate a better shape model, we might try to "bootstrap" the shape model by:
1. constructing a shape model from *found* peaks
2. applying that model to predicted peaks
3. constructing a shape model from the resulting *predi...In order to generate a better shape model, we might try to "bootstrap" the shape model by:
1. constructing a shape model from *found* peaks
2. applying that model to predicted peaks
3. constructing a shape model from the resulting *predicted* peaks
4. applying that model to the same predicted peaks
If we do not integrate the predicted peaks after step 2, this will fail, and we end up with a shape model with 0 peaks. This is not the intended behaviour.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/427Windows: manifest.h incorrrectly generated2022-06-03T08:48:11+02:00Raza, ZamaanWindows: manifest.h incorrrectly generatedGit commit has has some garbage appended...
```
#ifndef MANIFEST_H
#define MANIFEST_H
#define VERSION "0.1.0"
#define GIT_BRANCH "develop"
#define COMMIT_HASH "6e62a28Windows Subsystem for Linux has no installed distributions.
Use '...Git commit has has some garbage appended...
```
#ifndef MANIFEST_H
#define MANIFEST_H
#define VERSION "0.1.0"
#define GIT_BRANCH "develop"
#define COMMIT_HASH "6e62a28Windows Subsystem for Linux has no installed distributions.
Use 'wsl.exe --list --online' to list available distributions
and 'wsl.exe --install <Distro>' to install.
Distributions can also be installed by visiting the Microsoft Store:
https://aka.ms/wslstore"
#define APPLICATION_NAME "NSXTool"
#define APPLICATION_CLAIM "Peak integration for neutron single crystal diffraction"
#define REPO_URL "https://jugit.fz-juelich.de/mlz/nsxtool"
#define ORGANIZATION_NAME "NSXTool"
#define ORGANIZATION_DOMAIN "https://jugit.fz-juelich.de/mlz/nsxtool"
#endif
```https://jugit.fz-juelich.de/mlz/openhkl/-/issues/425Core: bad OpenMP parallelisation of integration algorithm2022-05-13T08:11:01+02:00Raza, ZamaanCore: bad OpenMP parallelisation of integration algorithmAt first glance, `IPeakIntegrator::integrate` seems fairly trivial to OpenMP parallelise; however, in retrospect it seems that the results it generates are inconsistent with serial integration results. This needs to be reexamined.At first glance, `IPeakIntegrator::integrate` seems fairly trivial to OpenMP parallelise; however, in retrospect it seems that the results it generates are inconsistent with serial integration results. This needs to be reexamined.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/418GUI: SubframePredictPeaks crosshair does not update SubframeAutoIndexer2022-05-04T10:46:38+02:00Raza, ZamaanGUI: SubframePredictPeaks crosshair does not update SubframeAutoIndexerWhen the direct beam corsshair is adjusted in `SubframePredictPeaks` *before* in `SubframeAutoindexer`, the coordinate spin boxes do not update correctly in the latter.When the direct beam corsshair is adjusted in `SubframePredictPeaks` *before* in `SubframeAutoindexer`, the coordinate spin boxes do not update correctly in the latter.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/371Core: better integration at high resolution2022-01-13T12:16:26+01:00Raza, ZamaanCore: better integration at high resolutionInadequate merge statistics seem to be a result of poor integration of high resolution peaks, namely due to the low quality of the *shape* assigned to the high resolution peaks. The shape assignment procedure assumes that strong peaks su...Inadequate merge statistics seem to be a result of poor integration of high resolution peaks, namely due to the low quality of the *shape* assigned to the high resolution peaks. The shape assignment procedure assumes that strong peaks surrounding a high resolution peak are well characterised enough to give a good estimate for the shape of the peak via the mean covariance. This is often not the case because 1. there are often few neighbours of high resolution peaks; 2. the neighbours of high resolution peaks often also have poorly characterised shapes.
The solution to this could come in the form of a "bootstrapping", first using the found peaks to generate an initial guess for the shapes of the predicted peaks, but then using these intiaal guesses to iteratively refine the shapes at high resolution.https://jugit.fz-juelich.de/mlz/openhkl/-/issues/367Core: shape determination of high-resolution peaks is bad2021-12-09T10:03:37+01:00Raza, ZamaanCore: shape determination of high-resolution peaks is badFrom a *simulated* data set, the instrument states (detector position, sample position and orientation and incident wavevector) are known, and perfectly defined, so the only thing that needs refining is the unit cell, removing many unkno...From a *simulated* data set, the instrument states (detector position, sample position and orientation and incident wavevector) are known, and perfectly defined, so the only thing that needs refining is the unit cell, removing many unknowns. Here is a high resolution peak `(11, 7, 14)`:
![hi-res](/uploads/c6e1adc86abb705662a2c3a40da67463/hi-res.png)
vs a low resolution peak `(1, -2, 11)`:
![low-res](/uploads/c0899c2f51d4fe2c49dd711594cd2aad/low-res.png)
The issue seems to be in the way the shape is determined. The high resolution peaks lack sufficient neighbouring strong peaks to accurately determine the shape, and since they tend to have a large angular width (mosaicity), they intersect many frames and the "precession" across the image is not captured.
One possible method of resolving this is "bootstrapping" to determine peak shapes. We first use the peaks from the image analysis step to build an initial `ShapeCollection`, and use these shapes to integrate the predicted peaks. Then we build a new `ShapeCollection` from the *predicted* peaks, in the hope that we have more good shapes in the high resolution regime.Raza, ZamaanRaza, Zamaanhttps://jugit.fz-juelich.de/mlz/openhkl/-/issues/331Core: detector position is never refined2022-07-30T15:23:34+02:00Raza, ZamaanCore: detector position is never refinedWhenever refinement is done, the deltas for the detector offset are always zeroWhenever refinement is done, the deltas for the detector offset are always zeroRaza, ZamaanRaza, Zamaan