Merge pull request 'feature/#26_enhance_storage_stock_display' (#37) from feature/#26_enhance_storage_stock_display into develop

Reviewed-on: #37

README.md

@@ -0,0 +1,192 @@
+# Shimatta Kenkyusho Parts Database
+
+## Installation
+
+### Prerequisites
+Shimatta Kenkyusho (しまった・研究所) is a Django based web application. It is highly recommended to run it using the supplied docker setup. This removes the need of any special installation on the host system. This guide assumes, that `nginx` is running on the host system and can serve as a reverse proxy and webserver. For easiest download, it is recommended to clone the desired release with `git`.
+
+Install the requirements:
+
+**For Debian / Ubuntu:**
+```
+# apt-get update
+# apt-get install docker docker-compose-plugin nginx git
+```
+
+**For Arch based Systems:**
+```
+# pacman -S nginx docker docker-compose git
+```
+
+### Setup Shimatta Kenkyusho
+
+Clone this repository:
+```
+$ git clone https://git.shimatta.de/mhu/shimatta-kenkyusho.git
+```
+
+> Note: Shimatta Kenkyusho is currently not stable yet and the newest verison is in the `develop` branch. This will change once actual releases are done and merged to the `master` branch. You will be able to get the latest stable version from the `master` branch or a respective tag. For now, the `develop` is recommended.
+
+
+Change directory into the `shimatta-kenkyusho` folder cloned by git.
+
+Copy the `example.env` file to `.env` and edit it according to your needs:
+
+The following settings are required to be adapted:
+- `DJANGO_STATIC_VOL`: The directory the application will extract its static data into, which needs to be served by your webserver. See the example reverse proxy setup for more details.
+- `DJANGO_MEDIA_VOL`: The directory all media files like images uploaded to the application are stored here. This folder must be served by your webserver on the configured media URL.
+- `PGDATA_VOL`: The directory, the postgres database will store its files.
+- `PORT`: The TCP/IP port that the whole setup will listen on. Use a reverse proxy to forward to this port. *Do not directly expose it to the internet!*
+- `DJANGO_SECRET_KEY`: Provide a secret, and randomly generated key. Do not share this with anybody!
+- `DJANGO_ALLOWED_HOST`: Set this to the domain, the application will be reached at. E.g: `lab.example.com`
+- `DJANGO_MEDIA_URL`: Set this to the media URL at which your webserver serves the `DJANGO_MEDIA_VOL` diretory. E.g: `media.lab.example.com/` Note the **slash at the end**. It is important.
+- `DJANGO_USER_ID`: The user ID to run the application inside the docker container. This is the user id, that is used to write the to `DJANGO_STATIC_VOL` and `DJANGO_MEDIA_VOL`. Make sure the user has access.
+- `DJANGO_USER_GID`: The group ID to run the application inside the docker container. This is the group id, that is used to write the to `DJANGO_STATIC_VOL` and `DJANGO_MEDIA_VOL`.
+
+> Note: It is not recommended to run the docker container without a set `DJANGO_USER_ID` and `DJANGO_USER_GID`. It will default to `0 (root)`.
+
+Once the environment is set up, the docker containers can be built and started. Run
+```
+$ docker compose build
+```
+This will generate two container images:
+1. `shimatta-kenkyusho-shimatta-kenkyusho-web`: The django application
+2. `postgres`: A alpine based docker container containing the postgres database.
+
+Start the application as a service with
+```
+$ docker compose up -d
+```
+> Note: The initial startup might need a minute because the whole database etc. needs to be initialized first.
+
+Use 
+```
+$ docker ps
+```
+to check if the `shimatta-kenkyusho-shimatta-kenkyusho-db` and the `shimatta-kenkyusho-shimatta-kenkyusho-web` container are running and report a *healthy  status*.
+
+### Setup Initial Login User
+When started for the first time with a fresh database without any superuser configured, a superuser `admin` with password `admin` will be automatically generated.
+Use this user to login for the first time. In the django admin panel you can then either change the password of the `admin` user or create a new superuser with your own username and delete the `admin` user.
+As long as there is at least one superuser configured, no admin user will be regenerated upon startup.
+
+
+### Example Reverse Proxy Setup Using nginx
+Once the setup is configured the reverse proxy setup is needed. This setup serves three purposes:
+1. Redirect incoming requests to the django application running on the port `PORT` configured in the `.env`
+2. Serve static files at the URL: (e.g. `lab.example.com/static`). See `ALLOWED_HOST` configuration.
+3. Serve the media volume at the media URL (e.g. `media.lab.example.com`). See `DJANGO_MEDIA_URL`
+
+Example nginx configuration for `nginx >v2.25` with SSL and http2 / http3 support:
+> Note: This is by no means a replacement for the documentation of nginx and only serves as an example. Consult the documentation of your nginx version reagrding security and other issues.
+```
+# Force redirection from http to https for application
+server {
+        listen 80;
+        listen [::]:80;
+        server_name lab.example.com; # This must match your ALLOWED_HOST. Adapt domain.
+        allow all;
+
+        return 301 https://lab.example.com$request_uri; # Adapt domain
+}
+
+# Force redirection from http to https for media url
+server {
+        listen 80;
+        listen [::]:80;
+        server_name media.lab.example.com; # Adapt domain name according to DJANGO_MEDIA_URL
+        allow all;
+        return 301 https://media.lab.example.com$request_uri; # Adapt domain name
+}
+
+# Reverse Proxy for application
+server {
+        listen       443 ssl;
+        listen       [::]:443 ssl;
+        http2        on;
+
+
+   # Add this for HTTP3. If your nginx is older than 2.25 this might not be available
+######################################################################################
+        # listen       443 quic reuseport;
+        # listen       [::]:443 quic reuseport;
+        # Enable QUIC and HTTP/3
+        # ssl_early_data on;
+        # add_header Alt-Svc 'h3=":443"; ma=86400';
+#######################################################################################
+
+    server_name  lab.example.com; # Adapt domain
+
+    # Use letsencrypt as SSL certificate provider.  
+    ssl_certificate      /etc/letsencrypt/live/lab.example.com/fullchain.pem;
+    ssl_certificate_key  /etc/letsencrypt/live/lab.example.com/privkey.pem;
+
+    ssl_protocols TLSv1.3;
+    ssl_prefer_server_ciphers on;
+
+    ssl_session_cache    shared:SSL:1m;
+    ssl_session_timeout  5m;
+
+    ssl_ciphers  HIGH:!aNULL:!MD5;
+
+    location / {
+        allow all;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP  $remote_addr;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_pass http://127.0.0.1:8000;  # Adapt PORT from .env
+    }
+
+    location /static/ {
+        # Adapt path to static volume here. Note the slash at the end
+        alias /path/to/DJANGO_STATIC_VOL/; 
+        allow all;
+    }
+
+    client_max_body_size 60m;
+
+}
+
+# Serve the media files
+server {
+        listen       443 ssl;
+        listen       [::]:443 ssl;
+        # Add this for HTTP3. If your nginx is older than 2.25 this might not be available
+######################################################################################
+        # listen       443 quic reuseport;
+        # listen       [::]:443 quic reuseport;
+        # Enable QUIC and HTTP/3
+        # ssl_early_data on;
+        # add_header Alt-Svc 'h3=":443"; ma=86400';
+#######################################################################################
+        http2        on;
+
+        server_name  media.lab.example.com; # Adapt according to DJANGO_MEDIA_URL
+
+        # Use letsencrypt as SSL certificate provider. 
+        ssl_certificate      /etc/letsencrypt/live/media.lab.example.com/fullchain.pem;
+        ssl_certificate_key  /etc/letsencrypt/live/media.lab.example.com/privkey.pem;
+
+        ssl_protocols TLSv1.3;
+        ssl_prefer_server_ciphers on;
+
+        ssl_session_cache    shared:SSL:1m;
+        ssl_session_timeout  5m;
+
+        ssl_ciphers  HIGH:!aNULL:!MD5;
+        error_page 502 /lab_down.html;
+
+        allow all;
+        root /path/to /DJANGO_MEDIA_VOL/; # Adapt this to the volume provided.
+}
+```
+Congratulations. Your shimatta kenkyusho installation is now fully setup.
+> Note that, the `compose.yaml` contains a restart-policy. By default the containers will restart automatically, even after a reboot of the host machine, if the docker service is enabled.
+
+
+## Backup and Restore
+> TODO
+
+## Debugging and Development
+> Todo
+

compose.yaml

@@ -5,7 +5,7 @@ services:
   shimatta-kenkyusho-web:
     <<: *restart_policy
     build: .
-    user: "${DJANGO_USER_ID}:${DJANGO_USER_GID}"
+    user: "${DJANGO_USER_ID:-0}:${DJANGO_USER_GID:-0}"
     volumes:
       - "${DJANGO_STATIC_VOL:-./run/static}:/var/static"
       - "${DJANGO_MEDIA_VOL:-./run/media}:/var/media"

shimatta_kenkyusho/api/serializers.py

@@ -44,6 +44,7 @@ class ComponentSerializer(serializers.HyperlinkedModelSerializer):
     ro_component_type = serializers.ReadOnlyField(source='component_type.class_name')
     ro_parameters = ComponentParameterSerializer(many=True, source='componentparameter_set', read_only=True)
     ro_distributor_numbers = ComponentDistributorNumSerializer(many=True, source='distributornum_set', read_only=True)
+    ro_dynamic_description = serializers.ReadOnlyField(source='dynamic_description')
 
     class Meta:
         model = parts_models.Component
@@ -60,12 +61,14 @@ class ComponentSerializer(serializers.HyperlinkedModelSerializer):
                   'ro_image',
                   'ro_component_type',
                   'ro_parameters',
-                  'ro_distributor_numbers']
+                  'ro_distributor_numbers',
+                  'ro_dynamic_description']
 
 class StockSerializer(serializers.HyperlinkedModelSerializer):
     id = serializers.ReadOnlyField()
     ro_package_name = serializers.ReadOnlyField(source='component.package.name')
     ro_component_name = serializers.ReadOnlyField(source='component.name')
+    ro_component_dynamic_description = serializers.ReadOnlyField(source='component.dynamic_description')
     ro_manufacturer_name = serializers.ReadOnlyField(source='component.manufacturer.name')
     ro_image = serializers.ReadOnlyField(source='component.get_resolved_image')
     class Meta:

shimatta_kenkyusho/parts/migrations/0015_componenttype_description_template.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.1.3 on 2025-01-31 21:14
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('parts', '0014_storage_expand_sub_storage_stocks'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='componenttype',
+            name='description_template',
+            field=models.TextField(blank=True),
+        ),
+    ]

shimatta_kenkyusho/parts/migrations/0016_componentparametertype_interfix.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.1.3 on 2025-01-31 21:44
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('parts', '0015_componenttype_description_template'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='componentparametertype',
+            name='interfix',
+            field=models.CharField(blank=True, max_length=10),
+        ),
+    ]

shimatta_kenkyusho/parts/migrations/0017_alter_componentparametertype_interfix_and_more.py

@@ -0,0 +1,23 @@
+# Generated by Django 5.1.3 on 2025-02-02 10:52
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('parts', '0016_componentparametertype_interfix'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='componentparametertype',
+            name='interfix',
+            field=models.CharField(blank=True, help_text='char to be used as decimal point in dynamic description eg. 2R2', max_length=10),
+        ),
+        migrations.AlterField(
+            model_name='componenttype',
+            name='description_template',
+            field=models.TextField(blank=True, help_text="Template to assemble the dynamic description. Use template syntax, access the component with 'object', parameters with 'param_*'."),
+        ),
+    ]

shimatta_kenkyusho/parts/models.py

@@ -6,6 +6,7 @@ from django.core.exceptions import ValidationError
 from django.core.validators import RegexValidator
 from django.dispatch import receiver
 from django.core.validators import MinValueValidator, MaxValueValidator
+from django.template import engines
 import os
 import uuid
 from shimatta_modules.EngineeringNumberConverter import EngineeringNumberConverter as NumConv
@@ -29,6 +30,7 @@ class ComponentParameterType(models.Model):
 	parameter_name = models.CharField(max_length=50, unique=True)
 	parameter_description = models.TextField(null=False, blank=True)
 	unit = models.CharField(max_length=10, null=False, blank=True)
+	interfix = models.CharField(max_length=10, null=False, blank=True, help_text="char to be used as decimal point in dynamic description eg. 2R2")
 	parameter_type = models.CharField(max_length=1, choices=TYPE_CHOICES, default='N')
 
 	def __str__(self):
@@ -49,6 +51,9 @@ class ComponentType(models.Model):
 	class_name = models.CharField(max_length=50, unique=True)
 	passive = models.BooleanField()
 	possible_parameter = models.ManyToManyField(ComponentParameterType, blank=True)
+	description_template = models.TextField(blank=True,
+										 help_text="Template to assemble the dynamic description. "
+										 "Use template syntax, access the component with 'object', parameters with 'param_*'.")
 
 	def __str__(self):
 		return '[' + self.class_name + ']'
@@ -78,20 +83,24 @@ class Storage(models.Model):
 	# caching variable for subtrees
 	storage_list = None
 
-	def get_path_components(self):
+	def get_path_components(self, top_level=None):
 		chain = []
 		iterator = self
 		chain.append(self)
 		while iterator.parent_storage is not None:
+
+			if top_level and iterator.parent_storage == top_level:
+				break
+
 			chain.append(iterator.parent_storage)
 			iterator = iterator.parent_storage
 
 		return chain
 
-	def get_full_path(self):
+	def get_full_path(self, top_level=None):
 		output = ''
 		
-		chain = self.get_path_components()
+		chain = self.get_path_components(top_level)
 
 		for i in range(len(chain) - 1, -1, -1):
 			output = output + '/' + chain[i].name
@@ -268,6 +277,22 @@ class Component(models.Model):
 		if sum is None:
 			sum = 0
 		return sum
+	
+	@property
+	def dynamic_description(self):
+
+		if not self.component_type or not self.component_type.description_template:
+			return ''
+
+		django_engine = engines["django"]
+		template = django_engine.from_string(self.component_type.description_template)
+
+		parameters = list(ComponentParameter.objects.filter(component=self))
+		parameters += list(PackageParameter.objects.filter(package=self.package))
+
+		context = {f'param_{param.parameter_type.parameter_name}': param for param in parameters}
+		context.update({'object': self})
+		return template.render(context)
 
 class AbstractParameter(models.Model):
 	class Meta:
@@ -302,6 +327,25 @@ class AbstractParameter(models.Model):
 		elif my_type == 'F':
 			return self.text_value
 
+	def resolved_value_as_short_string(self):
+		my_type = self.parameter_type.parameter_type
+
+		if my_type == 'E' or my_type == 'I':
+			# Engineering float number
+			(num, prefix) = NumConv.number_to_engineering(self.value, it_unit=(True if my_type=='I' else False))
+			result = f'{round(num, 3):g}'
+			interpostfix = (prefix if prefix else self.parameter_type.interfix or '.')
+			if '.' in result:
+				result = result.replace('.', interpostfix)
+			else:
+				result = result + interpostfix
+			return result
+		elif my_type == 'N':
+			# Standard float number
+			return  f'{round(self.value, 3):g}{self.parameter_type.unit}'
+		else:
+			return self.resolved_value_as_string()
+
 class ComponentParameter(AbstractParameter):
 	class Meta:
 		unique_together = ('component', 'parameter_type')

shimatta_kenkyusho/parts/templatetags/storage_tags.py

@@ -0,0 +1,9 @@
+import datetime
+from django import template
+
+register = template.Library()
+
+
+@register.filter(name="get_relative_storage_path")
+def get_relative_storage_path(storage, top_level):
+    return f'.{storage.get_full_path(top_level)}'

shimatta_kenkyusho/templates/parts/components-detail.html

@@ -76,6 +76,11 @@
                             No description available
                         </div>
                     {% endif %}
+                    
+                    {% if component.dynamic_description %}
+                    <h2>Dynamic Description</h2>
+                    <pre>{{ component.dynamic_description }}</pre>
+                    {% endif %}
                 </div>
                 <div class="col-4">
                     {% if component.pref_distri %}

shimatta_kenkyusho/templates/parts/components.html

@@ -54,7 +54,12 @@
                                 Manufacturer: {{comp.manufacturer}}
                             {% endif %}
                         </div>
-                        <span class="badge bg-primary rounded-pill">{{comp.get_total_amount}}</span>
+                        <div class="flex-grow-1 d-block ms-3" style="text-align: right;">
+                            <pre>{{ comp.dynamic_description }}</pre>
+                        </div>
+                        <div style="width: 10%; text-align: right;">
+                            <span class="badge bg-primary rounded-pill me-2">{{comp.get_total_amount}}</span>
+                        </div>
                     </li>
                     </a>
                 {% endfor %}

shimatta_kenkyusho/templates/parts/stocks-detail.html

@@ -2,6 +2,7 @@
 {% load qr_code %}
 {% load static %}
 {% load crispy_forms_tags %}
+{% load storage_tags %}
 
 {% block content %}
 <div class="container">
@@ -56,8 +57,8 @@
                             Responsible: {{ storage.responsible }}
                             </div>
                             <span class="badge ms-1 bg-primary rounded-pill" data-bs-toggle="tooltip" data-bs-placement="top" title="Total number of stored parts">{{storage.get_total_stock_amount}}</span>
-                            <span class="badge ms-1 bg-secondary rounded-pill" data-bs-toggle="tooltip" data-bs-placement="top" title="Number of stored lots">{{storage.get_total_stock_count}}</span>
-                            <span class="badge ms-1 bg-info rounded-pill" data-bs-toggle="tooltip" data-bs-placement="top" title="Number of substorages">{{storage.get_total_substorage_amount}}</span>
+                            <span class="badge ms-1 bg-secondary rounded-pill d-none d-lg-block" data-bs-toggle="tooltip" data-bs-placement="top" title="Number of stored lots">{{storage.get_total_stock_count}}</span>
+                            <span class="badge ms-1 bg-info rounded-pill d-none d-lg-block" data-bs-toggle="tooltip" data-bs-placement="top" title="Number of substorages">{{storage.get_total_substorage_amount}}</span>
                         </li>
                     </a>
                 {% endfor %}
@@ -97,14 +98,20 @@
                             {% if stock.component.manufacturer %}
                                 Manufacturer: {{stock.component.manufacturer}}
                             {% endif %}
+                            {% if stock.storage != storage %}
+                            <span class="text-secondary"><br>{{ stock.storage|get_relative_storage_path:storage }}</span>
+                            {% endif %}
                             {% if stock.lot %}
                                 <span class="text-secondary"><br>Lot: {{stock.lot}}</span>
                             {% endif %}
                         </div>
-                        <div class="flex-grow-2 ms-3 d-none d-lg-block">
+                        <div class="flex-grow-2 ms-3 d-none d-lg-block" style="text-align: right;">
+                            <pre>{{ stock.component.dynamic_description }}</pre>
+                        </div>
+                        <div class="flex-grow-2 ms-5 d-none d-lg-block">
                             {% qr_from_text stock.get_qr_code size="6" image_format="svg" %}
                         </div>
-                        <div class="ms-3">
+                        <div class="ms-3" style="width: 20%;">
                             Amount: {{stock.amount}}
                             {% if stock.watermark >= 0 %}
                             <br>Watermark: {{stock.watermark}}