HOAB

History of a bug

Fast browsing and DNS

Rédigé par gorki Aucun commentaire

Problem :

I was surfing on some sites blocked by my DNS provider (no, not yggtorrent. Absolutely not).

So Firefox provide DNS over HTTP with NextDNS, sometimes slower than my provider DNS but well, not so bad.

Then for some reason, I tried to host a local DNS resolver. Well, it WAS slow.

Solution :

Unbound is DNS resolver :

  • easy to install
  • cache request locally, so save a few ms for a lot of requests !
  • and support DNS over https, etc…

Setup is quite simple thanks to internet knowledge :

Installation :
(https://memo-linux.com/debian-installer-le-serveur-dns-unbound/

apt install unbound
cd /var/lib/unbound/ 
wget ftp://ftp.internic.net/domain/named.cache
mv named.cache root.hints && chown unbound:unbound root.hints
mv /etc/unbound/ 
unbound.conf.d/root-auto-trust-anchor-file.conf root-auto-trust-anchor-file.conf.original
mkdir /var/log/unbound
chown unbound: /var/log/unbound
# modify apparmor (see at the end)
systemctl restart unbound

My configuration file :

server:
statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes
verbosity: 3
interface: 127.0.0.1
port: 53
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
access-control: 127.0.0.0/8 allow ## j'autorise mon serveur
access-control: 0.0.0.0/0 refuse ## j'interdis tout le reste de         l'Internet !
auto-trust-anchor-file: "/var/lib/unbound/root.key"
root-hints: "/var/lib/unbound/root.hints"
hide-identity: yes
hide-version: yes
harden-glue: yes
harden-dnssec-stripped: yes
use-caps-for-id: yes
cache-min-ttl: 3600
cache-max-ttl: 86400
prefetch: yes
num-threads: 6
msg-cache-slabs: 16
rrset-cache-slabs: 16
infra-cache-slabs: 16
key-cache-slabs: 16
rrset-cache-size: 256m
msg-cache-size: 128m
so-rcvbuf: 1m
unwanted-reply-threshold: 10000
do-not-query-localhost: yes
val-clean-additional: yes
#use-syslog: yes
#val-log-level:2 (0: default, nothing, 2: full)
logfile: /var/log/unbound/unbound.log
harden-dnssec-stripped: yes
cache-min-ttl: 3600
cache-max-ttl: 86400
prefetch: yes
prefetch-key: yes

And an additional apparmor configuration to be able to write in a dedicated file :
(https://b4d.sablun.org/blog/2018-09-27-when-unbound-wont-write-logs/)

vim /etc/apparmor.d/local/usr.sbin.unbound

# Site-specific additions and overrides for usr.sbin.unbound.
# For more details, please see /etc/apparmor.d/local/README.
/var/log/unbound/unbound.log rw,

 

Lire la suite de Fast browsing and DNS

Bash and the empty optional arguments on command line

Rédigé par gorki Aucun commentaire

Problem :

Well, I know that having named parameter is better “-file=” etc..

But for a simple task, I wanted to give :

./mycommand arg1 arg2 ‘’ ‘’ arg5

And pass those parameters to a function… 

Solution :

Not so lost in internet but easy to do at the end ! 

So basically, as simple as : 

# Solution OK : use arrau
all_args=("$@");
myfunction "${all_args[@]}"

# Loop over parameters
for i in "${@}"; do
   echo "$i"
done
for i in "${all_args[@]}"; do
   echo "$i"
done

From :

#!/bin/bash

all_args=("$@");

myfunction() {
 arg1=$1
 arg2=$2
 arg3=${3:-'default3'}
 arg4=${4:-'default4'}
 arg5=${5:-'default5'}

 echo "arg1=$arg1"
 echo "arg2=$arg2"
 echo "arg3=$arg3"
 echo "arg4=$arg4"
 echo "arg5=$arg5"
}

echo "--------------- args hard-codede"
myfunction 1 2 "" "" yes
echo "--------------- explode array with quote"
myfunction $(printf ""%s" " "${all_args[@]}")
echo "--------------- working just expand array"
myfunction "${all_args[@]}"

With the following command line : 

./test.sh 1 2 "" "" yes
--------------- args hard-codede
arg1=1
arg2=2
arg3=default3
arg4=default4
arg5=yes
--------------- explode array with quote
arg1="1"
arg2="2"
arg3=""
arg4=""
arg5="yes"
--------------- working just expand array
arg1=1
arg2=2
arg3=default3
arg4=default4
arg5=yes

 

 

 

Cannot create GC thread but a lot of memory

Rédigé par gorki Aucun commentaire

Problem :

Launching a JVM I have the message : "Cannot create GC thread. Out of system resources"

  • Enough memory
  • Enough swap
  • Enough ulimit
  • Enough threads-max
  • Enough CPU

Event extend the PID limit...

Important (at the end) : debian version = 10.11

Solution :

After a hours of googling, I found :

But none of these solutions works and none was matching the number I had :

  • number of open files < ulimit -n
  • maximum process/tasks < ulimit -u

But in a thread, I found something that was working : UserTasksMax.
I'm running SystemD, I have around 10805 task running for my user.
And from : https://manpages.debian.org/stretch/systemd/logind.conf.5.en.html

UserTasksMax=

Sets the maximum number of OS tasks each user may run concurrently. This controls the TasksMax= setting of the per-user slice unit, see systemd.resource-control(5) for details. If assigned the special value "infinity", no tasks limit is applied. Defaults to 33%, which equals 10813 with the kernel's defaults on the host, but might be smaller in OS containers.

For my suspect PID (a lot of files) :

  • cat /proc/21890/status | grep Thread => 1 thread
  • ls /proc/21890/task | wc
  • confirmed by the usual command : ps -eLf | grep calrisk | wc

I have around 10805 threads running for a given JVM very close to the limit.

Complete guide :

https://www.journaldufreenaute.fr/nombre-maximal-de-threads-par-processus-sous-linux/

Parameters not present in all man page, it could grown up to 12288 on latest version.

To be check !

 

 

Logitech Media Server and Synology DS218play

Rédigé par gorki Aucun commentaire

Problem :

Logitech media server is not supported anymore by Synology (bouuuuh.)

Spotted by number of threads :

  • https://community.synology.com/enu/forum/1/post/141628
  • https://www.avforums.com/threads/synology-and-lms.2332981/
  • https://community.jeedom.com/t/synology-maj-package-perl-lms-squeezebox-hs/57576
  • https://www.homecinema-fr.com/forum/source-dematerialisee-haute-fidelite-et-dac/logitech-squeezebox-t29927392-7185.html

I have a DS218play, ARMv8 based

Solution :

Instead of switching back to the old package, or create my new one, I decided to install docker.

I followed this thread : https://stackoverflow.com/questions/52520008/can-i-install-docker-on-arm8-based-synology-nas

Quite easy.

I follow also the script here : https://raw.githubusercontent.com/wdmomoxx/catdriver/master/install-docker.sh but I download docker from original site instead.

I use this image : https://registry.hub.docker.com/r/lmscommunity/logitechmediaserver/#!

And run this command, not perfect (log, daemon, docker image should be started... will update this later).

#!/bin/bash

docker run --network=host -it \
      -v "/volume1/partage/docker/lms/config":"/config":rw \
	  -v "/volume1/music":"/music":ro \
	  -v "/volume1/partage/docker/lms/playlist":"/playlist":rw \
	  -v "/etc/localtime":"/etc/localtime":ro \
	  -v "/etc/TZ":"/etc/timezone":ro \
	  -v "/dev":"/dev" \
       lmscommunity/logitechmediaserver &> /volume1/partage/docker/lms-docker.log

Added :  --network=host  (and remove mapping port) and -v "/dev":"/dev"

Upgrade debian et lost network

Rédigé par gorki Aucun commentaire

Problem :

I manage a dedicated server in OVH and I upgrade my debian from jessie to buster. Upgrade works quite well (it seems...) and I try to restart.

Server reboot fails as unreachable, fortunately OVH rescue mode allows me to login.

I check error log and first lost myself in RAID error message, but it was more simple than that.

Solution :

I check the /etc/network/interfaces file, it was OK

I check the logs files, clean, reboot, check again, still OK except that network was unreachable for named.

I finally remember that Debian switch to systemD in latest version so I tried to create system networking file manually : too complicate, it was not working.

In rescue mode, you can access your files as a mounted point so usual commands as systemctl does not work.

The solution was to chroot a shell :

  1. mkdir /mnt/md2
  2. mount /dev/md2 /mnt/md2
  3. chroot /mnt/md2 bash
  4. systemctl enable networking

And it works...

Now I have to check all other system to be sure that everything is working...

Begining with :

sudo apt-get update

sudo apt-get clean

sudo apt-get autoremove

sudo apt-get update && sudo apt-get upgrade

sudo dpkg --configure -a

 

Fil RSS des articles de cette catégorie