Monday, July 8, 2024

Rsync and the Trailing or Omitted Slash

I learned this recently, and it can be a real headache if one isn't aware of it. Rsync copies source FILES into the destination when a trailing slash is used on source. It copies source FOLDER and FILES into destination when trailing slash is OMITTED.

So...

rsync -avh /home/bobby/Documents/ bobby@192.168.0.10:/mnt/300gb/

results in all the files and subdirectories of /home/bobby/Documents/ being copied to /mnt/300gb/.


But if you omit the trailing slash on the source, like this

rsync -avh /home/bobby/Documents bobby@192.168.0.10:/mnt/300gb/

you get all the files and subdirectories of /home/bobby/Documents being copied to /mnt/300gb/Documents/.

REMEMBER THIS.

HOW TO Install Pkgs in Embedded XigmaNAS (FreeBSD) System

 I found this useful information and wanted to pass it along. I haven't tried it yet, but I may need to in the future. I almost needed to use this to get apcupsd installed on embedded XigmaNAS, but instead opted to write a command script (to be run post-init via System/Advanced/Command Scripts) that copies the necessary apcupsd files from a local mount point to their normal locations (/usr/sbin/ along with the directory with most of the files /usr/local/etc/apcupsd/ at every boot. Embedded installations run from RAM to limit writes to the USB stick, so any pkgs that are installed do not persist beyond reboot.

jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

[HOW TO] Install pkgs in an embedded install

#1

Post by jamaroney »

You can install and run pkgs in an embedded install via unionfs_mount. I have done this for close to 10 years without any problems whatsoever. No jails or VM, and it's quick and easy to set up.

INSTRUCTIONS:
  • Choose one of your data discs (NOT system disc). Mine is /mnt/HDD.
  • Enter the following commands (change the name "extensions" to whatever you choose):

Code: Select all

cd /mnt/HDD
mkdir extensions
mkdir extensions/usr
mkdir extensions/usr/local
mkdir extensions/var
mkdir extensions/var/db
mkdir extensions/var/db/pkg
mkdir extensions/var/cache
mkdir extensions/var/cache/pkg
  • Then go to "Command Scripts" and add the following commands in this order (all PostInit and usually the first commands). They're all important!

Code: Select all

rsync -a /usr/local/ /mnt/HDD/extensions/usr/local/ 
mount_unionfs -o rw /mnt/HDD/extensions/usr/local/ /usr/local/
mount_unionfs -o rw /mnt/HDD/extensions/var/db/pkg /var/db/pkg
mkdir /var/cache && mkdir /var/cache/pkg
mount_unionfs -o rw /mnt/HDD/extensions/var/cache/pkg /var/cache/pkg
  • Of course, you should modify the above commands to reflect the name of your selected data disc and the name you choose for your primary directory (in my case /extensions)
  • Reboot XNAS
You should now be able to add (and later upgrade) all desired pkgs, and they will remain and operate the same as if you did a full install.

>>>>>>>>>>>>>>>>>>>>>

You might hear of the so-called "dangers" of unionfs_mount - THEY DON'T APPLY TO AN EMBEDDED INSTALL, which by its very nature makes it impossible to screw up your system. Should there ever be any problem installing a package(and I've never experienced any), all one has to do is disable the command script items and reboot for a "clean" install.
Last edited by jamaroney on May 10th, '24, 04:05, edited 2 times in total.
gerdpeter
New User
New User
Posts: 1
Joined: Jul 13th, '23, 03:39
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#2

Post by gerdpeter »

Hi jamaroney,

thank you for this relay helpful information
I'm new with XigmaNAS.
for example, I will install wireguard and as second part zrep for remote Snapshots (Backups) with your setup.
can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
And can you tell me the path, where the config files are, I have to editing for persistent changes?
Or in other words: where are the used .../etc/ directory in which I can editing
For one more example: where, on which path I can editing the sshd_config file on command line for persistent changes after a reboot?

regards,
gerdpeter
User avatar
raulfg3
Site Admin
Site Admin
Posts: 917
Joined: Dec 13th, '20, 00:15
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#3

Post by raulfg3 »

gerdpeter wrote: Jul 13th, '23, 03:53 Hi jamaroney,

thank you for this relay helpful information
I'm new with XigmaNAS.
for example, I will install wireguard and as second part zrep for remote Snapshots (Backups) with your setup.
can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
And can you tell me the path, where the config files are, I have to editing for persistent changes?
Or in other words: where are the used .../etc/ directory in which I can editing
For one more example: where, on which path I can editing the sshd_config file on command line for persistent changes after a reboot?

regards,
gerdpeter
please do a full install instead, and you have no problem to install packages .

And remember, some packages can broke your XigmaNAS install, so prefeed way is to install on a jail.
XigmaNAS 12.3.0.4.8919 (ZFSonRoot) on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz 36GB of ECC RAM, 12x3TB disk in 1 vdev = Vpool = 36TB Raw size , so 29TB usable size (I Have other NAS as Backup)

Wiki
Last changes
HP T510
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#4

Post by jamaroney »

I'll repeat myself yet again for the disbelievers and others who simply WANT me to be wrong:

I HAVE DONE THIS FOR CLOSE TO 10 YEARS WITHOUT ANY PROBLEMS WHATSOEVER.

and

Should there ever be any problems (and I've never experienced ANY), all one has to do is disable the command script items and reboot for a "clean" install.
All packages previously installed will remain intact on the data drive, but simply not started, so no data or configs are lost.
Last edited by jamaroney on Jul 14th, '23, 06:43, edited 7 times in total.
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#5

Post by jamaroney »

Can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
YES - do the install exactly as in a full install.

And can you tell me the path, where the config files are, I have to editing for persistent changes?
Your config files are in the same place as a full install. But in an embedded install, the unionfs_mount to another drive keeps them persistent.
benoliver999
New User
New User
Posts: 1
Joined: Sep 15th, '23, 03:06
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#6

Post by benoliver999 »

I am using this method with some success, thanks for the guide! I'm pretty hands-off with the server and was going round in circles trying to set up jails.

One thing: I'm trying to install mkvtoolnix to remux some discs I've ripped. It works fine, but doesn't survive a reboot. I get

Code: Select all

Shared object "libQt5Core.so.5" not found, required by "mkvmerge"
Do you have any idea what could be causing this?
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#7

Post by jamaroney »

My guess is that the pkg install is creating the file somewhere other than /usr/local/, which is why it isn't surviving reboot. But I've installed many packages, and have never encountered this before.

If you're able to get mkvtoolnix running again, do a search for "libQt5Core.so.5" to find where it's located.
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#8

Post by jamaroney »

To test things, I installed the pkg mkvtoolnix (and its many dependencies), and "libQt5Core.so.5" is right where it should be, in folder /usr/local/lib/Qt5/, so unless there's a script within mkvtoolnix telling it to do funky things to such files, there's no reason it wouldn't survive a reboot.

Please verify that the file is there - if so, then your problem lies elsewhere, and not in the fact that it's an embedded install.

Saturday, March 18, 2023

Changing Default Web Browser in Lubuntu

 After much frustration trying unsuccessfully the normal ways of setting a browser (brave-browser) as the default in Lubuntu, I found that printfgabriel posted the work-around:

Configuring x-www-browser in Alternatives, XDG’s default-web-browser and Lubuntu’s default internet browser application in Session Settings is not enough. Turns out Chrome looks at /etc/xdg/xdg-Lubuntu/mimeapps.list and you have to edit it as sudo and replace the instances of the “firefox” string with your browser of choice.

I replaced all instances of "firefox" with "brave-browser" and that worked. Whew!

Monday, April 11, 2022

Using SCP with Filenames That Contain Special Characters

 Are you having trouble using scp to transfer files that have special characters? The error looks something like this:

syntax error near unexpected token `('

Your next move is to try escaping those special characters with a '\' preceding each one. But you still may get the same error.

Then you may try to put the filename and path in single quotes. Alas, the error persists. After some searching on the interwebs, you find that nested quotation marks may be the answer. Something like this:

scp joe@remote.server.com:"'/home/joe/Documents/This is a really stupid way to name files.txt'"

Now you may get the following error because of security updates to software that added a check for nefarious commands.

protocol error: filename does not match request

However, along with the update, there was added a '-T' parameter to scp command to work around this added check, for use in cases where it is preventing WANTED files from being transferred.

So a nested quote with -T will do the trick, thusly:

scp -T joe@remote.server.com:"'/home/joe/Documents/This is a really stupid way to name files.txt'"

Tuesday, December 8, 2020

Stray Voltage in the Stock Water Trough

 After about 20 years of being a horse owner, something strange happened to me recently. This winter, a day after hooking up the de-icer for the horse water trough, I saw the horses standing at the trough licking their lips with desire but not drinking. After a while longer, they were still there. I've used the simple submerged type of de-icing water trough heaters for many years, and they have gone bad and stopped heating, but I've never had a problem with them introducing voltage to the water trough. In fact, that thought never even occurred to me until this happened.

I tested the water for voltage with my multi-meter by stabbing one probe into the wet soil and the other into the water. It read about 1.5 volts. That's isn't a lot of electricity, but it's certainly enough to make the horses stop drinking... obviously.

I tested the outdoor receptacle and got some strange reading of apparent stray voltage. I decided to replace the receptacle, thinking maybe it was bad after being exposed to extreme temperatures and humidity for 14 years. The new receptacle didn't improve anything. Then I examined the circuit all the way back to the panel in the house... everything looked good. I called the manufacturer of the de-icer and was practically cut-off mid-sentence when I mentioned that I have an electric fence. I was told to drive a grounding rod into the earth near the water trough and connect a copper wire to it and then connect it to the trough to ground it. The company representative promised the electricity in the water would drain through that and everything would be fine. I did just that, and it changed nothing.

I tested the heater in a bucket of water using other receptacles that were actually installed at the house. Same 1.5 volts going through the water. That eliminated the theory that the cause was some anomaly with the fence/trough receptacle.

After much research on internet forums, I learned that 1) I'm not the only one to experience this and 2) it seems to baffle a lot of folks, from owners of one or two horses all the way to big dairy operations. I read all kinds of complicated proposed solutions from building an "equipotential pad" on which the animals must stand to drink (expensive & complicated) to having the utility company install a "neutral isolation" device. I came across one forum post that sounded almost too simple to be true, but had a ring of logic to it. This person originally found this tip in the 10/2014 post on the Henry Milker blog, which is no longer active, but at the time of this post can be found in the way back machine here. The solution? Plug the three-pronged de-icer into a 3-prong-to-2-prong electrical plug adapter with a metal grounding tab. Connect the grounding tab on the adapter to a copper wire that is connected to the grounding rod that is driven into the ground near the water trough. Plug it all in. The grounding wire going directly to the grounding rod provides a strong and close connection to earth ground right where it is needed. I can't provide any more technical explanation than that. In fact, a professional electrician friend of mine struggled to explain a lot of this whole problem. The important thing is that it works. When I tested the water with the de-icer installed this way, the voltage in the water was zero.

If you can shed light on any of this from an authoritative, technical point of view, please do. Meanwhile, I won't have to break ice this winter. Now I just need to convince the horses that it's safe to drink the water again.

Tuesday, September 1, 2020

Install Windows 10 after Ubuntu / Linux


When I rebooted, Windows 10 was not on the Grub menu, but that was an easy fix. After booting into Ubuntu, simply use a terminal to run "os-prober" which should find Windows 10, then run "update-grub"
 
-----

Install Windows after Ubuntu/Linux

As you know, the most common, and probably the most recommended way of dual booting Ubuntu and Windows is to install Windows first and then Ubuntu. But what if you want to install Windows second on a computer with Ubuntu/Linux already installed?

This guide assumes that you have Ubuntu already installed on your system, and will guide you through installing Windows 7/8/8.1/10 when you have a working Ubuntu/Linux partition.

The problem here is that the Windows installer tends to overwrite Grub bootloader or whatever making only the Windows installation bootable. But the good news is that your Linux partition is untouched, including the original bootloader and the other Grub configurations. So restoring or recovering Grub is just a matter of running a few commands on the terminal.

Install Windows on a Separate Partition

If your Ubuntu installation occupies the entire hard drive, to be able to create free space for the Windows installation, you may have to shrink an existing partition using a disk utility like GParted. GParted comes preinstalled on Ubuntu LiveCD.

Boot into a Windows installation media and complete the installation

Once installed, because of Microsoft’s false assumption that there are no non-Microsoft operating systems you’ll be taken straight into Windows.

Restore / Reinstall Grub 2 with a Ubuntu Live Media

  1. Now boot into a Ubuntu Live/USB or CD.
  2. Open a terminal. (Ctrl + Alt + t)
  3. Use a command like lsblk, blkid or GParted to identify your Linux root, and boot partitions. You may or may not have a separate boot partition.
  4. Find Linux Partitions

     $ lsblk -f
     NAME                  FSTYPE     MOUNTPOINT
     sda                                                                            
     ├─sda1                ext4       /boot
     ├─sda2                ext4       /
     ├─sda3                ntfs       
    

    In my case,

    /dev/sda1 is the Linux boot partition

    /dev/sda2 is the Linux root partition

    /dev/sda3 is the Windows partition

  5. Setup a Chroot Environment

    Now we’ll be repairing our old grub bootloader in a chroot environment.

    Mount your root filesystem under /mnt

     sudo mount /dev/sda2 /mnt/
    

    If you’re using BTRFS (New Linux B-Tree File System) instead of EXT4, the above command should change to sudo mount -t btrfs -o subvol=@ /dev/sda2 /mnt/

    Run ls /mnt to verify that the correct partition is mounted. It should list the content of your root.

     $ ls /mnt
     bin   dev  home        lib64       media  opt   root  sbin  sys  usr  vmlinuz
     boot  etc  initrd.img  lost+found  mnt    proc  run   srv   tmp  var  vmlinuz.old
    

    Mount the boot partition. (SKIP if you don’t have a separate boot partition)

     sudo mount /dev/sda1 /mnt/boot
    

    If you have an EFI partition you should mount that also. It is basically a small FAT32 partition around 100mb. Please refer to the section on how to reinstall Grub in an EFI system.

    Bind these directories, so grub can detect other operating systems, like so.

     sudo mount --bind /dev /mnt/dev
     sudo mount --bind /dev/pts /mnt/dev/pts
     sudo mount --bind /proc /mnt/proc
     sudo mount --bind /sys /mnt/sys
    

    Let’s chroot into our existing Ubuntu/Linux system on the hard disk.

     sudo chroot /mnt
    
  6. Reinstall Grub.

     grub-install /dev/sda (specify the disk `/dev/sdX`, not `/dev/sdaX`)
     grub-install --recheck /dev/sda
     exit
    

    Once you’ve successfully reinstalled Grub, restart the computer. On next reboot Grub will be the default bootloader and you’ll be presented with a list of operating systems to choose from (Ubuntu and Windows, of course).

Reinstalling Grub in an EFI Based System

Reinstalling Grub bootloader in a UEFI based computer is no different from installing Grub on a legacy BIOS system. Here we also mount the EFI partition before chrooting, as we’d do with the boot partition.

Case #2
$ lsblk -f
NAME                  FSTYPE     MOUNTPOINT
sda                                                                            
├─sda1                ext4       /boot/efi
├─sda2                ext4       /
├─sda3                ntfs       

Here /dev/sda1 is the EFI partition and /dev/sda2 is the root.

sudo mount /dev/sda2 /mnt/

Mount EFI partition under ‘/mnt/boot/efi’

sudo mount /dev/sda1 /mnt/boot/efi

Forgetting to do so will result in errors being thrown while trying to execute grub-install: “/boot/efi does not appear to be the EFI partition”

As shown previously, bind the /dev, /dev/pts, /proc, and /sys directories.

Finally,

sudo chroot /mnt
grub-install
exit

In some rare occasions, Grub might not detect Windows. In that case just login to Ubuntu (installed one on the disk, not the bootable Live USB/CD) and execute,

sudo os-prober
sudo update-grub
sudo reboot

Happy Dual-Booting!


 

Wednesday, August 5, 2020

Making an Image of a Disk (or Partition) Using dd WITHOUT Including Extra Empty Space

This write-up helped me a lot with this.

I have a Raspberry Pi that gets a lot of use as a media control station, and as a result, it periodically needs to be re-imaged. This is a lot faster and easier than re-installing everything from scratch and then setting up Mythtv, all my network settings, etc. So, a long time ago I used dd in Linux to make an image of a fresh install with all the basic things setup for my network, including Mythtv client. That install was on a microSD card that was 8 GB, so the resulting .img file was 8 GB even though at least half of that was unused. When I tried to write that image to a different 8 GB microSD card, it failed because of insufficient space. If one card had even one bit less (for reasons I do not know), it would be too small. What I needed was to get rid of all that empty space in the .img file.

Write the .img file to a larger memory card that will hold all 8 GB. Then use GParted to shrink the partition that has all the unused space in it to a reasonable size. Now, use sudo fdisk -l -u=sectors to find the number of the last sector used by the last partition.

Disk /dev/sdb: 7.4 GiB, 7960788992 bytes, 15548416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00ed4

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdb1         4096   147455   143360   70M  c W95 FAT32 (LBA)
/dev/sdb2       147456  6348892  6421187  7.4G 83 Linux
 

Make a new .img file from this memory card with the smaller partition. The key here is the parameter "count."

sudo dd if=/dev/mmcblk0 of=NO_WASTED_SPACE.img bs=512 count=6348892

where the count value is the number of the last sector of the last partition you saw in the fdisk listing. That makes dd stop reading anything after the last partition, namely all that extra empty space. Now you have an image file that is just the size you need, it won't take up extra storage space because of the unused space in the image, and when you write the image to a disk (memory card), you won't get errors and failure because the image is larger than the card you are trying to write it on.
 
Something important I learned, the sector size on the block device (memory card) shown via fdisk was 512. When telling it the sector to stop on, you have to make sure the input and output blocksize (bs=) is the same as the sector size, in this case 512. The first time I tried it, the image file was turning out to be huge because I use a block size of 4096 (bs=4096), but the number I used as the last sector to read (count=) was based on a sector size (block size) of 512. When I switched bs back to 512 (I believe the default), it worked fine. Now my image file is 3 GB instead of 8 GB, less space sitting on my hard disk drive, and it won't be picky writing to any device with 3 GB space or more.