Monday, July 8, 2024

Rsync and the Trailing or Omitted Slash

I learned this recently, and it can be a real headache if one isn't aware of it. Rsync copies source FILES into the destination when a trailing slash is used on source. It copies source FOLDER and FILES into destination when trailing slash is OMITTED.

So...

rsync -avh /home/bobby/Documents/ bobby@192.168.0.10:/mnt/300gb/

results in all the files and subdirectories of /home/bobby/Documents/ being copied to /mnt/300gb/.


But if you omit the trailing slash on the source, like this

rsync -avh /home/bobby/Documents bobby@192.168.0.10:/mnt/300gb/

you get all the files and subdirectories of /home/bobby/Documents being copied to /mnt/300gb/Documents/.

REMEMBER THIS.

HOW TO Install Pkgs in Embedded XigmaNAS (FreeBSD) System

 I found this useful information and wanted to pass it along. I haven't tried it yet, but I may need to in the future. I almost needed to use this to get apcupsd installed on embedded XigmaNAS, but instead opted to write a command script (to be run post-init via System/Advanced/Command Scripts) that copies the necessary apcupsd files from a local mount point to their normal locations (/usr/sbin/ along with the directory with most of the files /usr/local/etc/apcupsd/ at every boot. Embedded installations run from RAM to limit writes to the USB stick, so any pkgs that are installed do not persist beyond reboot.

jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

[HOW TO] Install pkgs in an embedded install

#1

Post by jamaroney »

You can install and run pkgs in an embedded install via unionfs_mount. I have done this for close to 10 years without any problems whatsoever. No jails or VM, and it's quick and easy to set up.

INSTRUCTIONS:
  • Choose one of your data discs (NOT system disc). Mine is /mnt/HDD.
  • Enter the following commands (change the name "extensions" to whatever you choose):

Code: Select all

cd /mnt/HDD
mkdir extensions
mkdir extensions/usr
mkdir extensions/usr/local
mkdir extensions/var
mkdir extensions/var/db
mkdir extensions/var/db/pkg
mkdir extensions/var/cache
mkdir extensions/var/cache/pkg
  • Then go to "Command Scripts" and add the following commands in this order (all PostInit and usually the first commands). They're all important!

Code: Select all

rsync -a /usr/local/ /mnt/HDD/extensions/usr/local/ 
mount_unionfs -o rw /mnt/HDD/extensions/usr/local/ /usr/local/
mount_unionfs -o rw /mnt/HDD/extensions/var/db/pkg /var/db/pkg
mkdir /var/cache && mkdir /var/cache/pkg
mount_unionfs -o rw /mnt/HDD/extensions/var/cache/pkg /var/cache/pkg
  • Of course, you should modify the above commands to reflect the name of your selected data disc and the name you choose for your primary directory (in my case /extensions)
  • Reboot XNAS
You should now be able to add (and later upgrade) all desired pkgs, and they will remain and operate the same as if you did a full install.

>>>>>>>>>>>>>>>>>>>>>

You might hear of the so-called "dangers" of unionfs_mount - THEY DON'T APPLY TO AN EMBEDDED INSTALL, which by its very nature makes it impossible to screw up your system. Should there ever be any problem installing a package(and I've never experienced any), all one has to do is disable the command script items and reboot for a "clean" install.
Last edited by jamaroney on May 10th, '24, 04:05, edited 2 times in total.
gerdpeter
New User
New User
Posts: 1
Joined: Jul 13th, '23, 03:39
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#2

Post by gerdpeter »

Hi jamaroney,

thank you for this relay helpful information
I'm new with XigmaNAS.
for example, I will install wireguard and as second part zrep for remote Snapshots (Backups) with your setup.
can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
And can you tell me the path, where the config files are, I have to editing for persistent changes?
Or in other words: where are the used .../etc/ directory in which I can editing
For one more example: where, on which path I can editing the sshd_config file on command line for persistent changes after a reboot?

regards,
gerdpeter
User avatar
raulfg3
Site Admin
Site Admin
Posts: 917
Joined: Dec 13th, '20, 00:15
Location: Madrid (ESPAÑA)
Contact:
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#3

Post by raulfg3 »

gerdpeter wrote: Jul 13th, '23, 03:53 Hi jamaroney,

thank you for this relay helpful information
I'm new with XigmaNAS.
for example, I will install wireguard and as second part zrep for remote Snapshots (Backups) with your setup.
can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
And can you tell me the path, where the config files are, I have to editing for persistent changes?
Or in other words: where are the used .../etc/ directory in which I can editing
For one more example: where, on which path I can editing the sshd_config file on command line for persistent changes after a reboot?

regards,
gerdpeter
please do a full install instead, and you have no problem to install packages .

And remember, some packages can broke your XigmaNAS install, so prefeed way is to install on a jail.
XigmaNAS 12.3.0.4.8919 (ZFSonRoot) on Intel(R) Xeon(R) CPU L5640 @ 2.27GHz 36GB of ECC RAM, 12x3TB disk in 1 vdev = Vpool = 36TB Raw size , so 29TB usable size (I Have other NAS as Backup)

Wiki
Last changes
HP T510
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#4

Post by jamaroney »

I'll repeat myself yet again for the disbelievers and others who simply WANT me to be wrong:

I HAVE DONE THIS FOR CLOSE TO 10 YEARS WITHOUT ANY PROBLEMS WHATSOEVER.

and

Should there ever be any problems (and I've never experienced ANY), all one has to do is disable the command script items and reboot for a "clean" install.
All packages previously installed will remain intact on the data drive, but simply not started, so no data or configs are lost.
Last edited by jamaroney on Jul 14th, '23, 06:43, edited 7 times in total.
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#5

Post by jamaroney »

Can I use the default FreeBSD commands for installing package "pkg install <package>" an not the: "pkg_add -r <package>" command?
YES - do the install exactly as in a full install.

And can you tell me the path, where the config files are, I have to editing for persistent changes?
Your config files are in the same place as a full install. But in an embedded install, the unionfs_mount to another drive keeps them persistent.
benoliver999
New User
New User
Posts: 1
Joined: Sep 15th, '23, 03:06
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#6

Post by benoliver999 »

I am using this method with some success, thanks for the guide! I'm pretty hands-off with the server and was going round in circles trying to set up jails.

One thing: I'm trying to install mkvtoolnix to remux some discs I've ripped. It works fine, but doesn't survive a reboot. I get

Code: Select all

Shared object "libQt5Core.so.5" not found, required by "mkvmerge"
Do you have any idea what could be causing this?
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#7

Post by jamaroney »

My guess is that the pkg install is creating the file somewhere other than /usr/local/, which is why it isn't surviving reboot. But I've installed many packages, and have never encountered this before.

If you're able to get mkvtoolnix running again, do a search for "libQt5Core.so.5" to find where it's located.
jamaroney
Advanced User
Advanced User
Posts: 176
Joined: Dec 13th, '20, 02:55
Status: Offline

Re: [HOW TO] Install pkgs in an embedded install

#8

Post by jamaroney »

To test things, I installed the pkg mkvtoolnix (and its many dependencies), and "libQt5Core.so.5" is right where it should be, in folder /usr/local/lib/Qt5/, so unless there's a script within mkvtoolnix telling it to do funky things to such files, there's no reason it wouldn't survive a reboot.

Please verify that the file is there - if so, then your problem lies elsewhere, and not in the fact that it's an embedded install.

Saturday, March 18, 2023

Changing Default Web Browser in Lubuntu

 After much frustration trying unsuccessfully the normal ways of setting a browser (brave-browser) as the default in Lubuntu, I found that printfgabriel posted the work-around:

Configuring x-www-browser in Alternatives, XDG’s default-web-browser and Lubuntu’s default internet browser application in Session Settings is not enough. Turns out Chrome looks at /etc/xdg/xdg-Lubuntu/mimeapps.list and you have to edit it as sudo and replace the instances of the “firefox” string with your browser of choice.

I replaced all instances of "firefox" with "brave-browser" and that worked. Whew!

Monday, April 11, 2022

Using SCP with Filenames That Contain Special Characters

 Are you having trouble using scp to transfer files that have special characters? The error looks something like this:

syntax error near unexpected token `('

Your next move is to try escaping those special characters with a '\' preceding each one. But you still may get the same error.

Then you may try to put the filename and path in single quotes. Alas, the error persists. After some searching on the interwebs, you find that nested quotation marks may be the answer. Something like this:

scp joe@remote.server.com:"'/home/joe/Documents/This is a really stupid way to name files.txt'"

Now you may get the following error because of security updates to software that added a check for nefarious commands.

protocol error: filename does not match request

However, along with the update, there was added a '-T' parameter to scp command to work around this added check, for use in cases where it is preventing WANTED files from being transferred.

So a nested quote with -T will do the trick, thusly:

scp -T joe@remote.server.com:"'/home/joe/Documents/This is a really stupid way to name files.txt'"

Tuesday, December 8, 2020

Stray Voltage in the Stock Water Trough

 After about 20 years of being a horse owner, something strange happened to me recently. This winter, a day after hooking up the de-icer for the horse water trough, I saw the horses standing at the trough licking their lips with desire but not drinking. After a while longer, they were still there. I've used the simple submerged type of de-icing water trough heaters for many years, and they have gone bad and stopped heating, but I've never had a problem with them introducing voltage to the water trough. In fact, that thought never even occurred to me until this happened.

I tested the water for voltage with my multi-meter by stabbing one probe into the wet soil and the other into the water. It read about 1.5 volts. That's isn't a lot of electricity, but it's certainly enough to make the horses stop drinking... obviously.

I tested the outdoor receptacle and got some strange reading of apparent stray voltage. I decided to replace the receptacle, thinking maybe it was bad after being exposed to extreme temperatures and humidity for 14 years. The new receptacle didn't improve anything. Then I examined the circuit all the way back to the panel in the house... everything looked good. I called the manufacturer of the de-icer and was practically cut-off mid-sentence when I mentioned that I have an electric fence. I was told to drive a grounding rod into the earth near the water trough and connect a copper wire to it and then connect it to the trough to ground it. The company representative promised the electricity in the water would drain through that and everything would be fine. I did just that, and it changed nothing.

I tested the heater in a bucket of water using other receptacles that were actually installed at the house. Same 1.5 volts going through the water. That eliminated the theory that the cause was some anomaly with the fence/trough receptacle.

After much research on internet forums, I learned that 1) I'm not the only one to experience this and 2) it seems to baffle a lot of folks, from owners of one or two horses all the way to big dairy operations. I read all kinds of complicated proposed solutions from building an "equipotential pad" on which the animals must stand to drink (expensive & complicated) to having the utility company install a "neutral isolation" device. I came across one forum post that sounded almost too simple to be true, but had a ring of logic to it. This person originally found this tip in the 10/2014 post on the Henry Milker blog, which is no longer active, but at the time of this post can be found in the way back machine here. The solution? Plug the three-pronged de-icer into a 3-prong-to-2-prong electrical plug adapter with a metal grounding tab. Connect the grounding tab on the adapter to a copper wire that is connected to the grounding rod that is driven into the ground near the water trough. Plug it all in. The grounding wire going directly to the grounding rod provides a strong and close connection to earth ground right where it is needed. I can't provide any more technical explanation than that. In fact, a professional electrician friend of mine struggled to explain a lot of this whole problem. The important thing is that it works. When I tested the water with the de-icer installed this way, the voltage in the water was zero.

If you can shed light on any of this from an authoritative, technical point of view, please do. Meanwhile, I won't have to break ice this winter. Now I just need to convince the horses that it's safe to drink the water again.

Tuesday, September 1, 2020

Install Windows 10 after Ubuntu / Linux


When I rebooted, Windows 10 was not on the Grub menu, but that was an easy fix. After booting into Ubuntu, simply use a terminal to run "os-prober" which should find Windows 10, then run "update-grub"
 
-----

Install Windows after Ubuntu/Linux

As you know, the most common, and probably the most recommended way of dual booting Ubuntu and Windows is to install Windows first and then Ubuntu. But what if you want to install Windows second on a computer with Ubuntu/Linux already installed?

This guide assumes that you have Ubuntu already installed on your system, and will guide you through installing Windows 7/8/8.1/10 when you have a working Ubuntu/Linux partition.

The problem here is that the Windows installer tends to overwrite Grub bootloader or whatever making only the Windows installation bootable. But the good news is that your Linux partition is untouched, including the original bootloader and the other Grub configurations. So restoring or recovering Grub is just a matter of running a few commands on the terminal.

Install Windows on a Separate Partition

If your Ubuntu installation occupies the entire hard drive, to be able to create free space for the Windows installation, you may have to shrink an existing partition using a disk utility like GParted. GParted comes preinstalled on Ubuntu LiveCD.

Boot into a Windows installation media and complete the installation

Once installed, because of Microsoft’s false assumption that there are no non-Microsoft operating systems you’ll be taken straight into Windows.

Restore / Reinstall Grub 2 with a Ubuntu Live Media

  1. Now boot into a Ubuntu Live/USB or CD.
  2. Open a terminal. (Ctrl + Alt + t)
  3. Use a command like lsblk, blkid or GParted to identify your Linux root, and boot partitions. You may or may not have a separate boot partition.
  4. Find Linux Partitions

     $ lsblk -f
     NAME                  FSTYPE     MOUNTPOINT
     sda                                                                            
     ├─sda1                ext4       /boot
     ├─sda2                ext4       /
     ├─sda3                ntfs       
    

    In my case,

    /dev/sda1 is the Linux boot partition

    /dev/sda2 is the Linux root partition

    /dev/sda3 is the Windows partition

  5. Setup a Chroot Environment

    Now we’ll be repairing our old grub bootloader in a chroot environment.

    Mount your root filesystem under /mnt

     sudo mount /dev/sda2 /mnt/
    

    If you’re using BTRFS (New Linux B-Tree File System) instead of EXT4, the above command should change to sudo mount -t btrfs -o subvol=@ /dev/sda2 /mnt/

    Run ls /mnt to verify that the correct partition is mounted. It should list the content of your root.

     $ ls /mnt
     bin   dev  home        lib64       media  opt   root  sbin  sys  usr  vmlinuz
     boot  etc  initrd.img  lost+found  mnt    proc  run   srv   tmp  var  vmlinuz.old
    

    Mount the boot partition. (SKIP if you don’t have a separate boot partition)

     sudo mount /dev/sda1 /mnt/boot
    

    If you have an EFI partition you should mount that also. It is basically a small FAT32 partition around 100mb. Please refer to the section on how to reinstall Grub in an EFI system.

    Bind these directories, so grub can detect other operating systems, like so.

     sudo mount --bind /dev /mnt/dev
     sudo mount --bind /dev/pts /mnt/dev/pts
     sudo mount --bind /proc /mnt/proc
     sudo mount --bind /sys /mnt/sys
    

    Let’s chroot into our existing Ubuntu/Linux system on the hard disk.

     sudo chroot /mnt
    
  6. Reinstall Grub.

     grub-install /dev/sda (specify the disk `/dev/sdX`, not `/dev/sdaX`)
     grub-install --recheck /dev/sda
     exit
    

    Once you’ve successfully reinstalled Grub, restart the computer. On next reboot Grub will be the default bootloader and you’ll be presented with a list of operating systems to choose from (Ubuntu and Windows, of course).

Reinstalling Grub in an EFI Based System

Reinstalling Grub bootloader in a UEFI based computer is no different from installing Grub on a legacy BIOS system. Here we also mount the EFI partition before chrooting, as we’d do with the boot partition.

Case #2
$ lsblk -f
NAME                  FSTYPE     MOUNTPOINT
sda                                                                            
├─sda1                ext4       /boot/efi
├─sda2                ext4       /
├─sda3                ntfs       

Here /dev/sda1 is the EFI partition and /dev/sda2 is the root.

sudo mount /dev/sda2 /mnt/

Mount EFI partition under ‘/mnt/boot/efi’

sudo mount /dev/sda1 /mnt/boot/efi

Forgetting to do so will result in errors being thrown while trying to execute grub-install: “/boot/efi does not appear to be the EFI partition”

As shown previously, bind the /dev, /dev/pts, /proc, and /sys directories.

Finally,

sudo chroot /mnt
grub-install
exit

In some rare occasions, Grub might not detect Windows. In that case just login to Ubuntu (installed one on the disk, not the bootable Live USB/CD) and execute,

sudo os-prober
sudo update-grub
sudo reboot

Happy Dual-Booting!


 

Wednesday, August 5, 2020

Making an Image of a Disk (or Partition) Using dd WITHOUT Including Extra Empty Space

This write-up helped me a lot with this.

I have a Raspberry Pi that gets a lot of use as a media control station, and as a result, it periodically needs to be re-imaged. This is a lot faster and easier than re-installing everything from scratch and then setting up Mythtv, all my network settings, etc. So, a long time ago I used dd in Linux to make an image of a fresh install with all the basic things setup for my network, including Mythtv client. That install was on a microSD card that was 8 GB, so the resulting .img file was 8 GB even though at least half of that was unused. When I tried to write that image to a different 8 GB microSD card, it failed because of insufficient space. If one card had even one bit less (for reasons I do not know), it would be too small. What I needed was to get rid of all that empty space in the .img file.

Write the .img file to a larger memory card that will hold all 8 GB. Then use GParted to shrink the partition that has all the unused space in it to a reasonable size. Now, use sudo fdisk -l -u=sectors to find the number of the last sector used by the last partition.

Disk /dev/sdb: 7.4 GiB, 7960788992 bytes, 15548416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00ed4

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdb1         4096   147455   143360   70M  c W95 FAT32 (LBA)
/dev/sdb2       147456  6348892  6421187  7.4G 83 Linux
 

Make a new .img file from this memory card with the smaller partition. The key here is the parameter "count."

sudo dd if=/dev/mmcblk0 of=NO_WASTED_SPACE.img bs=512 count=6348892

where the count value is the number of the last sector of the last partition you saw in the fdisk listing. That makes dd stop reading anything after the last partition, namely all that extra empty space. Now you have an image file that is just the size you need, it won't take up extra storage space because of the unused space in the image, and when you write the image to a disk (memory card), you won't get errors and failure because the image is larger than the card you are trying to write it on.
 
Something important I learned, the sector size on the block device (memory card) shown via fdisk was 512. When telling it the sector to stop on, you have to make sure the input and output blocksize (bs=) is the same as the sector size, in this case 512. The first time I tried it, the image file was turning out to be huge because I use a block size of 4096 (bs=4096), but the number I used as the last sector to read (count=) was based on a sector size (block size) of 512. When I switched bs back to 512 (I believe the default), it worked fine. Now my image file is 3 GB instead of 8 GB, less space sitting on my hard disk drive, and it won't be picky writing to any device with 3 GB space or more.

Thursday, October 29, 2015

Print Enlarged Documents & Images with PDFPoster or PDFImages & Rasterbator.net

Two convenient methods of printing enlarged document or images follow:

1. PDFPoster is a command line utility for linux that will enlarge an input PDF document so that you can print it out on many sheets of paper.

2. PDFImages is a command line utility for linux that extracts the images from a PDF document. Then, you can use that image at http://Rasterbator.net, which will configure the image for printing an enlarged version on many sheets of paper. Pretty slick.

Tuesday, August 4, 2015

Nicholson bench... more progress


Today, I had to redo the top braces that go horizontally between the tops of each end/leg assembly because I mistakenly made them the wrong dimensions (1/4" short) based on the original design before we made a few minor modifications (changing it to be a split-top bench).

After doing that, I milled the "under-aprons" which beef up the thickness of both aprons. Each got planed on the face that was crowning, then straight-lined and ripped. I clamped one apron in place onto the legs and marked the under-apron to fit between the legs.

Next, we glued up each of the two under-aprons to its corresponding apron using PVA glue (Titebond original). We had to work fast. I spread the glue onto the planed side of the under-apron, flipped it into place, put two clamps on, adjusted it to be flush on what will be the top edge, & drove two screws in to hold the alignment. Then I repeated the process on the other pair.

After gluing and screwing both sets, we stacked them up and began putting clamps on. While that was setting up, I glued and screwed the new top braces into the leg assemblies with Little Brown glue.

Next will be getting the top pieces planed, ripped, and cut to length, then glued up to their "under-top" pieces like I did with the under-aprons.

Clamping apron to legs to check fit and to use for marking under-aprons

Clamping apron to legs to check fit and to use for marking under-aprons
Clamping glued aprons and under-aprons
Leg assemblies with top brace installed

Thursday, July 30, 2015

Nicholson Bench Leg Assembly

We did last-minute tune-up work on the legs today, like cleaning up the notches in the legs where the lower cross braces attach. I had to power plane the cross braces a bit more to make them fit flush, and one of the leg ledges where the apron sits required a little chisel work to trim a little more. Then we "sized the joint" by smearing some glue (in this case, warmed hide glue... specifically Little Brown Glue) on the joints and letting it sit for several minutes. This helps to seal the areas of end grain so they won't absorb all the glue and "starve" the joint. After it partially dried, we applied additional glue to all those surfaces, then assembled and clamped the parts. In this case, screws were added where called for.

I also finished drilling the 19 dog holes in the remaining apron. We made good progress today.




Thursday, July 23, 2015

New Work Apron

When I began woodworking, the only apron I had was one from the kitchen. It wasn't the greatest kitchen apron, but it certainly wasn't well-suited for the shop. It was better than nothing... barely, so that's what I've been wearing. By far, my biggest complaint is the strap that goes around my neck to hold the apron up. I can't stand the feeling of a strap pulling on the back of my neck all the time. It's a serious distraction and drives me nuts. The kitchen apron also lacked appropriate pockets and strength for a shop apron.

I've seen handmade shop aprons from a maker in Texas that seem well made with waxed (or non-waxed) canvas, leather straps that cross in the back and do not go around the neck, and custom sized pockets to fit the phone and other utensils that the wearer wants to always have at hand. The price of these aprons reflected their quality and kept me from joining in on the quality apron fun... until I proposed a joint project with my wife.

She bought some canvas at a discount fabric store, and we bought some leather (buffalo) straps, a leather hole punch, slides, D-rings, copper rivets, and a rivet set thingy.

After discovering that her machine would not work with a double needle (which would've made the double stitching easier and more precise), she resolved to do the work with a single needle. She laid out the apron according to my body and how I wanted it to fit and proceeded to sew the edges and the pockets.

Upon completion of the sewing, I cut the leather to length, installed the slides for adjustability, put the D-rings in the appropriate places, and attached the leather with copper rivets. The apron is wearable now, but I plan to put copper rivets at all pocket corners and stress points for durability. They're (rivets) surprisingly easy to install. The apron that inspired mine has a separate strap that goes horizontally from one side of the apron to the other with a clip. It's a butt strap. I'm not certain I will put this on mine because, so far, it seems superfluous.

Oh, I almost forgot to mention... I also got a bar of paraffin wax and rubbed it across the outside of the canvas, then melted it into the fabric with my heat gun (it's good for something besides reflowing solder on my PS3). Not only does the wax make the apron water, dirt, & glue resistant, it also gives it a "broken in" look that I like.
Added the slides for adjusting the shoulder straps

Shoulder straps riveted to slides

Punching holes in the leather straps (the ones that hold the D-rings on the top of the apron)

Setting the copper rivets
Setting the copper rivets

Cutting excess rivet shank

Finished rivets after blunting the cut shank with the rivet tool

Comfy, durable, & STYLISH apron ready to catch some wood shavings and dust

Tuesday, July 21, 2015

Design modifications, legs, hardware

Since my last post, we've made some modifications to the design in SketchUp such as increasing the height. We're also going to use "two-by" material for the apron backer instead of "one-by." The bench will now be a split-top with 2" between the two top boards, necessitating eight cap screw holes in the top instead of six (two on each end of each top board).

The glue on the legs was given too much time to dry before cleaning up, so the messy edges were shaved on the rip saw. I used a stacked dado set on the table saw to make the notches in the legs for the lower cross braces. I cut and milled the upper ends, lower cross braces, and top leg braces to size.

I have acquired the cap screws, washers, and nut plates, and I've also purchased four Gramercy Holdfasts.

The legs need to be assembled next. I have dog holes drilled in one apron, but many holes are left to drill in the other apron and the two top boards.






Sunday, July 5, 2015

Building My Knockdown Nicholson Workbench

My first workbench will be a variation of the Nicholson bench that can easily be disassembled and stored flat if necessary. I'm using the plans from Lost Art Press. You can read and see more about this bench here.

I've chosen to use fir from my local home center... Seven 2x12x16s. So far, I've identified which boards will be used for what pieces (based on characteristics of the wood like checks and knots that will have structural and/or visual implications), rough cut, milled and glued up the legs, and begun drilling the dog holes in the front and back aprons.

Legs glued up:
 

An apron getting dog holes drilled:
 

Thursday, August 7, 2014

Get Rid of Stupid Disappearing Scrollbars in Ubuntu 12.04+

I have no idea how anyone thought disappearing scrollbars were a good idea, but I don't. Here's how to make the disappearing scrollbars disappear. In the terminal, type:

gsettings set com.canonical.desktop.interface scrollbar-mode normal

Thursday, February 28, 2013

Rescue Asus TF101

I used PERI tool to rescue my ailing Asus TF101 from being stuck at the setup screen. Yippee.

Friday, September 28, 2012

Asus Transformer TF101 clockworkmod recovery

ROM Manager won't boot into recovery on my TF101, so to boot into Clockworkmod Recovery, hold power and volume down buttons. The internal storage (/sdcard) wouldn't mount with RM version 6, so I used ROM Manager version 5.8.3.4. First, backup with Titanium Backup and backup ROM... ensuring both backups are on the removable SD card.

Saturday, August 11, 2012

Extract Image From Video Using FFMPEG

ffmpeg -i video.flv -an -s 320×240 -ss 00:00:02 -r 1 -y %d.jpg

-s 320×240 specifies the frame size of the output (Default is the size of the original video.)

-ss 00:00:02 Seek to given time position in seconds.

Thursday, February 2, 2012

Change FourCC setting in Linux Using Mencoder

Sometimes an AVI video file will not play on my PS3, resulting in a "data type not supported" error message. I finally figured out the problem this time. It wasn't audio codec, and it wasn't the weird fps it had; It was a FourCC setting that was incompatible. I tried using cfourcc to do this, but it did not result in playable AVI files. So, I tried mencoder and it worked fine, all without remuxing.

mencoder <input-file> -oac copy -ovc copy -ffourcc XVID -o <output-file>

-oac stands for "output audio codec"
-ovc stands for "output video codec"
-ffourcc is for forcing the changing of the fourcc setting. The non-playable files had a FourCC setting of FMP4. I changed it to XVID, which made them playable.
-o is for output file

Example:

mencoder The.Big.Show.S01E03.avi -oac copy -ovc copy -ffourcc XVID -o The.Big.Show.S01E03-XVID.avi

Tuesday, December 13, 2011

MKVmerge Batch Commands, Remove Tracks

To remove a particular track from an MKV file, the following notation should work:

mkvmerge -o output.mkv -atrack !2 input.mkv 

But I wasn't able to get the exclamation mart (meaning DON'T copy this track) to work. For some reason I kept getting "track not found" message. This should work with --vtrack or --strack (subtitle) or whatever.

Fortunately, I wanted to remove all the subtitles from my files, so I used the following instead:

mkvmerge -o output.mkv --no-subtitles input.mkv

If you only want to do this on one file, the mkvmergeGUI is fast and easy, but if you want to do it on a whole directory of files, you want to use this at the command line in a "For Loop," like so:

for i in *mkv; do mkvmerge -o ./newfiles/`basename "$i" mkv`.mkv --no-subtitles "$i" ; done

This will remove all subtitle tracks from all MKV files in the current folder, then put the new files in a previously-created sub-folder named "newfiles".

Thursday, December 8, 2011

More Video Processing

HOWTO: converting mkv files to play on the PS3

(Originally found at Julien Simon's website.)

There's no doubt that Matroska is a great open source container that can store anything (and the kitchen sink too). Unfortunately, as of today, it's not well supported by consumer devices. So, unless you decide to use your computer for viewing, you need to convert mkv files into something that your device can play.

The purpose of this article is to show how to do that for the PS3. It may very well work on other devices, but don't blame if it doesn't :)

This article is a work in progress. Are covered so far:
  • Example #1: (mkv file, H.264, AC3) --> (MPEG-2 TS file, H.264, AC3)
  • Example #2: (mkv file, H.264, AC3) --> (MPEG-2 PS file, H.264, AC3)
  • Example #3: (mkv file, H.264, AC3) --> (MP4 file, H.264, AAC)
  • Example #4: (mkv file, H.264 level 5.1, DTS 5.1) --> (MP4 file, H.264 level 4.1, AAC)
  • Example #5: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3)
  • Example #6: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3 5.1)
  • Example #7: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MP4 file, H.264 video, AAC)
1) Pre-requisites

Before we start, let's prepare our toolbox:
  • mediainfo : the best tool to learn everything there is know on the true nature of your media files. I showed you how to build it in a previous article.
  • mkvtoolnix : a collection of tools to inspect, extract and build mkv files.

    ubuntu% sudo apt-get install mkvtoolnix
  • ffmpeg : the best general-purpose audio & video transcoding tool. I also showed you how to build it in a previous article.
  • MP4Box : a nice MP4 manipulation tool, which we'll use for multiplexing.

    ubuntu% sudo apt-get install gpac
  • tsMuxer : another great tool for multiplexing. You can download a pre-compiled version here. Just copy the tsMuxer binary in /usr/local/bin.
2) Audio & video formats supported by the PS3
Let's take at a look at what the PS3 actually supports, That way, we will avoid any unnecessary transcoding. Remember that the content we're dealing has already been compressed, so any additional processing would certainly degrade quality. Also, video encoding takes a LOT of time, especially with HD files...

According to the online manual, the PS3 supports:
  • MPEG-1 video, with MPEG-1 layer 2 audio
  • MP4 container : H.264/MPEG-4 AVC High Profile video, with AAC LC audio
  • MPEG-2 PS container (aka VOB file): MPEG-2 video, with either one of these audio formats : MPEG-2 Audio Layer 2, AAC LC, AC3 (Dolby Digital), LPCM audio
  • MPEG-2 TS container :
    • MPEG-2 video, with either one of these audio formats : MPEG-2 Audio Layer 2, AAC LC, AC3 (Dolby Digital)
    • H.264/MPEG-4 AVC video, with AAC LC audio
  • AVCHD, AVI, DivX and VC-1 (WMV), which are less likely to be encountered in mkv files.
So what does this mean? Well, here are a few rules:
  • Obviously, the PS3 does not support the mkv container. Any input file in this format must be at least remuxed.
  • The PS3 does support DTS audio, but only when played from actual disks, i.e. not from files. Thus, any input stream in this format must be transcoded.
  • MP3 audio, Vorbis audio and so on are not supported and must be transcoded.
  • If you want AC3 audio, you must use a MPEG-2 TS/PS container.
  • If you want to stick with the MP4 container, you really have no options but H.264 video and AAC audio.
3) Getting things done
Now that we know what is supported and what isn't, we can pick the right solution, i.e. preserve the quality of the input file and save time.

The approach will always be the same:
  • inspect the mkv file and identify the nature of its streams,
  • extract the video stream and the audio stream (possibly among several available ones)
  • depending on what your audio setup supports, select a format for the audio output (AC3 will do you no good if you don't have a Dolby Digital setup)
  • based on the previous item and on what the PS3 supports, transcode the audio and/or video stream
  • remux the audio and the video stream into a container supported by the PS3
  • play the file :)
Let's look at real-life examples and sort them out!

4) Example #1: (mkv file, H.264, AC3) --> (MPEG-2 TS file, H.264, AC3)
Let's look at our input file with mediainfo (edited for brevity):

ubuntu% mediainfo video.mkv
General
Complete name : video.mkv
Format : Matroska
File size : 1.09 GiB
Duration : 41mn 48s
Overall bit rate : 3 740 Kbps

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Codec ID : V_MPEG4/ISO/AVC
Width : 1 280 pixels
Height : 720 pixels
Frame rate : 23.976 fps

Audio
Format : AC-3
Format/Info : Audio Coding 3
Codec ID : A_AC3
Channel(s) : 6 channels
Channel positions : Front: L C R, Surround: L R, LFE

You can also use mkvinfo (part of the mkvtoolnix package):

ubuntu% mkvinfo video.mkv
| + Track number: 1
| + Track type: video
| + Codec ID: V_MPEG4/ISO/AVC
| + Video track
| + Pixel width: 1280
| + Pixel height: 720

| + Track number: 2
| + Track type: audio
| + Codec ID: A_AC3
So what is this file? It's a 2-track mkv file: track 1 is H.264 video (1280x720), track 2 is AC3 6-channel audio. Both formats are supported by the PS3, and since they're high quality it would be a shame to transcode them. Let's try a simple remuxing into... no, not MP4 because it doesn't support AC3: we have to use MPEG-2 TS.

This is where tsMuxer comes in. The good thing about it is that we don't have to extract the individual streams, this will be handled for us.

Although tsMuxer is a command-line tool, it can only read its parameters from a file (let's call it tsmuxer.meta):
MUXOPT --no-pcr-on-video-pid --new-audio-pes --vbr
V_MPEG4/ISO/AVC, video.mkv, level=4.1, insertSEI, contSPS, track=1, lang=eng
A_AC3, video.mkv, track=2, lang=eng
Ignore the first line and look at the next two. On each of them, we're describing a stream and telling tsMuxer what format it is and where to find it (file name and track number). Don't worry, this information is dumped by mediainfo or mkvinfo (go back and check), so you shouldn't have any problem writing the file :)

Now, we're ready to remux:

ubuntu% tsMuxeR tsmuxer.meta video.m2tsoutput removed
Mux successful complete.
Muxing time: 5 min 25 sec

That's all: this file plays fine on the PS3 (I'm streaming it right now using mediatomb). Total time: about 10 minutes. And we preserved the original quality of the input file.

5) Example #2: (mkv file, H.264, AC3) --> (MPEG-2 PS file, H.264, AC3)
Here, we'll use the same input file as example #1, but we'll remux it to MPEG2-PS (aka the VOB format). This is also interesting if you want to do without tsMuxer.
Wouldn't it be nice to be able to just do this?

ubuntu% ffmpeg -i video.mkv -acodec copy -vcodec copy -r 23.976 -f vob video.mpg[NULL @ 0x8076d00]error, non monotone timestamps 15030 >= 7470
av_interleaved_write_frame(): Error while opening file


As you can see, it doesn't work. Not sure why... Let's use a two-step solution instead. First, let's extract the streams from the mkv file:

ubuntu% mkvextract tracks video.mkv 1:video.h264 2:audio.ac3
Now, let's remux the streams into a MPEG2-PS container:

ubuntu% ffmpeg -i video.h264 -i audio.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -r 23.976 -f vob video.mpg
That's it. This solution is a little bit longer than the previous one, but it works and doesn't require you to write the tsMuxer meta file :)
6) Example #3: (mkv file, H.264, AC3) --> (MP4 file, H.264, AAC)
Here, we'll also use the same input file as example #1. For whatever reason, we need to use the MP4 container (maybe to be able to play that file on another device). This means that we must also drop AC3 in favor of AAC.

We have already inspected the file, so let's extract the streams:

ubuntu% mkvextract tracks video.mkv 1:video.h264 2:audio.ac3

Then, we need to transcode the audio stream to AAC (let's use high-quality VBR):

ubuntu% ffmpeg -i audio.ac3 -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac

Now, we'll use MP4Box to remux the initial video stream and the new audio stream into an MP4 container:

ubuntu% MP4Box -new -add audio.aac -add video.h264 -fps 23.976 video.mp4AAC import - sample rate 44100 - MPEG-4 audio - 2 channels
AVC-H264 import - frame size 1280 x 720 at 23.976 FPS
Import results: 60146 samples - Slices: 1027 I 36894 P 22225 B - 1 SEI - 968 IDR
Stream uses B-slice references - max frame delay 2
Saving video-ps3.mp4: 0.500 secs Interleaving


That's it :)

7) Example #4: (mkv file, H.264 level 5.1, DTS 5.1) --> (MP4 file, H.264 level 4.1, AAC)
Here, we'll use a new sample file (for the sake of brevity, I will just display the relevant lines):
ubuntu% mediainfo video2.mkv
General
Complete name                    : video2.mkv
Format                           : Matroska

Video
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L5.1
Muxing mode                      : Container profile=Unknown@5.1
Codec ID                         : V_MPEG4/ISO/AVC
Frame rate                       : 23.976 fps

Audio
Format                           : DTS
Format/Info                      : Digital Theater Systems
Codec ID                         : A_DTS

ubuntu% mkvinfo video2.mkv

| + A track
|  + Track number: 1
|  + Track type: video
|  + Codec ID: V_MPEG4/ISO/AVC

| + A track
|  + Track number: 2
|  + Track type: audio
|  + Codec ID: A_DTS
OK, so track 1 is H.264 video and track 2 is DTS audio. Let's extract them:

ubuntu% mkvextract tracks video2.mkv 1:video.h264 2:audio.dts
According to what I've said earlier, H.264 is supported, so all we have to do is to transcode the audio... right?

Wrong :) Take another look at the video stream: format profile is "High @ Level 5.1" (note: this has nothing to do with 5.1 audio!). Unfortunately, the PS3 cannot play anything above Level 4.1, so this video stream needs to be fixed. Does this mean we have to reencode it and lose quality?

Nope :) We're going to change the level right in the binary file. For this purpose, you need an hexadecimal editor. I recommend ghex2, which can easily be installed with 'sudo apt-get install ghex'.

just go 'ghex2 video.h264' and look at the first 8 bytes of the file: they should read '00 00 00 01 67 64 00 33'. Hmm... 0x33 in decimal is '51', which means 'level 5.1' : let's change this byte to 0x29 ('41' in decimal) to 'downgrade' the video to level 4.1 and save the file.

Now, let's take care of the audio stream. As above, we'll use high quality VBR:

ubuntu% ffmpeg -i audio.dts -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac

The last operation is to remux the initial video stream and the new audio stream into an MP4 container:

ubuntu% MP4Box -new -add audio.aac -add video.x264 -fps 23.976 video.mp4
That's it. I'm not sure the level hack will always work, but it did in this case and we avoided any video transcoding.

8) Example #5: (mkv file, MPEG-2 video, DTS 5.1+ AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3)
Let's take a look at this one (output edited for brevity)
ubuntu% mediainfo video3.mkv 
General
Complete name                    : video3.mkv
Format                           : Matroska
File size                        : 73.9 MiB
Duration                         : 50s 240ms
Overall bit rate                 : 12.3 Mbps

Video
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@High
Format settings, Matrix          : Default
Codec ID                         : V_MPEG2
Width                            : 1 920 pixels
Height                           : 1 080 pixels

Audio #1
Format                           : DTS
Format/Info                      : Digital Theater Systems
Codec ID                         : A_DTS
Language                         : Russian

Audio #2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Codec ID                         : A_AC3
Language                         : English

ubuntu% mkvinfo video3.mkv 
|  + Track number: 1
|  + Track type: video
|  + Codec ID: V_MPEG2

|  + Track number: 2
|  + Track type: audio
|  + Codec ID: A_DTS

|  + Track number: 3
|  + Track type: audio
|  + Codec ID: A_AC3
Track #1 is full HD MPEG-2 video, track #2 is DTS audio in Russian and track #3 is AC3 audio in English. Let's extract them:

ubuntu% mkvextract tracks video3.mkv 1:video.mpeg 2:audio.dts 3:audio.ac3

If like me you don't speak Russian, you'll pick track #3 :) If you do speak Russian, please refer to the next example for DTS-->AC3 conversion.

One option would be to remux tracks #1 and #3 into an MPEG-2 TS container (like we did in example #1). Unfortunately, tsMuxer chokes on the video stream ('Can't detect stream type').

Let's try to remux in an MPEG-2 PS container. For this, we'll use ffmpeg but without any transcoding:

ubuntu% ffmpeg -i video.mpeg -i audio.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -f vob video.mpg
That's it!

9) Example #6: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3 5.1)
We'll use the same input file as in example #5, but this time we'll use convert the DTS audio stream and convert it to AC3.

Let's take another look at the DTS stream:

ubuntu% mediainfo audio.dts
General
Complete name : audio.dts
Format : DTS
Format/Info : Digital Theater Systems
File size : 9.04 MiB
Duration : 49s 381ms
Overall bit rate : 1 536 Kbps

Audio
Format : DTS
Format/Info : Digital Theater Systems
Bit rate mode : Constant
Bit rate : 1 536 Kbps
Channel(s) : 6 channels
Channel positions : Front: L C R, Surround: L R, LFE
Sampling rate : 48.0 KHz
Resolution : 16 bits
Since the DTS stream has 6 channels, we can either transcode it to 2-channel AC3 or 6-channel AC3. Let me show you both:

ubuntu% ffmpeg -i audio.dts -acodec ac3 -ar 48000 -ab 448k -ac 2 audio-2ch.ac3


ubuntu% ffmpeg -i audio.dts -acodec ac3 -ar 48000 -ab 448k -ac 6 audio-6ch.ac3


448 KBit/s is the maximum authorized AC3 bitrate for DVDs. However, with the Playstation 3, we could go up to 640Kbit/s.

Now, let's remux the video and audio streams (I'll use 6-channel AC3 here):

ubuntu% ffmpeg -i video.mpeg -i audio-6ch.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -f vob video.mpg
That's it.

10) Example #7: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MP4 file, H.264 video, AAC)

We'll use the same input file as in example #5. Let's assume that - like in example #3 - we need to use the MP4 container. This means that we must convert the video stream to H.264 and our preferred audio stream (AC3 for this purpose) to AAC.

Please note that this is not the best option! The video compression will be lossy and all the longer that we're dealing with a 1920x1080 video stream. As far as audio is concerning, we're moving from 6-channel AC3 to stereo AAC. So, unless you really need the MP4 container, I strongly suggest that you use the solution described in example #6.

Anyway, let's get started and extract the MPEG-2 and AC3 streams:

ubuntu% mkvextract tracks video3.mkv 1:video.mpeg 3:audio.ac3

Then, let's encode the audio stream to AAC:

ubuntu% ffmpeg -i audio.ac3 -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac

Now, the big one: encoding the MPEG-2 stream to H.264:

ubuntu% ffmpeg -i video.mpeg -f rawvideo - | x264 --level 4.1 --crf 21 --bframes 16 --b-pyramid --ref 4 --mixed-refs --weightb --partitions all --threads 2 -o video.h264 - 1920x1080

Let's explain this line a bit: as you can see, we're using ffmpeg to decode the video stream to raw video and we're piping the output into the x264 encoder. The reason for this is avoid any H.264 default value that could be set by ffmpeg: indeed, piping guarantees that all H.264 flags will be set by x264.

What about the x264 flags? '--level 4.1' sets the H.264 level (the PS3 can't play anything above 4.1, please see example #4 for more information). '--crf 21' select Constant Rate Factor encoding, with default quality. '--bframes 16 --b-pyramid --ref 4 --mixed-refs --weightb --partitions all' are frame options which seem to strike the right balance between quality and encoding time (please see this article for more information). Finally, '--threads 2' will create 2 encoding threads.

FYI, on a dual-core PC @ 1.83 GHz, this command took 8 minutes to encode a 50-second video! Now you understand why you don't want to do this unless you have no other option.

Now, the final step: remuxing into an MP4 container

ubuntu% MP4Box -new -add video.h264 -add audio.aac -fps 25 video.mp4

That's it!