06 Nov 09 String replacement with sed

Sed – stream editor is a powerful tool to manipulate strings. It will take STDIN as well as operating on a file:

The most common usage is to replace text:  echo “this is a test string” | sed s/i/z/g will replace every instance of ‘i’ with a ‘z’:  thzs zs a test strzng

You can delete a particular word with say echo “this is a test string”| sed s/test//g leaving: this is a  string

You can operate on a file with:

echo “this is a test string” >> file; sed -e s/test//g file Leaving: this is a  string

You can also use regular expressions with sed.

Tags: echo, sed, stdin

05 Apr 09 Simple text sorting

On the command line we have a number of powerful tools available to us. I’m going to cover some text sorting methods here.

I have a file called ‘testfile’ within this file is the following:

test:~# cat testfile
line1
line3
abcdefg

test
line9

this is a test
test file

test
(more…)

Tags: awk, cat, grep, sed, sort, text sort, uniq

07 Mar 09 Linux piping and redirecting stdin stderr stdout

We have three relevant streams when dealing with passing data around on the command line. STDIN (0), STDOUT (1) and STDERR (2)

echo “hello” will return “hello” to STDOUT

echo “hello” | sed s/llo/y/g
Returns: ‘hey’

echo “hello” will print “hello” to STDOUT which we pipe to sed’s STDIN. The shell will fork both processes, echo and sed, and create a pipe between one’s STDOUT to the other’s STDIN. A ‘broken pipe’ will occur when one terminates unexpectedly.

strace echo “hello” will print the system calls that the command makes. Lets say I just want to print out open() calls.

strace echo “hello” | grep open does not work. It seems that the grep is ignored.

This is because strace sends it’s output to STDERR and not STDOUT. In this case we must redirect STDERR to STDOUT so grep can pick it up on it’s STDIN.

strace echo “hello” 2>&1 | grep open will work successfully.

What if we want to redirect STDOUT and STDERR to a file? We simply redirect STDOUT to a file and then redirect STDERR to STDOUT.

strace echo “hello” >/tmp/strace.output 2>&1

A nonstandard method of achieving the same by redirecting everything in one go is strace echo “hello” &>/tmp/strace.output however this is not guaranteed to work across all implementations.

* Post edited thanks to observations from Adam Bolte (16/11/09)

Tags: echo, grep, pipe, redirect, sed, stderr, stdin, stdout

05 Oct 08 APNIC Box – Linux on a Mikrotik 532a, Part 3 – Installing Debian, Prebuilt Disk Image

Follow on from 01 Oct 08 APNIC Box – Linux on a Mikrotik 532a, Part 2

The device runs a 2.4.30 kernel on a debian woody (mipsel) environment. If anyone can contribute anything for 2.6.x and debian etch, that would be great.

In this part, I’m going to provide a download link to a prebuilt image which you can write directly onto your own CF card with dd or similar tool. I’m also going to provide a step by step to installing debian yourself without my prebuilt disk image.

Prebuilt image is here:

The prebuilt image also contains a number of scripts and tools that I’ll comment on and come to in later parts.

Now installation instructions:
(more…)

Tags: 532a, apnicbox, apt-get, atftpd, CF, compile, cross compile, Custom Kernel, dd, debian, dhcpd, dhcpd.conf installer, forward, initrd, initrd.gz, iptables, ip_forward, kernel, Linux, linuxrc, masquerade, mikrotik, mipsel, mkfs, mknod, mount, partition, prebuild image, routerboard, routerboard 532a, scripts, sed, tftpd, vmlinux, wget