Author: Phil

Stripping quotes from a file

July 17, 2012 / / 0 Comments

I had a whole lot of large CSV files, totalling about 10.8 GB. All fields were delimited by speech marks (double-quotes). I wanted to load them into PostgreSQL using a COPY statement, but apparently, empty timestamp fields (looking like “”) weren’t of the right shape to be imported directly. I got an error message like this:

ERROR: invalid input syntax for type timestamp with time zone: ""
CONTEXT: COPY rawdata, line 218, column actual_off: ""

So I decided to get rid of all the speech marks. That’s OK, because there are no commas in any of the data … so I don’t know why the source would include them all anyway.

I started off with this:

sed 's/\"//g' data.csv > data-noquotes.csv

but it was terribly slow. After a bit of looking around, I got this:

cat data.csv | tr -d '"' > data-noquotes.csv

which was a whole lot faster.

Serving (and saving) Minecraft

January 2, 2012 / / 0 Comments

So the boys found Minecraft, and had all sorts of fun. They played in single player mode and did great exploits, and then they started looking around for ways to play collaboratively. They had some trouble getting going with the network server thingy, but eventually figured it out and got things under way.

Of course, they ran into trouble almost immediately. Well, not trouble, but inconvenience. Mat’s computer was the most powerful, so it was being the server. All well and good, until he was out when someone else wanted to play in their world. So, I figured I would step in and, being a Dad with some IT skills, set up a Linux VM for Minecraft on my computer. My PC generally doesn’t stop, so that would save them having to share passwords.

With the VM set up, I then transferred their world to the new server. All good, and always accessible. I punched a hole in our firewall, forwarding the port to that VM, and so now their friends are able to play as well. Everyone was happy.

Until. Until that fateful day, when someone gave the server address to someone else, who came in and griefed their world. We had no backup. Oops. We blacklisted the kid, and switched to whitelisting, but the damage was already done. The players started the repair work, and life went on.

Actually, it didn’t. What happened was that I had let Nathan use my PC for the repair work, because it’s really quick. Unfortunately, he accidentally hit the “Sleep” key on my keyboard, and (quite reasonably) the PC went to sleep. My PC doesn’t wake up. If it goes into sleep mode, it’s functionally the same as pressing the restart button, but without the same sense of finality. It provides you with that momentary sense of relief: wow, that was close, I could have hit the reset button instead! But then it won’t wake up. Grr. I had to switch the machine off and then on again, and, sure enough, all the VMs had to boot, and the Minecraft world had some problems. Major problems. The world was pretty much corrupted beyond my ability to repair.

We had a round table conference, and after much discussion, including teleconferencing in one of the remote players, we decided to drop the world and start from scratch. I committed to getting backups working, and we went our happy way.

Lightning struck a second time. Actually, we’re not sure if it was actually lightning, but the power went out. Actually, we are sure. It was toast being pushed down into the toaster that flipped out the breaker on that end of the house on the morning after the griefing, and before I’d set up the backups. I have three UPSs around the place, and it turns out that the one supporting my computer just wasn’t up to the task. Yes, I know I should have tested with that UPS, but that’s the cookie crumbling. Fortunately, no-one was playing at the time, and the minor corruption was able to be repaired. This galvanised me into action. Backups, here we come.

I figured that the server would have to stop for backups, and then be restarted. So the way to do that would be to have the server running as a SysV daemon, stop it at backup time, make the backup, and then restart it.

I scoured around the net for SysV startup scripts for minecraft, and found this one. That link will take you to a newsgroup article with a link to download the script. I downloaded it, started to install it, and then started running into problems. I figured that I would just get the server running as a daemon started by root. Uh uh. This one won’t do it. That made sense when I thought it through, because I don’t know how secure the server program is, and I don’t want to have to su to root when I want to fiddle with it.

OK, so I had a “minecraft” user already set up, and I figured that I would just do it with that user. Nope. The README file for this script says that that particular username is too long. Use “miner”, it says. So I set up the “miner” user, and put the rc.minecraft script into /etc/init.d. Once I’d done that, I needed to get it so that it would start the server at boot. I did this with the following command:

# chkconfig -s rc.minecraft on

I also needed to make some configuration stuff available to the server. According to the doco, the script looks in /etc and /etc/init for a minecraft.conf file. Fine, so I did the following:

# cd /etc/init.d
# ./rc.minecraft dumpconfig > /home/miner/.minecraft.conf
# chown miner.miner /home/miner/.minecraft.conf
# cd ../init
# ln -s /home/miner/.minecraft.conf minecraft.conf

OK, that’s good. Now the server will start when the computer starts, stop when the computer stops, and I can edit ~/.minecraft.conf as miner to change any of the script parameters.

Now what about the backup? As the miner user, I created a script file with the following lines:

#!/bin/bash
/etc/init.d/rc.minecraft stop
tar cfz "/home/miner/backup.`date +%Y%m%d`.tgz" /home/miner/mc-server/
/etc/init.d/rc.minecraft start

Then I edited the miner crontab file ($crontab -e) and added the following line:

0 3 * * * /bin/bash /home/miner/backup.sh

This means that at 3am every morning, the server will be stopped, a backup will be made and put into a tarball named with today’s date, and the server will start back up again. Perfect!

Oh, and I switched off the sleep key on my keyboard. Sigh.

Many thanks are due to Dagmar d’Surreal for his init script.

 

Wooing Your Wife – A Practical Guide

April 14, 2010 / / 1 Comment

Kids, just walk away. This is not for you. Wives, you can read if you want, but really, this is targeted at your husband. Maybe it will help if you know what he’s trying to achieve, but give him a go on his own, and for heaven’s sake, don’t try and tell him how he ought to proceed.

One more starting thought: I am not a marriage counsellor. If you are in an abusive relationship, GET HELP FROM A PERSON, NOT A WEB PAGE. These are just a few thoughts for someone who’d rather restore something he’s losing than quit, from someone who’s been married almost 20 years, and loves his wife.

Here are some basic thoughts for you (the husband), in no particular order.

  • Your marriage was established because you and your wife believed that you wanted to be together forever (assuming you used the same vows I did).
  • The relationship that you have with your wife is the most important human relationship in your life until you die (and I’m going to continue to assume you used similar vows to what I did).
  • Your relationship evolves over time, through engagement, honeymoon, settling, kids, kids as teenagers, and so on.

Let me talk about love for a moment. The church tends to talk about three different types of love: phileo, eros and agape. There is a fourth (storge), but I’m keeping to the other three, because, frankly, I hadn’t heard about the fourth until this morning. Phileo is friendly or brotherly love, and usually manifested within a “friend” relationship. This is (hopefully) the way your relationship with your wife started. Erotic love is physical love, usually manifested in bed. Where these first two are generally conditional upon reciprocation, agape love is what I like to describe as “an unconditional commitment to an imperfect person.” That’s a direct quote from a minister named David Jones. Agape love should be developed over time, and will sustain when the other two seem a bit lacking. All three should be woven together to make a strong, good marriage because commitment without the ability to enjoy it is pretty rough on both partners.

Because of time and experience, the love and behaviours that get you together with your wife are not the same love and behaviours that will sustain a marriage. Quite obviously, the skills required to cope with kids breaking legs, mortgages and retrenchments are very different to those required for getting through a first date.

So, our behaviours change. And surprise, surprise, we find ourselves “falling out of love.” Little things your wife does (or doesn’t do) annoy you. You get accused of not listening, not pulling your weight, letting her down in many ways. And to be honest, there is probably truth in the criticism. But is that enough to destroy a marriage? I should hope not!

I’ve read a lot of John Eldredge’s writing. In “Wild at Heart,” John says that there are three things a man needs: a battle to fight; an adventure to live; and a beauty to rescue. In “Captivating,” John and his wife Stasi say that there are three things a woman longs for: to be romanced; to play an irreplaceable role in a great adventure; and to unveil beauty. Argue with these if you want; however, they seem to resonate with people of both sexes with whom I’ve discussed it.

The problem is that in our western culture, there isn’t much in the way of battles to fight (legally), apart from in boardrooms and on the sports field. By the time you’ve reached a point where reading this article is going to be helpful, you’ve probably realised that boardroom battles are hollow, and your body may not be in a great space for sport. Adventures are hard to come by because we’re too focused on work, or we can’t engage in them because of health and safety concerns. And our wives just don’t need rescuing. I mean, it’s not like there are lions roaming the streets, right?

And your wife’s needs? Well, gosh! You’re married! You don’t need to be all soppy do you? There are no great adventures any more (can’t take too many risks), and beauty is only skin deep. Gee honey, you’re 40! No-one expects you to try any more … it just doesn’t matter!

And that, my brother, is how to push a knife in your wife’s heart. Take away even the hope of romance, adventure and the chance to unveil beauty, and you might as well have ripped out her heart and thrown it onto the freeway.

So, our culture puts us behind the eight-ball. It is up to you to work past these things, and find the battles that you can fight. Find a way to adventure. Rescue your wife, and woo her. Invite her to join you in adventures, and delight in the beauty she unveils. The key is somehow to establish the right conditions so that she will want to participate. And that means helping her to feel loved.

All of the above is big picture stuff, and an overview. So let’s get down to where the rubber hits the road. How do you make your wife feel loved on a day-to-day basis? For this, I turn to Gary Chapman, and “The Five Love Languages.”

Gary Chapman’s thesis is that we all have “love tanks” that need to be kept filled if we are to feel loved, appreciated, and secure. Different people need their tanks filled in different ways, and the main groupings that Dr Chapman has come up with are acts of service, quality time, words of affirmation, physical touch, and gifts. Usually, a person is going to respond very well to one or two of these, while the others don’t do much for them. Someone whose tank is filled by physical touch is going to appreciate a backrub or a massage far more than having you do the dishes or sweep the floor when it’s not your turn.

So, what to do? First, determine your wife’s love language(s). Don’t just ask her … put a bit of time aside, and have a conversation. Ask questions like “what did you most like about our honeymoon? The walk on the beach? When I bought you those flowers?” Be inventive, and listen for key phrases like “I really liked it when you…” or “I really miss how you used to…” LISTEN TO THE ANSWERS. Be prepared to open up a bit yourself. Remember that your wife is not a puzzle to be solved, but a mystery to be explored. There are rooms full of surprising beauty to be discovered, as well as the odd locked closet that she won’t want to show you until she feels safe. There will be fragility there too (just as there is in you), but as you become someone that she trusts again, even that will open up.

The other main way to determine how best to love your wife is to look at the ways she tries to show love to you. When you were last sick or injured, what did she do? Try to give you a foot rub (physical touch)? Make you breakfast in bed (an act of service)? Sit with you (quality time)? Tell you that she was impressed by how you were standing up to the circumstances (words of affirmation)? Buy you a book (a gift)? Odds are, she will try to make you feel loved in the way that she wants to be made to feel loved.

So when you’re pretty sure you’ve got it nailed, start ramping up a little more on that area. Don’t neglect the other love languages … you could be wrong.

Here are a few suggestions:

  • Physical touch: Hold her hand. If you’re sitting together, put your arm around her. Kiss her frequently. Play with her hair. But almost nothing beats a massage. Get some candles going, get some scented oil, put some pillows on the table, and give her a massage. Make sure the air is warm enough, and make sure you have some steps so that she doesn’t have to clamber up ungracefully. Ask for guidance as you go. Harder, softer, up a bit, down a bit. Again, listen to her. She’ll tell you what she needs if you listen. Have a washer in some warm soapy water when you’re done so you can wash off the oil (make sure she’s not allergic to the soap). Have a warm towel ready to dry off her back. Don’t surprise her with it. Make a date, and let her anticipate. And don’t start with the expectation that sex will follow. This is all about her, not about you.
  • Acts of service: when you see her about to take out the garbage, get up and beat her to it. If she normally does the dishes, take over and send her out of the kitchen. Make her breakfast in bed on Saturday or Sunday morning. Take the kids to soccer. Offer to do things for her when it’s going to make a difference to her.
  • Words of affirmation: “Hey honey, dinner was delicious … thanks so much.” “Gosh, that looks good on you.” “I love what you’ve done with your hair.” “The curtains you chose really suit the couch.” “I love the way you handled that situation.” “You are very dear to me.” “I love you.” For heaven’s sake, get this right. Don’t get patronising, don’t be insincere. Just give voice to the appreciation you have for her.
  • Quality time: If she normally does the dishes alone, wipe up while she washes. If she normally takes the kids to soccer, go with her. Make her breakfast, and eat with her. Don’t read the paper over breakfast while she’s there. Find something that you normally do alone that she can join in. Go for the odd walk along the beach together. If you’re putting on a bit of weight, announce that you’re going to start walking around the block. Then do it for a week or so, and then invite her along. “Hey honey, there have been some spectacular sunrises the last couple of mornings, and I’d like to share them with you. Would you like to come along?” Don’t compete with her, walk at her pace, and talk as you stroll.
  • Gifts: This doesn’t have to be expensive stuff. It has to be something that says “I thought of you while I was out, and wanted to show you I care.” Bring home flowers every now and then. The odd box of chocolates (find out if she prefers dark or light, soft centres or hard). I read about a guy who got to the point where he and his wife just weren’t talking. He got a bunch of little gifts, wrapped them, and left one around the house each day as he went to work. As time went on, he put them in slightly more obscure places, but the first thing his wife did after he left in the morning was search for the gift. Apparently, this saved their marriage.

Remember that this is not about you. This is all about her to start with. You don’t want to start complaining about how she’s treated you while you’re rubbing her back. Seriously. Don’t even complain about how tough life is at work while you’re wiping the dishes. Man up! This is all about her. Remember that I said above that our culture makes it so that our wives don’t need rescuing? Well, this is a rescue in progress. You may not be slaying a dragon, but this is your battle, and you’re fighting for your wife’s heart.

Possibly the biggest thing you can do for your wife is protect and nurture her heart. Be the man who comes to her rescue. Always side with her in public. Your kids will try and come between you … don’t let them. People will try to run her down and insult her … vigorously defend her. People will try and use her up … don’t let them. If she loves you, your words will have great power over her life and how she sees herself. With great power, comes great responsibility. Always encourage her, always love her. She will complete you just as you complete her.

Books

Wild at Heart
Captivating
The Five Love Languages

When I Went Walking

October 19, 2009 / / 0 Comments

At the beginning of the year, I asked each of my sons for some ideas about some activity that they really wanted to undertake. Mathew wanted to try fencing (with swords, not posts and rails), Nathan wanted to take up archery, and Josh wanted to go on an overnight hike with me. Well, the first two were achieved pretty quickly, but it took until the weekend just gone to go with Josh for an overnight.

You can look at the guide for the walk that we took here.

Now let me say up front that it was a precious weekend that we will both remember, for various reasons. It was very affirming for Josh, and he has realised some pretty good things about his own strength now. I also have discovered a few things, and they have to do with my age and a few other illusions that I held about myself. I’d like to believe that the following can be called wisdom … or maybe it’s something related, but not quite as good. Anyway, here are a few thoughts that occurred to me on the walk.

Let me describe myself first. I’ve just turned 42. I’m 184 cms tall, and before this walk, I weighed 107 kgs. Although this means that I’m obese according to the BMI, I don’t look it. I’m admittedly overweight, but I have high muscle density. Without any preparation at my sons’ school one night, I deadlifted 240 kg and did a standing broad jump of around 2 metres. Dumb, but there you go. I can run a couple of kilometres without stopping, so I considered myself in pretty reasonable shape.

Are my peers listening? If you are thinking of doing such a walk, please consider the following first.

Illusion Reality
They write “Hard” in the rating to stop people in wheelchairs attempting the walk. Apparently, “Insanely difficult unless you are (a) 12 years old; or (b) prepared” is too long to fit in the space they have available for the rating field.
The times they write for each section of the walk are maximum times. The times they write on the guide are determined by extreme hikers who carry nothing more than clothing and discover caches of “Mother” on the track. Or by 12 year olds carrying full packs.
My Asics are great walking shoes. My Asics are great running shoes. They breathe. Unfortunately, this means that they inhale dust, which becomes grit and causes blisters.
My favourite socks are the most comfortable socks. My favourite socks have been through the washing machine many times, and the “cushioned soles” have become quite hard and cause blisters.
My feet are fine. Cut your toenails before you go, or they will cut you.
My feet are fine. My feed are soft. “Tenderfoot” is a real description of my current pampered state.
I have a lot of muscle … and that’s good. More muscle means more that can cramp when I’m trying to reach my feet to get my socks off. Never before have I had my whole upper front and inner leg (groin to knee) cramp at once. Both legs. While in sleeping bag, so I can’t stretch.
I can handle anything my 12 year old son can handle. I can still beat my 12 year old son in most mind games, rumbles and arm wrestles. In any activity involving any sort of endurance, he will leave me in the dust.
“Steep” is a word that means I might break a sweat on this part of the trail. “Steep” is almost a synonym for “precipitous,” and I think “precipitous” is the more accurate word.
There would be a couple of nice stops on the way to catch my breath. On the “steep” part, I actually had to stop every 30-odd steps to catch my breath and wait for my quads to stop cramping. Actually, I wasn’t able to catch my breath. I had to wait for it to come back. Catching anything other than a cold was completely beyond me.
I’d done the math and figured with only 8 or 9 hours of actual walking, I wouldn’t need much extra water, so took 2.5 litres. We actually did the entire thing in 24 hours (including feigning sleep on the top of a mountain), but 4 litres would have been more appropriate. Fortunately, I found a functioning bubbler at the oval in Patonga, and over the next hour drank another 1.5 lt.
I don’t need a hiking staff. Josh made one before we left, and I used it more than he did. There came many points (usually part way through a “steep” part of the track) where I discovered that my legs would no longer lift my vast (and increasing) bulk up to the next step. The stick was a great way of allowing my arms to take over part of the burden.
I’m not old. I have hip joints, and they dislike abuse. Oh dear. Come the end of it, I think my grandfather would have outpaced me. And he’s dead. And before he died, he had blood clots, emphysema and an artificial leg.

Seriously, to see my hobbling gait today (the day after we came home), you would be overwhelmed by the twin urges to pity and laugh. At least, my family is.

If you are like me (sedentary, carrying a bit much weight, but otherwise fairly healthy) and you are considering undertaking such a walk, let me submit to you a couple suggestions.

  • Multiply the times from the guide by about 1.5.
  • Get some new socks.
  • Get good walking boots and wear them every day for a couple of weeks.
  • If you go for a daily stroll, take your pack with the same amount of weight that you’ll be taking on your walk.
  • If you don’t go for a daily stroll, start. And take a pack.
  • Take more water than you think you are going to need.
  • For an overnight walk, you don’t really need any more clothing than you wear at the start. Unless you happen to go swimming, or it starts raining.
  • Take sunscreen. I did, and what’s more, I used it! Woo hoo!
  • Chafing. You will chafe. Some of this chafing won’t become apparent until you’re showering or using the toilet when you get home. I have no idea what you can do about this, apart from losing the excess weight before you go. And have talcum powder ready.
  • Get a hiking stick or staff.

It was a great experience, and not one I’m likely to forget. Josh had a great time, having his Dad to himself for a whole weekend. There is boundless beauty along the track, and it’s worth doing walks like this regularly. But for heaven’s sake, don’t think you can just do it without preparation and come out unscathed.

Commitment, Options and Risk

May 27, 2009 / / 0 Comments

I think that we are a risk-averse society. Eh? Where did that come from? Aren’t there heaps of risk-takers about? Extreme sportspeople, for example? OK, what percentage of our society are extreme sportspeople? I’m talking about the “mass of men” that “lead lives of quiet desperation.”

I’ve been speaking fairly regularly with Nigel Allan. Something that he speaks of every now and again is the aeronautical term “V1”. V1 is the go/no-go point of getting an aircraft off the ground. If you get to V1, you have the choice of getting yourself into the air or crashing. Doesn’t sound like much of a choice to me. The thing is that at that point, you have to be committed to getting off the ground, and you must have discarded all of the other options that you think may have been feasible.

So bring this back to every day life. How many ways do we avoid reaching V1?

Once upon a time, I was in a car with my Dad. When I pointed out that there was less traffic in the left (slow) lane and, based on my experience, the trip is faster, he said no, he wanted to stay in the middle lane to keep his options open. Have you ever held off accepting a job, because you had another couple of irons in the fire, and wanted to keep your options open? Held off making a decision between movies and a picnic because you weren’t sure about the weather?

What about these stock market “risk takers”? That is possibly the most extreme example of a lack of commitment in the world. When you’re one of the big players, you always keep your options open. You have exit strategies for any position, and if something looks like it’s going bad, you bug out. You have alarms that go off if it looks like a position is taking a hit, and you bail as soon as you can. You don’t hold on to a stock that looks like it’s going bad just because you like the company … there are no friends in the stock market. There is no long term commitment there to anything except the game.

People live together before they marry because they don’t know if they want to commit. They have pre-nuptial agreements when they do marry because they aren’t prepared to commit for life and/or they aren’t prepared to believe that their intended spouse is prepared to commit for life.

Commitments involve risk. They involve getting to V1, and I believe that it’s getting to V1 that we avoid. We put off committing to a course of action; we keep our options open. There is an inherent problem with that: with multiple options in mind, we have to juggle them and be aware of them. They cost us time and money. They cause us stress because they aren’t necessarily compatible. And when we finally get to a point where we have to make a decision, we’re not prepared for it. We’ve spent all of this time and energy in keeping options open, and we’re tired.

So let’s think about that pilot. She’s cruising down the runway, but by the time she got into the plane, she’d committed to a destination. By implication, she’d committed to taking off at V1. She’s looking for reasons why it would be unwise or unsafe to take off, but her destination is set. She’s not wondering whether she’s going to Tahiti or New York or whether she’s going to circle the airport a couple times, or maybe just taxi around for a couple of hours. She’s committed to her destination.

So here’s my advice, for what it’s worth. Commit! Mean it! Take the risk! Decide on the picnic, and risk the weather. Do yourself a favour and stop juggling all the options. If necessary, take a few minutes and write down pros and cons for the biggest few options before you decide, but commit to your destination well before you have to make a decision. Then start to breathe the air and smell the roses … you don’t have to be so stressed any more.