tag:blogger.com,1999:blog-55260756708225009882024-02-19T04:36:51.659+01:00Pablo S. TorralbaWe are what we do, what we think, how we act. Don't ask me who am I. Just judge by yourself.Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-5526075670822500988.post-9024368074536101362007-08-29T02:15:00.000+02:002007-08-30T11:18:43.458+02:00Shell date and time tricksOne of the things I hate the most when programming is dealing with time and date functions and all the special cases which exists. Even more, I have the feeling I'm walking a path many times walked before, so it becomes one of the most unpleasant tasks for me.<br /><br />Fortunately, it's true, so many others walked before that path so here are some quick tricks for shell programming using the wonderful <span style="font-size:85%;">date <span style="font-size:100%;">UN*X program</span></span>:<br /><ol><li>Converting epoch:</li><ol><li>From epoch to anything else: <span style="font-size:85%;"><span style="font-weight: bold;">date -d @$epoch_value +FORMAT</span><span style="font-size:100%;"> (where FORMAT is of course as described on date(1), and '@' makes the actual undocummented trick).<br /></span></span></li><li><span style="font-size:85%;"><span style="font-size:100%;">From anything else to epoch: <span style="font-weight: bold;font-size:85%;" >date +%s</span><br /></span></span></li></ol><li>Calculating times</li><ol><li>One day forward: <span style="font-weight: bold;font-size:85%;" >date -d "1 day"</span></li><li>One day backwards: <span style="font-weight: bold;font-size:85%;" >date -d "1 day ago"</span></li><li><span style="font-size:85%;"><span style="font-size:100%;">Just imagine "1 month", "3 months ago" and the like. Not only google is so friendly with human language ;-)<br /></span></span></li></ol><li>More format conversion: '<span style="font-weight: bold;font-size:85%;" >-d</span>' option accepts several other formats as input, even with calculations:</li><ol><li><span style="font-size:85%;"><span style="font-weight: bold;">date -d "1977-08-19 30 years"</span><span style="font-size:100%;">, yeah! my 30th birthday was on sunday. Thanks date, and it was (as epoch): <span style="font-size:85%;"><span style="font-weight: bold;">date -d "1977-08-19 30 years" +%s</span></span>... 1187474400 :-D</span></span></li><li><span style="font-size:85%;"><span style="font-size:100%;">Funny ls: <pre><br />ls -l | while read perms links user group size d t name<br />do<br />echo $perms $links $user $group $size $( date -d "$d $t 1 day" ) $name<br />done<br /></pre>You can, of course, change the way date is shown. That's because 'YYYY-MM-DD hh:mm' is also a valid input format for date as it is 'YYYY/MM/DD'.</span></span></li></ol></ol>So '@', '1 day' and 'date' saved me the day.<br /><br /><span style="font-weight: bold;">Update: </span><span style="font-style: italic;">This features and more are indeed described in the coreutils info manual. Thanks </span>mp<span style="font-style: italic;"> for pointing it out.</span>Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com10tag:blogger.com,1999:blog-5526075670822500988.post-42323143515802527502007-07-19T00:21:00.000+02:002007-07-19T00:42:00.620+02:00Damn GFSJust a few hours before the public release of the portal we work on one of the cluster machines get overloaded by issues no relevant for the point. The fact is once the machine started to not answer properly so it was fenced by some other node, but the problem was once this automatic action was held, the whole 6 machines GFS cluster went down letting all the machines unusable.<br /><br />This, in addition to all of the previous issues we suffered on GFS, made us really thinking about purging GFS in favor of NFS. It was no an easy decision as it was fully against all our previous decisions but we weren't confident about GFS in the production systems. So we migrated it in a time record configuring everything by night so at six o'clock service would be held properly. And we managed to fulfill this purpose. We made it!<br /><br />We are now tired after about 27 continuous working hours but the overall result was quite acceptable. I'm still proud about or design (not so much about my own decissions) which allowed us to make this king of changes so quickly.<br /><br />But you can be sure I don't think we will never again think about installing GFS on any system as it seems not being production suitable (as RedHat even says so). And it is not only because the buggy GFS2 (at least, at present date) but for the sensation of instability all over the time we had it installed.<br /><br />So in a few hours our architecture has been changed, but it was setup for the very moment we were accepting requests.Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com18tag:blogger.com,1999:blog-5526075670822500988.post-31031489489931800062007-06-18T19:46:00.000+02:002007-06-18T22:27:53.893+02:00IPMIIPMI stands for "Intelligent Platform Management Interface", an interesting feature deployed on nowadays servers. I can't forget that question I made some months ago about an extra ethernet port on the IBM server machines we were working with on those days. The answer was quite simple: it is almost of no use at all; at most you can get some stats and diagnostics but it is mainly intended for hardware technicians.<br /><br />Liars (or ignorants)! Anyway, it is quite useful. It is the IPMI port which can be used for several managements actions (even from the running system) such as power cycle the machine, get some stats, establish a watchdog interface and the like.<br /><br />Uses?, are they need to be told?. As a first glance you can get reports about the machine from the operating system itself without the need of physical access to the datacenter. You can also restart a machine which is failing to reboot by itself in case you can login one more time in the system. But... even more, you can do all that (and more) from a remote system (don't exhitate, it is password protected, at least, in case you configure it so). So the next obvious uses are for cluster fencing and stonith (shot the other node in the head) without the need of power control hardware.<br /><br />It is a pity we didn't knew that when we configured the SAP cluster in my previous job despite the serial cable almost did it work. Sure Mr Navas will be interested in knowing about this technology :-)<br /><br />Oh, a simple example:<br /><br /><span style="font-size:85%;"><span style="font-weight: bold;"># ipmitool -I open chassis status</span><br />System Power : on<br />Power Overload : false<br />Power Interlock : inactive<br />Main Power Fault : false<br />Power Control Fault : false<br />Power Restore Policy : always-off<br />Last Power Event :<br />Chassis Intrusion : inactive<br />Front-Panel Lockout : inactive<br />Drive Fault : false<br />Cooling/Fan Fault : false<br />Sleep Button Disable : not allowed<br />Diag Button Disable : allowed<br />Reset Button Disable : not allowed<br />Power Button Disable : allowed<br />Sleep Button Disabled: false<br />Diag Button Disabled : true<br />Reset Button Disabled: false<br />Power Button Disabled: true</span><br /><br />Another one:<br /><br /><span style="font-weight: bold;font-size:85%;" >ipmitool -I open chassis power reset</span><br /><br />Ooooooops. Next post after crash :-)Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com0tag:blogger.com,1999:blog-5526075670822500988.post-46575533536922822892007-06-06T16:11:00.000+02:002007-06-06T18:50:48.320+02:00GFS choiceWe are designing at work a clustered architecture for the site we are developing. For that purpose we needed a clustered/distributed filesystem such as NFS, GFS, OCFS, and the like. We all have already had bad experiences using NFS as it usually hangs a client if server goes down. In addition, on that design we have a SPOF (single point of failure) we were trying to avoid. So we give up with it.<br /><br />The following candidates were GFS and OCFS. Despite there are some other suitable filesystems they are not so widely supported in the linux kernel and distributions so we couldn't fit with our hosting service level agreements bringing them out of question. And before you ask, linux itself was indeed a requirement. On a housing situation, things would have been different.<br /><br />Back on FS, at first glance on our tests OCFS results wasn't as good as GFS ones. In addition, no all distribution supported online resizing in the packaged version of ocfs tools so we finally decided to use GFS as the clustered filesystem. But once we have almost all infrastructure ready, later tests showed it wasn't as good as we thought, and not so documented neither supported by RedHat itself as we expected. In fact, performance was an issue which was worrying us as it would slow down the whole site. Fortunately, after several dark options tests the solution was much simpler than that: GFS is a distributed FS which relies on network performance. Switching to giga ethernet made the trick. It boosted performance enough to make us more confident about our design and inversion.<br /><br />Now we can focus on another of all that matters remaining before our product launching soon, quite soon, frightening soon.Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com1tag:blogger.com,1999:blog-5526075670822500988.post-65658650687485545892007-05-22T00:34:00.000+02:002007-05-22T00:39:21.527+02:00Emulation vs. VirtualizationThese days virtualization is the breaking new appliance for the enterprise. Everybody is migrating to this "new" concept but, which is the difference with emulation?<br /><br />And interesenting one. Just for the sake of correctness:<br /><ul><li><span style="font-style: italic;"><span style="font-weight: bold;">Emulation</span></span> involves emulating the virtual machines hardware and architecture. Microsoft's VirtualPC is an example of an emulation based virtual machine. It emulates the x86 architecture, and adds a layer of indirection and translation at the guest level, which means VirtualPC can run on different chipsets, like the PowerPC, in addition to the x86 architecture. However, that layer of indirection slows down the virtual machine significantly.</li><li><span style="font-style: italic;"><span style="font-weight: bold;">Virtualization</span></span>, on the other hand, involves simply isolating the virtual machine within memory. The host instance simply passes the execution of the guest virtual machine directly to the native hardware. Without the translation layer, the performance of a virtualization virtual machine is much faster and approaches native speeds. However, since the native hardware is used, the chipset of the virtual machine must match. Usually, this means the Intel x86 architecture. VMWare is an example of this type of application<br />for Windows.</li></ul>Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com13tag:blogger.com,1999:blog-5526075670822500988.post-43875584501130773732007-04-16T18:03:00.000+02:002007-05-18T21:05:35.258+02:00Photo techIn the last two weeks I've discovered (or was told about) some programs quite useful for an outdoor technician (amateur photographer):<br /><ul><li><span style="font-weight: bold;">gpscorrelate</span> which adds GPS tags on exif fields correlating gps tracks and photgraph shooting time. Finally, I am able to geoposition my trekking pictures automagically!. Next step: some way to create google maps paths with POIs.</li><li><span style="font-weight: bold;">hugin</span> for image blending. The following image was obtained in Gredos composed from four pictures.<br /></li></ul><div style="text-align: center;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmAG4XZ6MF8PJo9ytAhnGEa3Tr4TBkJaxWtS9ZNQQX2Xx2XcgaJPCmu4bdLtZ3FZlieZEwVFTDDZZM6p-6OTiXgr17IY75hJ9O5hLcF7xG89C5mG84SKBKDuSV8VpjaRworJSLNoTdv6g/s1600-h/circo_gredos_clean_scaled.jpg"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmAG4XZ6MF8PJo9ytAhnGEa3Tr4TBkJaxWtS9ZNQQX2Xx2XcgaJPCmu4bdLtZ3FZlieZEwVFTDDZZM6p-6OTiXgr17IY75hJ9O5hLcF7xG89C5mG84SKBKDuSV8VpjaRworJSLNoTdv6g/s400/circo_gredos_clean_scaled.jpg" alt="" id="BLOGGER_PHOTO_ID_5054060409059583890" border="0" /></a><br /><div style="text-align: left;"><br /></div></div>Pablo S. Torralbahttp://www.blogger.com/profile/09576944800522304236noreply@blogger.com0