Announcement

Collapse
No announcement yet.

Byte Level Replication under Linux

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Byte Level Replication under Linux

    Does anyone here have any experience with any Linux based byte level replication packages? I'm looking for something that is preferably free and open source. I need to take a VM server running on a FC6 host and do real-time byte replication of the .VMDK files over to a secondary FC6 VM host.

    I looked at a few commercial packages but would also like to explore open source alternatives. I'm about to start playing with Pratima but wasn't sure if any of you other Linux IT professionals have used other solutions.
    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  • #2
    Re: Byte Level Replication under Linux

    You might can use "dd" under the command line. The command would be something like "dd if=/foo of=foogoeshere". Take a look at the man files to see if it will work for you.

    I am by no means an IT pro and definitely not a linux expert but I have used dd in my job to replicate images before and it worked quite well.
    Retired 6th DB

    Comment


    • #3
      Re: Byte Level Replication under Linux

      I would strongly recommend `dd`, unless it's known not to work with this sort of file (i.e., backing up DRM-protected game discs). You can use `dd` in combination with `netcat`, or its secure brother, `snetcat` (or encrypt it yourself).

      I'm not an IT professional, so I'm not sure if this is useful advice. I don't know what a VMDK file is.
      The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
      I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
      Feel free to quote me. ~

      Comment


      • #4
        Re: Byte Level Replication under Linux

        DD isn't going to do real-time byte level replication. It's going to make an image of the disk that would need to be transferred. Doing this with multiple 100+ GB files is impractical. The only package I found that will do what I want is the NetVault: Replicator package from BakBone. Unfortunately we would be looking at around $12,000 in licensing.

        The only other solution I've found is using Unison over SSH. It can do block-level changes on files so it will limit the amount of data that will need to be transferred between the machine (and over "slow" links to a remote site). It's free and open source, so at least it will be within the corporate budget to roll out. It's not real-time, but depending on how much data gets changed, I might be able to run replicas every 15 minutes.
        Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

        Comment


        • #5
          Re: Byte Level Replication under Linux

          Let me clarify:

          In the storage/HA industry, byte level replication refers to only replicating bytes within a file or volume that have changed rather than the entire file or volume. It's considerably different than a byte level copy.
          Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

          Comment


          • #6
            Re: Byte Level Replication under Linux

            The impression I get from http://www.vmware.com/interfaces/vmdk.html is that these files are not updated in such a way that realtime transfer would be necessary. However, since apparently they are, is there a way to set up the environments themselves to monitor and report all changes to the environment (i.e., a kernel-space module)? (This would only be practical in a UNIX environment.)
            The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
            I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
            Feel free to quote me. ~

            Comment


            • #7
              Re: Byte Level Replication under Linux

              Originally posted by Apophis View Post
              DD isn't going to do real-time byte level replication. It's going to make an image of the disk that would need to be transferred. Doing this with multiple 100+ GB files is impractical. The only package I found that will do what I want is the NetVault: Replicator package from BakBone. Unfortunately we would be looking at around $12,000 in licensing.

              The only other solution I've found is using Unison over SSH. It can do block-level changes on files so it will limit the amount of data that will need to be transferred between the machine (and over "slow" links to a remote site). It's free and open source, so at least it will be within the corporate budget to roll out. It's not real-time, but depending on how much data gets changed, I might be able to run replicas every 15 minutes.
              Ah gotcha! Unison looks like a viable solution since it uses rsync. I use rsync at home to back up my wifes work computer to our server. The initial transfer is huge but after that is over with just the files that have changed get sync'ed.
              Retired 6th DB

              Comment


              • #8
                Re: Byte Level Replication under Linux

                Unison might be troubled to change to meet your needs, as it appears to be halfway there. I wish I knew more about this stuff; I'd develop it myself.
                The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
                I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
                Feel free to quote me. ~

                Comment


                • #9
                  Re: Byte Level Replication under Linux

                  Originally posted by ednos View Post
                  Unison might be troubled to change to meet your needs, as it appears to be halfway there. I wish I knew more about this stuff; I'd develop it myself.
                  Well, Unison is based off rsync, which is something I've used rather extensively over the past many years. I used rsync to keep web content on two Solaris 2.6 x86 in sync for failover purposes. I was initially going to just use rsync, but Unison seems to be a better product.

                  What do you mean about Unison being troubled to change to meet my needs? I'm not quite following you on that one.
                  Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

                  Comment


                  • #10
                    Re: Byte Level Replication under Linux

                    I mean it should be only a moderately challenging project to modify it to monitor files in realtime and report changes, which is what I have interpreted as your requirement. Correct me if I am incorrect.
                    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
                    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
                    Feel free to quote me. ~

                    Comment


                    • #11
                      Re: Byte Level Replication under Linux

                      Originally posted by ednos View Post
                      I mean it should be only a moderately challenging project to modify it to monitor files in realtime and report changes, which is what I have interpreted as your requirement. Correct me if I am incorrect.
                      Ahh.. Yes. I would like it to work in real-time, but that's not necessarily an absolute requirement of the project. If I proposed a 15 minute replication cycle, I'm sure they would be fine with it.

                      I thought about using some code that monitors the inode tables (via loadable kernel module) to trigger the replication, but that's honestly more trouble than it's worth for me to do.

                      Ahsay as a nice add-on to their OBS product that does real-time byte level replication. It's also pretty cheap, but unfortunately it only does replication between OBS servers and not between the file systems directly.

                      At one point in time I looked at using Ahsay and OBS to provide off-site network backup services for TG supporting members, but it was cost-prohibitive. I could always support it as an add-on service for TG folks and do it at cost, but I'm not sure what the demand is for this type of service within TG.
                      Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

                      Comment


                      • #12
                        Re: Byte Level Replication under Linux

                        Originally posted by Apophis View Post
                        I thought about using some code that monitors the inode tables (via loadable kernel module) to trigger the replication, but that's honestly more trouble than it's worth for me to do.
                        And already been done. IIRC inotify has been in the kernel for a while now, and provides file-change monitoring services.

                        Comment


                        • #13
                          Re: Byte Level Replication under Linux

                          Hmm, I've never seen it. Then again, I haven't used a kernel beyond version 2.6.14.
                          The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
                          I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
                          Feel free to quote me. ~

                          Comment


                          • #14
                            Re: Byte Level Replication under Linux

                            Update:

                            After quite a bit of research and product trials I came down to a few solutions.

                            One of which was not actually replicating the data on our RHEL server but just doing normal byte level replication inside the Windows hosts that we run under VMware on the RHEL box. That proved to be costly as software like DoubleTake isn't cheap. We would also need a pair of licenses for every Windows host we wanted to replicate.

                            I looked into another product called NetVault Replicator from BakTrack. This is truly a great product but it's quite pricey. It would do everything I needed but to the tune of $10,000 per pair of HA servers.

                            In the end I ended up playing around with DRBD, Heartbeat and Open-iSCSI.

                            Here's what I did:

                            I set up two machines under VM as a test. Installed Fedora 7 on each of them and updated to the latest kernels. I then downloaded DRBD and got that configured on each of the machines. I gave DRBD a physical disk to manage and copied a bunch of files onto the disk on the Primary server. That data was replicated at the block level over to the Secondary server. I then promoted the Secondary server to Primary with DRBD and mounted the volume and all my files were there. Excellent.

                            Once DRBD was fully tested I downloaded and installed Heartbeat V2. After a couple hours of reading and configuration, I had Heartbeat managing DRBD and the mount/dismount of the volumes. If I failed the primary server, Heartbeat would automatically promote the backup server to primary and mount the drive accordingly. Excellent. I now had automated failover between the two servers.

                            I tested that for a bit to make sure it was working as planned and attempted various methods of "breaking" the replication to make sure it was stable and would recover from failure in a predictable manner.

                            When all my testing was done. I went ahead and installed iSCSI on the box and set up fileio block volumes on the DRBD managed drive. I mounted the /dev/drbd0 drive under /mnt/array/ and created block volumes at /mnt/array/iscsi-target0.img, /mnt/array/iscsi-target1.img, and /mnt/array/iscsi-target2. Once the iSCSI targets were created, I used the wonderful (heh, right) iSCSI Initiator 2.04 on my XP machine and mounted the targets. Once mounted, I was able to copy a bunch of files from a local drive to one of the iSCSI drives just as you would to a local drive.

                            My final failover testing consisted of taking the primary offline and verifying that Heartbeat properly mounted the /dev/drbd0 drive on the secondary and then fired up the iSCSI daemon to export the targets. All worked well. Bringing the primary back on-line automatically triggered a reverse replication from the secondary back to the primary before stopping iSCSI and unmounting the drives on the secondary and then mounting the drives and starting iSCSI on the newly recovered primary.

                            The only issue I have now is getting the Microsoft iSCSI Initiator to look at at the virtual IP address that Heartbeat tosses back and forth between the servers rather than the physical IP addresses of the two boxes. It appears to just be a Microsoft iSCSI implementation issue, but I need to do a bit more testing to confirm that.

                            If anyone is in a position where they need to build extremely low-cost high availability file servers, this is a great way to go. I'd gladly share my config files and any tweaks I had to perform to get this running.
                            Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

                            Comment


                            • #15
                              Re: Byte Level Replication under Linux

                              Sweet! I was just searching for this thread about a week ago, since we might need a similar solution at work. I also want to play with this stuff on my own computers because it's so damn *cool*.
                              The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~
                              I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~
                              Feel free to quote me. ~

                              Comment

                              Connect

                              Collapse

                              TeamSpeak 3 Server

                              Collapse

                              Advertisement

                              Collapse

                              Twitter Feed

                              Collapse

                              Working...
                              X