Compress files before copying from sFTP site

Also, suppose you wish to only compress files with the extension ".txt", you can use the following command:

plink -ssh 015_test cd /home/username/brie ; find . -name "*.txt" | tar czfTP bacon.tgz - ; cd -

If you wish to search and compress multiple extentions, use the following code. Note the "-o" option in-between file types

plink -ssh 015_test cd /home/username/brie ; find . -name "*.txt" -o  -name "*.info" | tar czfTP bacon.tgz - ; cd -

So if you want three file extension type the command becomes:

plink -ssh 015_test cd /home/username/brie ; find . -name "*.txt" -o  -name "*.info" -o -name "*.get" | tar czfTP bacon.tgz - ; cd -

Obviously you can man find to find more options suitable for your specific purpose. Linux "man" is your friend!

Please post any new buttons you do related to this. I find them useful. :slight_smile:

Thanks for all those codes and information.

The main difference between your code, which is one line, and mine, which is much more complicated (and is using VBScript), is that you need to have a saved session in Putty. My code works for any sFTP server you are connected to with DOpus, and I find it much more useful as I do add sites only to DOpus address book. I don't add any sites to Putty. And as far as I understand, if I frequently connect to 5 different sFTP servers, I need 5 different "compress" buttons because the -ssh parameter will vary.

But your posts made me think about other possibilities that plink gives, like restarting servers, etc. I think I will use it much more often in the future. :slight_smile:

No worries. As long as you are happy with the command buttons you have, that's cool beans :slight_smile: You do raise an interesting problem though, which I would be very curious to see the solution for, just for my personal use.

In order to modify the code so that it uses any username, password and hostname, the following options are available for plink:

-l = login name
-pw = password

So the code can be modified to:

plink -ssh ${Hostname} -l ${loginID} -pw ${password} cd /home/username/brie ; tar -zcvf /home/username/brie/aiCarambaMoreCheese.tgz --exclude='*.txt' test_tar ; cd -

You can get ${Hostname} from the ftp path shown in the Dopus destination lister.

You can easily create the additional variables that accept dlgstrings for login name & password.

This works fine. However, it does not solve the problem of integration of putty with Dopus. From another post (links below), Dopus actually uses the putty pageant code, so I think it would somehow be possible to get plink to reference this info somehow.

Perhaps Leo or some of the other Mods might let us know whether this is possible. Ideally you want a command like the following:

plink -ssh ${Dopus_ftpAddressBook} & cd /home/username/brie ; tar -zcvf /home/username/brie/aiCarambaMoreCheese.tgz --exclude='*.txt' test_tar ; cd -

However, it's prob not possible to link things this way. I had the following questions that I hoped someone might be able to answer:

1) Is there any command to grab the username, password & hostname from the Dopus ftp_address book, depending on what ftp site you are connected to in the destination lister?

2) Where is the Dopus ftp address book stored and is it encrypted?

Also where is the

Many thanks! :slight_smile:

Address book is stored in
C:\Users[username]\AppData\Roaming\GPSoftware\Directory Opus\ConfigFiles\ftp.oxc
You can look at it and see that passwords are not plain-text. Anyway, because DOpus does not require any password when opening FTP address book or when starting DOpus itself, passwords security depends on Windows mechanisms that protect user files. If someone gets access to your profile files, he is most probably also able to decode those passwords.
This is, however, interesting how those passwords are encrypted/encoded.

A command to grab username/password/hostname/path may be very useful for things we do here. Using VBScript is not very convenient so I would also be very interested in commands to read these values.

I gave up using Opus for transferring large amounts of files to and from FTP's, it is such a slow process. I find Opus okay for transferring a few files but for large volumes of files it really needs to support multiple connections. You could try a dedicated FTP client like SmartFTP which even allows for simultaneous threads. It even comes with a basic terminal for accessing SSH.

I did notice that ftp was slow, but I always assumed it was the state of my connection and since I don't really need to transfer files so much as I tend to work remotely on other machines, it never reared its's head as an issue.

WinScp seems quite interesting as it has a command line & scripting. Therefore you could possibly use Dopus to send commands to it and let it handle the ftp transfer. Looks a bit hardcore though. :slight_smile:

winscp.net/eng/docs/commandline
winscp.net/eng/docs/executables
winscp.net/eng/docs/guide_dotnet#input

Remember that SCP is a different protocol, not FTP over SSH as is supported in Opus.

true. sorry i was sloppy in my wording. i know it's a bit off topic, but what I found quite interesting is something called high performance ssh. been wanting my work to implement it, but the wheels grind slowly. :frowning:

psc.edu/index.php/hpn-ssh

Ok. If you want to automatically set up passwordless ssh for any new sftp site that you are going to connect to, try the following button:

You only have to run this button once for every new sftp server you wish to connect to.

Caveats:

  • You need plink, pageant and pscp installed. Probably easiest to install the full putty package

  • I've added an environment variable to Windows that means I don't have to type the full path to putty. If you have not done this, edit the button so the path to the putty programs are valid.

  • You need to have generated a RSA private key called "id_rsa.ppk" and a public key called "id_rsa.pub" as per the instructions in an earlier post in this thread. Key has to be Rsa. If you choose DSA, button has to be modified. Think RSA is better.

  • the .ssh directory has to be installed in the default directory which in 90% of cases is /home/userid/.ssh. If home is in a non-default place, the button can probably be modified to reflect this on the pscp line if it's a consistent thing. If not a lot of remote machines have this home directory path format, then it's not worth it.

1) Execute this button

// Sets up username, password & Remote machine that you want to connect to
@set UserID {dlgstring|Enter Login ID}
@set PassWD {dlgstring|Enter Password}
@set RmachID {dlgstring|Enter Remote Machine Address}
// Opens up Pageant with your private key. Check Blue icon in system tray to indicate pageant is running
pageant "D:\Program Files (x86)\PuTTY\id_rsa.ppk"
// Copies your public key to the .ssh directory on the remote machine
pscp -l {$UserID} -pw {$PassWD} "D:\Program Files (x86)\PuTTY\id_rsa.pub" {$RmachID}:/home/bsgt/.ssh/temp_pub
// Keep this command as one line! Formats temp_pub contents on remote machine and formats the resulting string into a 
// format suitable to be added to the .ssh authorized_keys file
plink.exe -ssh {$RmachID} -l {$UserID} -pw {$PassWD} cd .ssh ; LocMachID=`less temp_pub | grep Comment | tr '"' ' ' | awk '{print $2}'` ; less temp_pub | grep -v BEGIN | grep -v END | grep -v Comment > temp_pub2 ; echo ${LocMachID} >> temp_pub2 ; less temp_pub2 | tr '\n' ' ' | sed -r 's/\s+//g' | sed 's/^/ssh-rsa /' | sed 's/==/== /' > temp_pub3 ; cat temp_pub3 >> authorized_keys ; rm temp_pub* ; cd -

2) Now if you add a new sftp site to your Dopus address book, make sure the password field contains the following string "nonsense"

Configure Pageant to automatically start up with Windows. e.g. Put a shortcut to it in the Windows start menu startup folder

From now on when you try to connect to that site, the string "nonsense" will fail authentication and the private key stored in pageant will authenticate you.

[quote]Make sure you do the following for best putty performance:

Configure the default session in putty.

a) enter your remote machine hostname & port (usually 22). Select ssh as connection type

b) Go to Category --> Connection -->Data
enter your user id for the remote machine in the auto-login bit

Go to Category -->Connection --> SSH
Check "Enable compression"
Move Blowfish to the position in-between AES and Arcfour
save your settings[/quote]

I would like to modify the button so that a further two commands are added at the end.

  1. Use Dopus to Connect to the sftp site immediately. I tried the following based on this in the manual:

[quote]FTP locations are referenced internally using a URL-style path format, with the ftp:// prefix. The full format of this path is:
ftp://:@://[/quote]

ftp://{$UserID}:{$PassWD}@{$RmachID}:22//home/{$UserID}

However, this does not connect.

  1. The last command is to automatically add the sftp site into the Dopus address book, similar to the menu command "FTP Add To Address Book". However, could not find anything in the manual that does this.

If anyone could help me on this, I would be obliged. :slight_smile:

Try giving the ftp url to the Go command instead of running the url as if it was a command itself.

But also note that will probably only work if you want to connect via FTP, not SFTP/SSH.

I'm not sure there is a way to do what you're trying to do there.

Thanks for the quick reply. :slight_smile:

Unfortunately, you are right, it's not possible to use go ftp with a sftp site.

Just to clarify, is there no inbuilt command to directly add entries into the address book? If not, I suppose the only way is to directly edit the ftp address book stored in ftp.oxc?

I wouldn't recommend editing the address book as we may change the format and the data may also be cached in memory while Opus is running.

No worries! Thanks for the info. :slight_smile: Looks like the address will have to be added manually.