Help with SFTP Download Script :: Support Forum

gina.bautista@gmail.com
Joined:: 2016-06-20
Posts:: 2
Location:: SLO, CA

Help with SFTP Download Script

2016-06-20 20:39

I need help. I am trying to build a script that downloads text files via SFTP from a remote site. I am using Powershell 3.0. I've tinkered with Powershell and SFTP add-on, but I'm not getting anywhere with that either. There are always going to be multiple text files and I only want it to grab and pull down those text files. It connects, but it does not download the files where I need them. Script is below.

try
{
    $logpath = "D:\Report-Copy-Move\Logs\CDCFH-demo_download_results.txt"

   #Source Folders, use absolute path
   #These folders must only have files that are to be downloaded
    $CDCFH_IN = "/username/RDA/IN/CC/MS4/DEM/"
        
   #Target Folders, use absolute path
    $CDCFH_OUT = "\\transferserver\dignity-billing\agch_fhmc\"
        
   #Archive Folder Location, use absolute path
    #$CDCFH_Archive = "D:\ininbound\ftp-archive\cdc-archive\"
        
   #Load WinSCP .NET assembly
    Add-Type -Path "WinSCPnet.dll"
 
   #Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.Protocol = [WinSCP.Protocol]::sftp
    $sessionOptions.HostName = "mft.dignityhealth.org"
    $sessionOptions.UserName = "username"
    $sessionOptions.Password = "password"
    $sessionOptions.PortNumber = "xxxx"
    
   #Use the following line with the SSH HostKey Fingerprint by replacing the xx:xx:xx...
    $sessionOptions.SshHostKeyFingerprint = "ssh-rsa 1024 97:83:f1:ec:8c:c4:3b:78:29:d9:c5:10:c8:fe:93:08"

    
   #ONLY FOR TEST PURPOSES, DO NOT USE AS IS
   #$sessionOptions.GiveUpSecurityAndAcceptAnySshHostKey = true
   #$sessionOptions.GiveUpSecurityAndAcceptAnyTlsHostCertificate = true
    
    $session = New-Object WinSCP.Session
    $session.SessionLogPath = "D:\Report-Copy-Move\Logs\CDCFH-demo_download-session-log.txt"  
 
    try
    {
       #Connect
        $session.Open($sessionOptions)
 
       #Force binary mode transfer
        $transferOptions = New-Object WinSCP.TransferOptions
        $transferOptions.TransferMode = [WinSCP.TransferMode]::Binary

       #$transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::Off
        $transferOptions.FilePermissions = $Null # This is default
        $transferOptions.PreserveTimestamp = $False
 
       #Upload file to the SFTP server directory     
       #Note use of absolute path
        $transferResult_CDCFH = $session.GetFiles($CDCFH_IN, $CDCFH_OUT, $False, $transferOptions)
                
       #Throw on any error to emulate "option batch abort"
        $transferResult_CDCFH.Check()
                
       #Printing output for CDC and FH
        $cdcfh_counter = 0
        foreach ($transfer in $transferResult_CDCFH.Transfers)
        {
            #Success or error?
            if ($transfer.Error -eq $Null)
            {
                #Print results
                Add-content -Path $logpath -value "Download of {0} succeeded $($transfer.FileName)"
                $cdcfh_counter++

                #Upload succeeded, delete source files
                Remove-Item $transfer.FileName $CDCFH_IN
            }
            else
            {
                #Upload Failed, throw error
                Add-content -Path $logpath -value "Upload failed: {1} $($transfer.FileName) $($transfer.Error.Message)"
            }
        }

        Add-content -Path $logPath -value "$($cdcfh_counter) files successfully downloaded from DIGNITY."
                        
    }
    finally
    {
       #Disconnect, clean up
        $session.Dispose()
        Add-content -Path $logpath -value "Download of Dignity demographics completed"
    }
 
    exit 0
}
catch [Exception]
{
    Add-content -path $logpath -value $_.Exception.Message
    exit 1
}

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,391
Location:: Prague, Czechia

Re: Help with SFTP Download Script

2016-06-23

Can you describe your problem better than "does not download the files where I need them"?

Can you attach a full session log file showing the problem (using the latest version of WinSCP)?

To generate the session log file, set Session.SessionLogPath. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

Guest

Re: Help with SFTP Download Script

2016-06-23 23:16

I've made some changes and I have some success. i will repost script here below. Now what I have is that I need to DELETE the text files on the remote server. I only want to delete the text files. There's a working directory *.working*. I want the script to ignore it and everything in it and only look at the text files. How do I get the script to do that? Script below with log files.

#06/15/2016 Created to download demographic files from Dignity via SFTP
#
try
{
    $logpath = "D:\Report-Copy-Move\Logs\CDCFH-demo_download_results.txt"

   #Source Folders, use absolute path
   #These folders must only have files that are to be transferred
    $CDCFH_IN = "/username/RDA/IN/CC/MS4/DEM/*.txt"
        
   #Target Folders, use absolute path
    $CDCFH_OUT = "\\transferserver\dignity-billing\agch_fhmc\*.txt"
        
   #Archive Folder Location, use absolute path
    #$CDCFH_Archive = "D:\ininbound\ftp-archive\cdc-archive\"
        
   #Load WinSCP .NET assembly
    Add-Type -Path "D:\Report-Copy-Move\SFTP\WinSCPnet.dll"
 
   #Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.Protocol = [WinSCP.Protocol]::sftp
    $sessionOptions.HostName = "mft.dignityhealth.org"
    $sessionOptions.UserName = "username"
    $sessionOptions.Password = "pa55w0rd"
    $sessionOptions.PortNumber = "4022"
    
   #Use the following line with the SSH HostKey Fingerprint by replacing the xx:xx:xx...
    $sessionOptions.SshHostKeyFingerprint = "ssh-rsa 1024 97:83:f1:ec:8c:c4:3b:78:29:d9:c5:10:c8:fe:93:08"

    
   #ONLY FOR TEST PURPOSES, DO NOT USE AS IS
   #$sessionOptions.GiveUpSecurityAndAcceptAnySshHostKey = true
   #$sessionOptions.GiveUpSecurityAndAcceptAnyTlsHostCertificate = true
    
    $session = New-Object WinSCP.Session
    $session.SessionLogPath = "D:\Report-Copy-Move\Logs\CDCFH-demo_download-session-log.txt"  
 
    try
    {
       #Connect
        $session.Open($sessionOptions)
 
       #Force binary mode transfer
        $transferOptions = New-Object WinSCP.TransferOptions
        $transferOptions.TransferMode = [WinSCP.TransferMode]::Binary

       #$transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::Off
        $transferOptions.FilePermissions = $Null # This is default
        $transferOptions.PreserveTimestamp = $False
 
       #Upload file to the SFTP server directory     
       #Note use of absolute path
        $transferResult_CDCFH = $session.GetFiles($CDCFH_IN, $CDCFH_OUT, $False, $transferOptions)
      Remove-Item $transfer.FileName $CDCFH_IN -inlcude * .txt -exclude *.working*
                
       #Throw on any error to emulate "option batch abort"
        $transferResult_CDCFH.Check()
                
       #Printing output for CDC and FH
        $cdcfh_counter = 0
        foreach ($transfer in $transferResult_CDCFH.Transfers)
        {
            #Success or error?
            if ($transfer.Error -eq $Null)
            {
                #Print results
                Add-content -Path $logpath -value "Download of {0} succeeded $($transfer.FileName)"
                $cdcfh_counter++

                #Upload succeeded, delete source files
            Invoke-Command {Remove-Item -path $CDCFH_IN[0] -Recurse} -ArgumentList
            #Get-ChildItem $CDCFH_IN -Recurse | Remove-Item -Recurse -Force
                #Remove-Item $transfer.FileName $CDCFH_IN -include *.txt
            }
            else
            {
                #Upload Failed, throw error
                Add-content -Path $logpath -value "Upload failed: {1} $($transfer.FileName) $($transfer.Error.Message)"
            }
        }

        Add-content -Path $logPath -value "$($cdcfh_counter) files successfully downloaded from DIGNITY."
                        
    }
    finally
    {
       #Disconnect, clean up
        $session.Dispose()
        Add-content -Path $logpath -value "Download of Dignity demographics completed"
    }
 
    exit 0
}
catch [Exception]
{
    Add-content -path $logpath -value $_.Exception.Message
    exit 1
}

SESSION LOG FILE:

CDCFH-demo_download_results.txt (106 Bytes)

CDCFH-demo_download-session-log.txt (68.3 KB)

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,391
Location:: Prague, Czechia

Re: Help with SFTP Download Script

2016-06-27

Anonymous wrote:

Now what I have is that I need to DELETE the text files on the remote server. I only want to delete the text files. There's a working directory *.working*. I want the script to ignore it and everything in it and only look at the text files. How do I get the script to do that?

So list the directory, filter the files you want and delete them.
Start with this example:
https://winscp.net/eng/docs/library_example_listing_files_matching_wildcard

Reply with quote

gina.bautista@gmail.com
Joined:: 2016-06-20
Posts:: 2
Location:: SLO, CA

SFTP Download Log File Blank and code 3 error

2016-09-15 18:44

Help! I have a WinSCP script with Powershell that is now downloading two text files from a remote host. It downloads to a local share. It deletes the files from the remote host. There are two log files, the session log and the results log. The session log is fine, but I see a code 3 error at the bottom and I don't know how to correct this problem. The results log is blank not giving me the data I expect. I am attaching the script here below. I will attach the two log files also. What am I doing wrong? What am I missing?
##
try
{
$logpath = "D:\Report-Copy-Move\Logs\DIGNITY-demo_download_results.txt"

#Source Folders, use absolute path
#These folders must only have files that are to be transferred
$Demos_French_Arroyo_IN ="/RDA_jkrupa/RDA/IN/CC/MS4/DEM/*.txt"

#Target Folders, use absolute path
$Demos_French_Arroyo_OUT = "\\transferserver\DIGNITY-Billing\agch_fhmc\"

#Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"

#Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions
$sessionOptions.Protocol = [WinSCP.Protocol]::sftp
$sessionOptions.HostName = "mft.dignityhealth.org"
$sessionOptions.UserName = "RDA_jkrupa"
$sessionOptions.Password = "xxxxxxxxx"
$sessionOptions.PortNumber = "4022"

#Use the following line with the SSH HostKey Fingerprint by replacing the xx:xx:xx...
#If unknown, obtain from log file
$sessionOptions.SshHostKeyFingerprint = "ssh-rsa 1024 97:83:f1:ec:8c:c4:3b:78:29:d9:c5:10:c8:fe:93:08"

$session = New-Object WinSCP.Session
$session.SessionLogPath = "D:\Report-Copy-Move\Logs\DIGNITY-demo_download-session-log.txt"

try
{
#Connect
$session.Open($sessionOptions)

#Force binary mode transfer
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.FileMask = "/RDA_jkrupa/RDA/IN/CC/MS4/DEM/*.txt"
$transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
$transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::Off
$transferOptions.FilePermissions = $Null # This is default
$transferOptions.PreserveTimestamp = $False
$session.GetFiles("/RDA_jkrupa/RDA/IN/CC/MS4/DEM/*.txt", "\\transferserver\DIGNITY-BILLING\agch_fhmc\*.txt", $True, $transferOptions)

# Download files to the server directory
# Note use of absolute path
$transferResult_Demos_French_Arroyo = $session.PutFiles($Demos_French_Arroyo_IN, $Demos_French_Arroyo_OUT, $True, $transferOptions)

# Throw on any error to emulate "option batch abort"
# $transferResult_Demos_French_Arroyo.Check()

# Printing output for Demos_French_Arroyo
$Demos_French_Arroyo_counter = 0
foreach ($transfer in $transferResult_Demos_French_Arroyo.Transfers)
{
# Success or error?
if ($transfer.Error -eq $Null)
{
# Print results
Add-content -Path $logpath -value ("Download of {0} succeeded, removing from source" -f
$transfer.FileName)
$Demos_French_Arroyo_counter++

# Download succeeded, remove file from source
$removalResult = $session.RemoveFiles($session.EscapeFileMask($transfer.FileName))

if ($removalResult.IsSuccess)
{
Add-Content -Path $logpath -value ("Removing of file {0} succeeded" -f
$transfer.FileName)
}
else
{
Add-Content -Path $logpath -value ("Removing of file {0} failed" -f
$transfer.FileName)
}
}
else
{
Add-Content -Path $logpath -value ("Download of {0} failed: {1}" -f
$transfer.FileName, $transfer.Error.Message)
}
}
}
finally
{
#Disconnect, clean up
$session.Dispose()
Add-content -Path $logpath -value "Download of Dignity Demographic Files Completed"
}

exit 0
}
catch [Exception]
{
Add-content -path $logpath -value $_.Exception.Message
exit 1
}
##

DIGNITY-demo_download-session-log.txt (44.97 KB)

DIGNITY-demo_download_results.txt (147 Bytes)

Description: This is the file that is blank with no line item data in it. I expect the "foreach" to write here.

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,391
Location:: Prague, Czechia

Re: SFTP Download Log File Blank and code 3 error

2016-09-19

You are effectively calling:

$session.PutFiles("/RDA_jkrupa/RDA/IN/CC/MS4/DEM/*.txt", "\\transferserver\DIGNITY-Billing\agch_fhmc\")

Didn't you want to call this instead?

$session.PutFiles("\\transferserver\DIGNITY-Billing\agch_fhmc\*.txt", "/RDA_jkrupa/RDA/IN/CC/MS4/DEM/")

Reply with quote

Help with SFTP Download Script