Iterating through multiple folders while connected :: Support Forum

Vann_the_Red
Joined:: 2020-01-21
Posts:: 27
Location:: NC, USA

Iterating through multiple folders while connected

2020-06-03 21:21

Hello, Martin! Hope you are well.

I have been using one of your example scripts (https://winscp.net/eng/docs/script_local_move_after_successful_upload) for some time with great success. I have a new implementation, however, that requires I follow that logic for a series of folders on one site. The workflow is connect, then for each folder in a list, download the files in that folder and, upon successful download, archive them to a separate folder and then proceed to the next folder in the list. A sample config would look like this:

#list of source folders, download folders, and archive folders:

/user/fromFolder1, c:/ftp/downloads/toFolder1, /user/archive/archiveFolder1
/user/fromFolder2, c:/ftp/downloads/toFolder2, /user/archive/archiveFolder2
/user/fromFolder3, c:/ftp/downloads/toFolder3, /user/archive/archiveFolder3
/user/fromFolder4, c:/ftp/downloads/toFolder4, /user/archive/archiveFolder4
/user/fromFolder5, c:/ftp/downloads/toFolder5, /user/archive/archiveFolder5
/user/fromFolder6, c:/ftp/downloads/toFolder6, /user/archive/archiveFolder6
/user/fromFolder7, c:/ftp/downloads/toFolder7, /user/archive/archiveFolder7
/user/fromFolder8, c:/ftp/downloads/toFolder8, /user/archive/archiveFolder8

Do you have any examples of using a similar configuration I could base my script on?

Thank you!

VtR

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,378
Location:: Prague, Czechia

Re: Iterating through multiple folders while connected

2020-06-05

It's not really WinSCP question, but a generic PowerShell programming question. Anyway this will work:

$foldersList = Import-Csv -Header ("Source", "Dest", "Archive") "paths.csv"
 
foreach ($folders in $foldersList)
{
    # ...
}

Into the loop, put the $session.PutFiles call and foreach ($transfer in $transferResult.Transfers) loop from the example code. Replace $localPath, $remotePath and $backupPath with $folders.Source, $folders.Dest and $folders.Archive.

Though the example is for upload, while you download, so adjust the mapping accordingly.

Reply with quote

Vann_the_Red
Joined:: 2020-01-21
Posts:: 27
Location:: NC, USA

2020-06-05 14:22

Martin,

You are a tremendous help and WinSCP is a fantastic product. I did not mean to imply this was a shortfall in WinSCP. I was just hoping there was a script somewhere that implemented something similar I could use as a template. Once I have it tested, would it be helpful for me to upload it here so others might use it as a template? If so, I'm happy to help.

Regards,

VtR

Reply with quote

Vann_the_Red
Joined:: 2020-01-21
Posts:: 27
Location:: NC, USA

2020-06-08 21:03

Martin,

I would like to download to a temp directory then move to the final directory once we confirm a successful download. Below is the "Success or error" I am currently using.

# Iterate over every transfer
foreach ($transfer in $transferResult.Transfers)
{
    # Success or error?
    if ($transfer.Error -eq $Null)
    {
        Write-Host "Upload of $($transfer.FileName) succeeded, moving to backup"
        # Download succeded, move source file to backup
        $session.MoveFile($transfer.FileName, $backupPath)
    }
    else
    {
        Write-Host "Upload of $($transfer.FileName) failed: $($transfer.Error.Message)"
    }
}

Is this the proper addition to move the freshly downloaded file to a different directory on the same server?

# Iterate over every transfer
foreach ($transfer in $transferResult.Transfers)
{
    # Success or error?
    if ($transfer.Error -eq $Null)
    {
        Write-Host "Upload of $($transfer.FileName) succeeded, moving to backup"
        # Download succeded, move source file to backup and download from temp to final
        $session.MoveFile($transfer.FileName, $backupPath)
        Move-Item $transfer.FileName $finalPath
    }
    else
    {
        Write-Host "Upload of $($transfer.FileName) failed: $($transfer.Error.Message)"
    }
}

Thanks,

VtR

Reply with quote

martin◆ Site Admin

2020-06-09

Vann_the_Red wrote:

Is this the proper addition to move the freshly downloaded file to a different directory on the same server?

That's bit ambiguous question. If you want to move the downloaded local file to another folder on the same local machine, then use Move-Item.

Reply with quote

Vann_the_Red
Joined:: 2020-01-21
Posts:: 27
Location:: NC, USA

2020-06-09 21:45

Yes, your assumption that I wanted to move the file from the local temp directory to the local final directory is correct. As configured, though, Move-Item is looking for the remote path rather than the local path.

# Iterate over every transfer
foreach ($transfer in $transferResult.Transfers)
{
    # Success or error?
    if ($transfer.Error -eq $Null)
    {
        Write-Host "Upload of $($transfer.FileName) succeeded, moving to backup"
        # Download succeded, move source file to backup and download from temp to final
        $session.MoveFile($transfer.FileName, $folders.backupPath)
        Move-Item $transfer.FileName $folders.finalPath
    }
    else
    {
        Write-Host "Upload of $($transfer.FileName) failed: $($transfer.Error.Message)"
    }
}

It seems I need to change the location $trasfer.FileName to the $folders.downloadPath while retaining the FileName. I'm researching. Once I get the script working, I'm happy to upload it for others.

Regards,

VtR

Reply with quote

Vann_the_Red
Joined:: 2020-01-21
Posts:: 27
Location:: NC, USA

2020-06-11 21:36

Martin and community,

I've got to admit I'm stumped at this point. Using the code displayed above, I get an error stating that it cannot move the file located at "C:\remotePath\correctFileName.csv". So, I figured $transfer.FileName must hold the full path it downloaded from and entered:

$fileName = Split-Path -Path $transfer.FileName -Leaf -Resolve
Move-Item "$folders.localPath\$fileName" $folders.finalPath

This gives me an error stating that Split-Path had an object not found.

"+ CategoryInfo : ObjectNotFound: (C:\dtl_ftp\bin...ntArgs.FileName:String) [Split-Path], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.SplitPathCommand"

Has anyone successfully parsed a filename out of an object like $transfer.FileName? Is that a FileInfo object that needs to be type cast into a string for this purpose?

In summation, I know the file is in $folders.localPath (I know this programmatically and the transfer from remotePath to localPath succeeds before this line of the code so the file is there). I know the name of it is contained in $transfer.FileName (I know this from looking at the error message powershell logs). I need to concatenate those two things together to form the first argument for the Move-Item command.

Thanks in advance,
VtR

Reply with quote