Filemask to sync on folders named by date (mmdd) :: Support Forum

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Filemask to sync on folders named by date (mmdd)

2021-02-24 11:37

Hello guys,

first of all, thanks for this wonderful client.

I'm trying to apply a filemask to sync a webdav site to local folders based on the names of the folders. Each folder is named with date in mmdd format (for ex, today's folder is 0224).
The complete structure of one of the path is this:
TME_12345678911_12345678911/yyyy/mmdd

The part until the first backlash never changes and there are many of them, then there are subfolders for years in yyyy format and for days in mmdd format.

This is the structure of one of the paths I have to deal with:
Root: /remote.php/webdav
Folder: TME_12345678911_12345678911/
Subfolders for year: 2017, 2018, 2019, 2020, 2021
Subfolders for day: all days per each year until today

My goal is to apply a filemask to search for any new file but only subfolders for the last 5 days based on current date.

Ex. Current date = 2021/02/24
Folder: TME_12345678911_12345678911/ (and all the other n folders)
Year: 2021
Day: 0224, 0223, 0222, 0221, 0220

Is there a way to do it?
Thanks in advance,

Andrea

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Filemask to sync on folders named by date (mmdd)

2021-02-26

Is the question about WinSCP scripting? Or GUI? Or .NET assembly?

Reply with quote

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Re: Filemask to sync on folders named by date (mmdd)

2021-02-26 09:59

Hello Martin,

I'm trying to apply the mask on the GUI and then also generate a code to use it in a batch file.

Thanks in advance,
Andrea

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Filemask to sync on folders named by date (mmdd)

2021-02-26

It's not possible in the GUI.

In the script, it's possible, but quite complicated. You can use something like:

-filemask="/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/%TIMESTAMP#mmdd%; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/%TIMESTAMP-1D#mmdd%; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/%TIMESTAMP-2D#mmdd%; ..."

If you have to deal with last years files at the beginning of the new year, you have to add another set of 5 day masks with .../%TIMESTAMP-5D#yyyy%/....

See:
https://winscp.net/eng/docs/file_mask
https://winscp.net/eng/docs/scripting#timestamp

Reply with quote

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Re: Filemask to sync on folders named by date (mmdd)

2021-02-26 15:42

What am I doing wrong? I've started to try to get current date's folder files, then I'll add the other five but with this I get

Error listing directory '/remote.php/webdav/TME_*/2021'.

ECHO Date: %DATE%
ECHO Time: %TIME%
 
SET _datetime=%DATE:~6,4%%DATE:~3,2%%DATE:~0,2%
SET _day=%DATE:~0,2%
SET _month=%DATE:~3,2%
SET _year=%DATE:~6,4%
ECHO %_datetime%
ECHO %_day%
ECHO %_month%
ECHO %_year%
 
    "get -filemask=/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/%_year%/; /remote.php/webdav/TME_*/%_year%/%_month%%_day%; -neweronly *" ^

Thanks again,
Andrea

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Filemask to sync on folders named by date (mmdd)

2021-02-26

I've forgot the quotes. I've edited my post. Though as you seem to specify the command on WinSCP command-line directly in the batch file, you need to double them:

    "get -filemask=""/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/%_year%/; /remote.php/webdav/TME_*/%_year%/%_month%%_day%;"" -neweronly *" ^

Also, why do you use the %DATE% instead of my proposed solution with %TIMESTAMP%? The %DATE% is unreliable and it will hardly help you calculating the previous date, what you need for your task. Maybe you had problems with the syntax in the batch file, as you need to double the %. Read the documentation: https://winscp.net/eng/docs/scripting#timestamp

Side note: The -neweronly makes little sense in this context.

Reply with quote

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Re: Filemask to sync on folders named by date (mmdd)

2021-03-01 09:25

This is my new string based on your indications (inserted timestamp and removed -neweronly):

"get -filemask=""/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/; /remote.php/webdav/TME_*/%TIMESTAMP#yyyy%/%TIMESTAMP#mmdd%;"" *" ^

I don't get the error anymore but the it starts looking into all subfolders (the oldest one for year is 2017 and the script starts from it instead of 2021), so I still probably missing or doing wrong.

Thanks again Martin,
Andrea

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Filemask to sync on folders named by date (mmdd)

2021-03-02

As I wrote above:

Maybe you had problems with the syntax in the batch file, as you need to double the %. Read the documentation: https://winscp.net/eng/docs/scripting#timestamp

So it should be:

  "get -filemask=""/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/%%TIMESTAMP#yyyy%%/; /remote.php/webdav/TME_*/%%TIMESTAMP#yyyy%%/%%TIMESTAMP#mmdd%%;"" *" ^

If you still have problems, post a session log file.

Reply with quote

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Re: Filemask to sync on folders named by date (mmdd)

2021-03-02 10:52

I've tried with the double % but nothing changes. I checked the log and the mask seems correct

> 2021-03-02 10:32:25.309 Script: get -filemask="/remote.php/webdav/TME_*/; /remote.php/webdav/TME_*/2021/; /remote.php/webdav/TME_*/2021/0302;" *

There must be something else I'm doing wrong. Can I send you the log privately? Is there a way to hide sensitive data (links, username or passwords) in the log file?

Thank you very much.
Andrea

Reply with quote

martin◆ Site Admin

Re: Filemask to sync on folders named by date (mmdd)

2021-03-02

You can attach the log privately here. The log does not contain passwords. Remove anything else you do not want to share.

Reply with quote

andrbazz
Joined:: 2018-03-05
Posts:: 8
Location:: Italy

Re: Filemask to sync on folders named by date (mmdd)

2021-03-03 09:51

Here is the log, thanks again Martin.
Andrea

Log.log (358.2 KB, Private file)

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Filemask to sync on folders named by date (mmdd)

2021-03-05

Sorry, I was wrong. It indeed cannot work this way.
It's difficult and unreliable to use plain WinSCP scripting for this complicated task.
Consider switching to PowerShell and WinSCP .NET assembly.
https://winscp.net/eng/docs/library
In PowerShell, you can easily iterate your TME_* folders, triggering a download for yyyy/mmdd/* folders for the past files days in each.

Reply with quote