2024.7.24.0

YT
YouTubeSettings: add 'DefaultVideoConvertNonAVC' property; fix 'OpenFolderInOtherProgram' property serialize and reset; add 'MusicPlaylistCreate_CreationMode' property
Update the 'CleanFileName' function to remove line breaks
Add the ability to convert non-avc codecs to avc
Add 'M3U8CreationMode' enum

SCrawler
API.OnlyFans: fix incorrect delimiter (rules parsing)
API.Threads: add saved posts downloading
Feed: add the hotkeys 'Esc' and 'Ctrl+W' to close the form; add the ability to search for missing files in special feeds
Scheduler: add the ability to execute a script after the scheduler plan is executed
Settings: add enable/disable the use of the 'Esc' to close the feed; add 'AutomationScript' and 'AutomationScript_ExcludeManual' properties
MainFrame: add the hotkey 'Ctrl+F' to show the feed; change the hotkey from 'Ctrl+F' to 'Alt+F' to show the search form

Changelog.md

@@ -1,3 +1,173 @@
+# 2024.7.24.0
+
+*2024-07-24*
+
+- Added
+  - YouTube (standalone app)
+    - ability to convert non-`AVC` codecs (eg `VP9`) to `AVC` (`Settings` - `Defaults Video` - `Convert non-AVC codecs to AVC`)
+    - add the ability to set the playlist creation mode: absolute links, relative links, or both (`Settings` - `Music` - `Create M3U8: creation mode`)
+  - Threads: **saved posts downloading**
+  - Feed
+    - hotkeys `Esc` and `Ctrl+W` to close the form
+    - the ability to search for missing files in *special feeds*
+  - Scheduler: the ability to execute a script after the scheduler plan is executed *(`Settings` - `Behavior`)*
+  - Main window:
+    - added hotkey `Ctrl+F` to show the feed
+    - changed the hotkey from `Ctrl+F` to `Alt+F` to show the search form
+- Updated
+  - yt-dlp up to version **2024.07.16**
+- Fixed
+  - YouTube (standalone app): video files with line breaks in the name do not download correctly
+  - OnlyFans: rules parsing bug
+  - Minor bugs
+
+# 2024.6.25.0
+
+*2024-06-25*
+
+**ATTENTION! To support downloading of DRM protected videos (OnlyFans), please update OF-Scraper to version [3.10.7](https://github.com/datawhores/OF-Scraper/releases/tag/3.10.7) (download `zip`, not `exe`).**
+
+- Added
+  - OnlyFans: **new dynamic rules updating algorithm**
+  - Feed: ability to set the last session as the current one
+- Updated
+  - gallery-dl up to version **1.27.1**
+- Fixed
+  - Minor bugs
+
+# 2024.6.10.0
+
+*2024-06-10*
+
+- Added
+  - YouTube (standalone app): add option to add extracted MP3 to playlist (`Settings` - `Defaults Video` - `Add extracted MP3 to playlist`)
+  - Feed
+    - settings to show/hide site name and file type from media title
+    - ability to move/copy files of a loaded feed/session to another location
+    - ability to reset current session
+- Fixed
+  - Minor bugs
+
+# 2024.6.6.0
+
+*2024-06-06*
+
+**ATTENTION!**
+1. **To support downloading of DRM protected videos (OnlyFans), please update OF-Scraper to version [3.10](https://github.com/datawhores/OF-Scraper/releases/tag/3.10) (download `zip`, not `exe`).**
+2. **If there is a `OFScraperConfigPattern.json` file in the SCrawler settings folder, replace the text of the file with [this text](https://github.com/AAndyProgram/SCrawler/blob/main/SCrawler/API/OnlyFans/OFScraperConfigPattern.json).**
+3. **Set the value to `Dynamic rules` (in the site settings) = `https://raw.githubusercontent.com/Growik/onlyfans-dynamic-rules/main/rules.json`.**
+
+- Added
+  - OnlyFans: new OF-Scraper option (`keydb_api`)
+  - Minor improvements
+- Fixed
+  - OnlyFans: **data is not downloading**
+  - Minor bugs
+
+# 2024.6.4.0
+
+*2024-06-04*
+
+**If you were using the [`yt-dlp-TTUser`](https://github.com/bashonly/yt-dlp-TTUser) plugin, you should remove it because this plugin was added to yt-dlp itself! Read more [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok-requirements).**
+
+- Added
+  - Added highlighting of scheduler plans (working, stopped, pending, etc.)
+  - YouTube (standalone app): add option to add the video upload date before/after the file name (`Settings` - `Defaults` - `Add date to file name`)
+  - Twitter: **`Communities` downloading**
+  - Feed: ability to select one of the download sessions and set it as the current session
+  - Minor improvements
+- Updated
+  - yt-dlp up to version **2024.05.27**
+  - gallery-dl up to version **1.27.0**
+- Fixed
+  - Twitter: deleting user directory when redownloading missing posts
+  - Minor bugs
+
+# 2024.5.19.0
+
+*2024-05-19*
+
+- Added
+  - YouTube (standalone app): add upload date to description (request #192) (`Settings` - `Info` - `Create description files: add upload date`, `Create description files: create without description`).
+- Fixed
+  - YouTube (SCrawler): advanced settings are not saved when changed
+
+# 2024.5.18.0
+
+*2024-05-18*
+
+- Added
+  - YouTube (standalone app): highlight frame rates higher/lower than this value (`Settings` - `Defaults Video` - `Highlight FPS (higher/lower)`).
+  - Sites
+    - Instagram: 'DownDetector' support to determine if the site is accessible
+    - Reddit: change the naming method of video files (hosted on Reddit) to the `YYYYMMDD_HHMMSS` pattern
+    - Twitter
+      - `Likes` downloading *(user settings)*
+      - **changed domain from twitter.com to x.com**
+  - Site settings: group options by category
+  - Minor improvements
+- PluginProvider
+  - `PropertyOption` attribute: set category name when `IsAuth = True`
+  - `ISiteSettings`: added `UserAgentDefault` property
+- Updated
+  - gallery-dl up to version **1.27.0-dev**
+- Fixed
+  - Sites
+    - Instagram: incorrect definition of pinned posts
+    - Threads: new posts are no longer downloaded from profiles with pinned posts
+    - Reddit: bypass error 429 for saved posts
+    - Twitter: **data is not downloading due to domain change from twitter.com to x.com**
+  - Minor bugs
+
+# 2024.5.4.0
+
+*2024-05-04*
+
+- Added
+  - YouTube (standalone app): setting to remove specific characters (`Defaults` - `Remove characters`)
+  - Instagram: simplify the `Connection closed` error
+  - Users search: add `Friendly name` to search results
+- Fixed
+  - YouTube (standalone app): incorrect download processing when the file name ends with a dot (Issue #188)
+  - The program is freezes when editing users in some cases
+  - Sites
+    - Reddit: token update error
+    - Threads: unable to obtain credentials (`ID`)
+
+# 2024.4.26.0
+
+*2024-04-26*
+
+- Added
+  - Site settings: the values that can be extracted from cookies immediately populate fields
+  - Feed: ability to load the last session of the current day (if it exists) as the current session after restarting SCrawler
+  - Users search: include friendly name matches in search result
+- Updated
+  - gallery-dl up to version **1.26.9**
+- Fixed
+  - xHamster: **saved posts aren't downloading**
+
+# 2024.4.14.0
+
+*2024-04-14*
+
+- Fixed
+  - Facebook: can't get tokens
+
+# 2024.4.13.0
+
+*2024-04-13*
+
+- Added
+  - Minor improvements
+- PluginProvider
+  - IPluginContentProvider: added `ResetHistoryData` function
+- Fixed
+  - Sites
+    - TikTok: remove last download date when erasing history data
+    - YouTube: remove last download date when erasing history data
+    - Instagram: **saved posts aren't downloading**
+
 # 2024.4.10.0
 
 *2024-04-10*

README.md

@@ -1,5 +1,5 @@
-<!-- # :rainbow_flag: Happy LGBT Pride Month :tada:
--->
+# 🏳️‍🌈 Happy LGBT Pride Month 🎉
+
 # 🏳️‍🌈 Social networks crawler 🏳️‍🌈
 
 [![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)
@@ -33,16 +33,16 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 ![YouTube application](ProgramScreenshots/AppYouTube.png)
 
 # What can program do:
-- Download pictures and videos from users' profiles and subreddits:
+- Download pictures and videos from user profiles:
   - YouTube videos, shorts, community feeds, users, artists, playlists, music, tracks;
   - Reddit images, galleries of images, videos, saved posts;
-  - Redgifs videos (https://www.redgifs.com/);
-  - Twitter images and videos, saved (bookmarked) posts;
-  - OnlyFans images and videos, saved (bookmarked) posts;
+  - Redgifs images and videos (https://www.redgifs.com/);
+  - Twitter images and videos, saved (bookmarked) posts, likes, communities;
+  - OnlyFans images and videos, saved (bookmarked) posts, stories;
   - JustForFans images and videos, saved (bookmarked) posts;
   - Mastodon images and videos, saved (bookmarked) posts;
   - Instagram images and videos, tagged posts, stories, saved posts;
-  - Threads images and videos;
+  - Threads images and videos, saved posts;
   - Facebook images and videos, stories, saved posts;
   - TikTok videos;
   - Pinterest boards, users, saved posts;
@@ -57,7 +57,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - Download [saved posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts)
 - Add users from parsed channel
 - **Advanced user management**
-- **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every ```X``` minutes)
+- **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every `X` minutes)
 - **Feed** ([feed](https://github.com/AAndyProgram/SCrawler/wiki#feed) of downloaded media files and subscriptions posts)
 - Multiple accounts support
 - Labeling users
@@ -79,15 +79,15 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **Reddit**
 - **Twitter**
 - **OnlyFans** *(partial support)*[^1]
-- **Mastodon**
 - **Instagram**
 - **Threads**
 - **Facebook**
 - JustForFans *(partial support)*[^1]
+- Mastodon *(out of support)*
 - TikTok
 - RedGifs
 - Pinterest
-- Imgur
+- Imgur *(out of support)*
 - Gfycat
 - LPSG
 - **PornHub**
@@ -109,7 +109,7 @@ First, the program downloads the full profile. After the program downloads only
 
 # Requirements
 
-- Windows 10, 11 with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
+- **Windows 10, 11** with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
 - **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 
 # Guide
@@ -157,7 +157,7 @@ First, the program downloads the full profile. After the program downloads only
 
 **Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder and enjoy.** :blush:
 
-**Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder)**
+**Don't put program in the `Program Files` system folder (this is portable program and program settings are stored in the program folder)**
 
 **I highly doubt you can run SCrawler on Linux or Mac. SCrawler is a program that is heavily dependent on Windows.**
 
@@ -183,7 +183,7 @@ The program has an intuitive interface.
 
 [![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
 
-Just add a user profile and **click the ```Download``` button**.
+Just add a user profile and **click the `Download` button**.
 
 ```mermaid
 stateDiagram

SCrawler.PluginProvider/Attributes/Attributes.vb

@@ -36,8 +36,22 @@ Namespace Plugin.Attributes
         Public Property IsInformationLabel As Boolean = False
         ''' <summary>Label text alignment.<br/>Default: <see cref="Drawing.ContentAlignment.TopCenter"/></summary>
         Public Property LabelTextAlign As Drawing.ContentAlignment = Drawing.ContentAlignment.TopCenter
+        Private _IsAuth As Boolean = False
         ''' <summary>This is an authorization property</summary>
-        Public Property IsAuth As Boolean = False
+        Public Property IsAuth As Boolean
+            Get
+                Return _IsAuth
+            End Get
+            Set(ByVal _IsAuth As Boolean)
+                Me._IsAuth = _IsAuth
+                If _IsAuth And String.IsNullOrEmpty(Category) Then
+                    Category = CategoryAuth
+                ElseIf Not _IsAuth AndAlso Not String.IsNullOrEmpty(Category) AndAlso Category = CategoryAuth Then
+                    Category = String.Empty
+                End If
+            End Set
+        End Property
+        Public Const CategoryAuth As String = "Authorization"
         Public Property Category As String = Nothing
         Public Property InheritanceName As String = Nothing
         ''' <summary>Initialize a new property option attribute</summary>

SCrawler.PluginProvider/Interfaces/IPluginContentProvider.vb

@@ -40,5 +40,6 @@ Namespace Plugin
         Sub GetMedia(ByVal Token As Threading.CancellationToken)
         Sub Download(ByVal Token As Threading.CancellationToken)
         Sub DownloadSingleObject(ByVal Data As IDownloadableMedia, ByVal Token As Threading.CancellationToken)
+        Sub ResetHistoryData()
     End Interface
 End Namespace

SCrawler.PluginProvider/Interfaces/ISiteSettings.vb

@@ -19,6 +19,7 @@ Namespace Plugin
         ReadOnly Property Site As String
         Property CMDEncoding As String
         Property EnvironmentPrograms As IEnumerable(Of String)
+        Property UserAgentDefault As String
         Sub EnvironmentProgramsUpdated()
         Property AccountName As String
         Property Temporary As Boolean

SCrawler.PluginProvider/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2024.4.10.0")>
-<Assembly: AssemblyFileVersion("2024.4.10.0")>
+<Assembly: AssemblyVersion("2024.5.18.0")>
+<Assembly: AssemblyFileVersion("2024.5.18.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Base/Structures.vb

@@ -78,6 +78,17 @@ Namespace API.YouTube.Base
         https = 1
         m3u8 = 2
     End Enum
+    <Editor(GetType(EnumDropDownEditor), GetType(UITypeEditor))>
+    Public Enum FileDateMode As Integer
+        None = 0
+        Before = 1
+        After = 2
+    End Enum
+    Public Enum M3U8CreationMode As Integer
+        Relative = 0
+        Absolute = 1
+        Both = 2
+    End Enum
     Public Structure MediaObject : Implements IIndexable, IComparable(Of MediaObject)
         Public Type As Plugin.UserMediaTypes
         Public ID As String

SCrawler.YouTube/Base/YouTubeFunctions.vb

@@ -22,6 +22,7 @@ Namespace API.YouTube.Base
         End Sub
         Public Shared Function StandardizeURL(ByVal URL As String) As String
             Try
+                URL = URL.StringTrim
                 Dim isMusic As Boolean = False, isShorts As Boolean = False
                 If Info_GetUrlType(URL, isMusic, isShorts) = YouTubeMediaType.Single Then
                     If Not isMusic And Not isShorts Then
@@ -45,6 +46,7 @@ Namespace API.YouTube.Base
         End Function
         Public Shared Function StandardizeURL_Channel(ByVal URL As String, Optional ByVal Process As Boolean = True) As String
             Try
+                URL = URL.StringTrim
                 Dim ct As YouTubeChannelTab = YouTubeChannelTab.All
                 Dim isMusic As Boolean = False
                 If Process AndAlso Info_GetUrlType(URL, isMusic,,,, ct) = YouTubeMediaType.Channel AndAlso Not isMusic Then
@@ -72,6 +74,7 @@ Namespace API.YouTube.Base
         Public Shared Function Info_GetUrlType(ByVal URL As String, Optional ByRef IsMusic As Boolean = False, Optional ByRef IsShorts As Boolean = False,
                                                Optional ByRef IsChannelUser As Boolean = False, Optional ByRef Id As String = Nothing,
                                                Optional ByRef ChannelOptions As YouTubeChannelTab = YouTubeChannelTab.All) As YouTubeMediaType
+            URL = URL.StringTrim
             If Not URL.IsEmptyString Then
                 IsMusic = URL.Contains("music.youtube.com")
                 IsChannelUser = False
@@ -118,6 +121,7 @@ Namespace API.YouTube.Base
                                      Optional ByVal Token As Threading.CancellationToken = Nothing, Optional ByVal Progress As IMyProgress = Nothing,
                                      Optional ByVal DateAfter As Date? = Nothing, Optional ByVal DateBefore As Date? = Nothing,
                                      Optional ByVal ChannelOption As YouTubeChannelTab? = Nothing, Optional ByVal UrlAsIs As Boolean = False) As IYouTubeMediaContainer
+            URL = URL.StringTrim
             If URL.IsEmptyString Then Throw New ArgumentNullException("URL", "URL cannot be null")
             If Not MyYouTubeSettings.YTDLP.Value.Exists Then Throw New IO.FileNotFoundException("Path to 'yt-dlp.exe' not set or program not found at destination", MyYouTubeSettings.YTDLP.Value.ToString)
             Dim urlOrig$ = URL
@@ -162,7 +166,7 @@ Namespace API.YouTube.Base
 
                 If result Then
                     container.Parse(Nothing, _CachePathDefault, isMusic, Token, Progress)
-                    If Not container.HasError Then container.URL = URL : container.IsShorts = isShorts : Return container
+                    If Not container.HasError Then container.URL = URL.ToMusicUrl(isMusic) : container.IsShorts = isShorts : Return container
                 End If
                 container.Dispose()
             End If

SCrawler.YouTube/Base/YouTubeSettings.vb

@@ -131,22 +131,28 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible(False), Category("EnvironmentFolder"), DisplayName("Open folders in another program"), DefaultValue(False)>
         Private Property IDownloaderSettings_OpenFolderInOtherProgram As Boolean Implements IDownloaderSettings.OpenFolderInOtherProgram
             Get
-                Return OpenFolderInOtherProgram.Use
+                Return OpenFolderInOtherProgram.Attribute.ValueTemp
             End Get
             Set(ByVal use As Boolean)
-                OpenFolderInOtherProgram.Use = use
+                OpenFolderInOtherProgram.Attribute.ValueTemp = use
             End Set
         End Property
+        Private Function ShouldSerializeIDownloaderSettings_OpenFolderInOtherProgram() As Boolean
+            Return DirectCast(OpenFolderInOtherProgram.Attribute, IGridValue).ShouldSerializeValue
+        End Function
         <Browsable(True), GridVisible(False), Category("EnvironmentFolder"), DisplayName("Open folders in another program (command)"),
             Description("The command to open a folder."), DefaultValue("")>
         Private Property IDownloaderSettings_OpenFolderInOtherProgram_Command As String Implements IDownloaderSettings.OpenFolderInOtherProgram_Command
             Get
-                Return OpenFolderInOtherProgram
+                Return OpenFolderInOtherProgram.ValueTemp
             End Get
             Set(ByVal command As String)
-                OpenFolderInOtherProgram.Value = command
+                OpenFolderInOtherProgram.ValueTemp = command
             End Set
         End Property
+        Private Function ShouldSerializeIDownloaderSettings_OpenFolderInOtherProgram_Command() As Boolean
+            Return DirectCast(OpenFolderInOtherProgram, IGridValue).ShouldSerializeValue
+        End Function
         <Browsable(True), GridVisible(False), XMLVN({"Environment"}, True), Category("Environment"), DisplayName("Check new version at start")>
         Friend ReadOnly Property CheckUpdatesAtStart As XMLValue(Of Boolean)
 #End Region
@@ -162,6 +168,12 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible, XMLVN({"Info"}), Category("Info"), DisplayName("Create description files"),
             Description("Create video description files. Default: false.")>
         Public ReadOnly Property CreateDescriptionFiles As XMLValue(Of Boolean)
+        <Browsable(True), GridVisible, XMLVN({"Info"}, True), Category("Info"), DisplayName("Create description files: add upload date"),
+            Description("Add the upload date to the top of the description file. Default: true.")>
+        Public ReadOnly Property CreateDescriptionFiles_AddUploadDate As XMLValue(Of Boolean)
+        <Browsable(True), GridVisible, XMLVN({"Info"}, True), Category("Info"), DisplayName("Create description files: create without description"),
+            Description("Create a description file with the upload date, even if the description does not exist. Default: true.")>
+        Public ReadOnly Property CreateDescriptionFiles_CreateWithNoDescription As XMLValue(Of Boolean)
         <Browsable(True), GridVisible, XMLVN({"Info"}, True), Category("Info"), DisplayName("Create thumbnail files (video)"),
             Description("Create video thumbnail files. Default: true.")>
         Public ReadOnly Property CreateThumbnails_Video As XMLValue(Of Boolean)
@@ -243,6 +255,18 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible(False), XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Program description"),
             Description("Add some additional info to the program info if you need")>
         Friend ReadOnly Property ProgramDescription As XMLValue(Of String)
+        <Browsable(True), GridVisible, XMLVN({"Defaults"}, "%"""), Category("Defaults"), DisplayName("Remove characters"),
+            Description("Remove specific characters from a file name")>
+        Public ReadOnly Property FileRemoveCharacters As XMLValue(Of String)
+        <Browsable(True), GridVisible, XMLVN({"Defaults"}, FileDateMode.None), Category("Defaults"), DisplayName("Add date to file name"),
+            Description("Add the video upload date before/after the file name")>
+        Public ReadOnly Property FileAddDateToFileName As XMLValue(Of FileDateMode)
+        <Browsable(True), GridVisible, XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Add date to title: video form"),
+            Description("Add video upload date before video title (visual only) in the video form")>
+        Public ReadOnly Property FileAddDateToFileName_VideoForm As XMLValue(Of Boolean)
+        <Browsable(True), GridVisible, XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Add date to title: video list"),
+            Description("Add video upload date before video title (visual only) in the video list")>
+        Public ReadOnly Property FileAddDateToFileName_VideoList As XMLValue(Of Boolean)
 #End Region
 #Region "Defaults ChannelsDownload"
         <Browsable(True), GridVisible, XMLVN({"Defaults", "Channels"}), Category("Defaults"), DisplayName("Default download tabs for channels"),
@@ -287,6 +311,9 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}, 1080), Category("Defaults Video"), DisplayName("Default definition"),
             Description("The default maximum video resolution. -1 for max definition")>
         Public ReadOnly Property DefaultVideoDefinition As XMLValue(Of Integer)
+        <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}), Category("Defaults Video"), DisplayName("Convert non-AVC codecs to AVC"),
+            Description("Convert non-AVC codecs (eg 'VP9') to AVC. Not recommended due to high CPU usage!")>
+        Public ReadOnly Property DefaultVideoConvertNonAVC As XMLValue(Of Boolean)
         <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}, False), Category("Defaults Video"), DisplayName("Embed thumbnail (video)"),
             Description("Embed thumbnail in the video as cover art. Default: true.")>
         Public ReadOnly Property DefaultVideoEmbedThumbnail As XMLValue(Of Boolean)
@@ -358,6 +385,15 @@ Namespace API.YouTube.Base
                 Throw New NotImplementedException("'GetFormat' is not available in 'FpsFormatProvider'")
             End Function
         End Class
+        <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}, 30), Category("Defaults Video"), DisplayName("Highlight FPS (higher)"),
+            Description("Highlight frame rates higher than this value." & vbCr & "Default: 30" & vbCr & "-1 to disable")>
+        Public ReadOnly Property DefaultVideoHighlightFPS_H As XMLValue(Of Integer)
+        <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}, -1), Category("Defaults Video"), DisplayName("Highlight FPS (lower)"),
+            Description("Highlight frame rates lower than this value." & vbCr & "Default: -1" & vbCr & "-1 to disable")>
+        Public ReadOnly Property DefaultVideoHighlightFPS_L As XMLValue(Of Integer)
+        <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}), Category("Defaults Video"), DisplayName("Add extracted MP3 to playlist"),
+            Description("If you also extract MP3 when download the video, add the extracted MP3 to the playlist. Default: false.")>
+        Public ReadOnly Property VideoPlaylist_AddExtractedMP3 As XMLValue(Of Boolean)
 #End Region
 #Region "Defaults Audio"
         <Browsable(True), GridVisible, XMLVN({"DefaultsAudio"}, "AAC"), Category("Defaults Audio"), DisplayName("Default codec"),
@@ -405,6 +441,9 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible, XMLVN({"Playlists"}), Category("Music"), DisplayName("M3U8 Append file number"),
             Description("Add file number to file name. Default: false.")>
         Public ReadOnly Property MusicPlaylistCreate_M3U8_AppendNumber As XMLValue(Of Boolean)
+        <Browsable(True), GridVisible, XMLVN({"Playlists"}, M3U8CreationMode.Relative), Category("Music"), DisplayName("Create M3U8: creation mode"),
+            Description("Set the playlist creation mode: absolute links, relative links, or both. If 'Both' is selected, two playlists will be created. Default: 'Relative'.")>
+        Public ReadOnly Property MusicPlaylistCreate_CreationMode As XMLValue(Of M3U8CreationMode)
 #End Region
 #End Region
 #Region "Defaults Subtitles"

SCrawler.YouTube/Controls/ButtonRC.vb

@@ -0,0 +1,23 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.YouTube.Controls
+    Public Class ButtonRC : Inherits Button
+        Private Const WM_CONTEXTMENU As Integer = 123 '&H7B
+        Private Const WM_CANCELMODE As Integer = 31 '&H1F
+        Private Const WM_INITMENUPOPUP As Integer = 279 '&H117
+        Private Const SMTO_NOTIMEOUTIFNOTHUNG As Integer = 8
+        Protected Overrides Sub WndProc(ByRef m As Message)
+            If m.Msg = WM_CONTEXTMENU Or m.Msg = WM_CANCELMODE Or m.Msg = WM_INITMENUPOPUP Or m.Msg = SMTO_NOTIMEOUTIFNOTHUNG Then
+                m.Result = IntPtr.Zero
+            Else
+                MyBase.WndProc(m)
+            End If
+        End Sub
+    End Class
+End Namespace

SCrawler.YouTube/Controls/VideoOption.vb

@@ -58,6 +58,11 @@ Namespace API.YouTube.Controls
                 If Not m.ID.IsEmptyString AndAlso m.ID.StringToLower.Contains(DRC) Then LBL_CODECS.Text &= $"{d}DRC"
                 If Not SelectedAudio.ID.IsEmptyString Then LBL_CODECS.Text &= $" / {SelectedAudio.Extension}{d}{SelectedAudio.Codec}{d}{SelectedAudio.Bitrate}k"
                 If Not SelectedAudio.ID.IsEmptyString AndAlso SelectedAudio.ID.StringToLower.Contains(DRC) Then LBL_CODECS.Text &= $"{d}DRC"
+
+                If MyYouTubeSettings.DefaultVideoHighlightFPS_H > 0 AndAlso m.FPS > MyYouTubeSettings.DefaultVideoHighlightFPS_H Then _
+                   BackColor = MyColor.DeleteBack : ForeColor = MyColor.DeleteFore
+                If MyYouTubeSettings.DefaultVideoHighlightFPS_L > 0 AndAlso m.FPS < MyYouTubeSettings.DefaultVideoHighlightFPS_L Then _
+                   BackColor = MyColor.UpdateBack : ForeColor = MyColor.UpdateFore
             End If
 
             Dim sv% = m.Size / 1024

SCrawler.YouTube/Controls/VideoOptionsForm.Designer.vb

@@ -65,11 +65,11 @@ Namespace API.YouTube.Controls
             Me.LBL_TIME = New System.Windows.Forms.Label()
             Me.LBL_URL = New System.Windows.Forms.LinkLabel()
             Me.TXT_FILE = New PersonalUtilities.Forms.Controls.ComboBoxExtended()
-            Me.BTT_BROWSE = New System.Windows.Forms.Button()
+            Me.BTT_BROWSE = New SCrawler.API.YouTube.Controls.ButtonRC()
             Me.BTT_DOWN = New System.Windows.Forms.Button()
             Me.BTT_CANCEL = New System.Windows.Forms.Button()
             Me.CMB_PLS = New PersonalUtilities.Forms.Controls.ComboBoxExtended()
-            Me.BTT_PLS_BROWSE = New System.Windows.Forms.Button()
+            Me.BTT_PLS_BROWSE = New SCrawler.API.YouTube.Controls.ButtonRC()
             Me.OPT_VIDEO = New System.Windows.Forms.RadioButton()
             Me.OPT_AUDIO = New System.Windows.Forms.RadioButton()
             Me.LBL_AUDIO_CODEC = New System.Windows.Forms.Label()
@@ -912,13 +912,13 @@ Namespace API.YouTube.Controls
         Private WithEvents TXT_SUBS_ADDIT As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents TXT_EXTRA_AUDIO_FORMATS As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents TXT_FILE As PersonalUtilities.Forms.Controls.ComboBoxExtended
-        Private WithEvents BTT_BROWSE As Button
+        Private WithEvents BTT_BROWSE As SCrawler.API.YouTube.Controls.ButtonRC
         Private WithEvents BTT_DOWN As Button
         Private WithEvents BTT_CANCEL As Button
         Private WithEvents TP_HEADER_INFO_2 As TableLayoutPanel
         Private WithEvents TXT_FPS As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents CMB_PLS As PersonalUtilities.Forms.Controls.ComboBoxExtended
-        Private WithEvents BTT_PLS_BROWSE As Button
+        Private WithEvents BTT_PLS_BROWSE As SCrawler.API.YouTube.Controls.ButtonRC
         Private WithEvents TXT_AUDIO_BITRATE As PersonalUtilities.Forms.Controls.TextBoxExtended
     End Class
 End Namespace

SCrawler.YouTube/Controls/VideoOptionsForm.vb

@@ -121,7 +121,7 @@ Namespace API.YouTube.Controls
                             img = ImageRenderer.GetImage(SFile.GetBytesFromNet(imgUrl, EDP.ReturnValue), EDP.ReturnValue)
                             If Not img Is Nothing Then ICON_VIDEO.Image = img : ICON_VIDEO.InitialImage = img
                         End If
-                        LBL_TITLE.Text = .Title
+                        LBL_TITLE.Text = $"{If(MyYouTubeSettings.FileAddDateToFileName_VideoForm.Value, $"[{ .DateAdded:yyyy-MM-dd}] ", String.Empty)}{ .Title}"
                         LBL_TIME.Text = AConvert(Of String)(.Duration, TimeToStringProvider, String.Empty)
                         TP_HEADER_INFO_2.ColumnStyles(1).Width = MeasureTextDefault(LBL_TIME.Text, LBL_TIME.Font).Width + PaddingE.GetOf({LBL_TIME}).Horizontal
                         TP_HEADER_INFO_2.Refresh()
@@ -610,7 +610,7 @@ Namespace API.YouTube.Controls
                                 $"Video|{AvailableVideoFormats.Select(Function(vf) $"*.{vf.ToLower}").ListToString(";")}" &
                                 $"|Audio|{AvailableAudioFormats.Select(Function(af) $"*.{af.ToLower}").ListToString(";")}"
                 f = SFile.SaveAs(f, "Select the destination of the video file",, ext, sPattern, EDP.ReturnValue)
-                f.Extension = ext
+                If Not f.IsEmptyString Then f.Extension = ext
             End If
 #Enable Warning
             f = CleanFileName(f)

SCrawler.YouTube/Declarations.vb

@@ -6,6 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Runtime.CompilerServices
 Imports PersonalUtilities.Tools
 Imports PersonalUtilities.Forms.Toolbars
 Imports PersonalUtilities.Functions.RegularExpressions
@@ -56,11 +57,19 @@ Namespace API.YouTube
         Friend ReadOnly TitleHtmlConverter As Func(Of String, String) = Function(Input) Input.StringRemoveWinForbiddenSymbols().StringTrim()
         Friend ReadOnly ProgressProvider As IMyProgressNumberProvider = MyProgressNumberProvider.Percentage
         Public ReadOnly TrueUrlRegEx As RParams = RParams.DM(Base.YouTubeFunctions.TrueUrlPattern, 0, EDP.ReturnValue)
+        Friend ReadOnly MusicUrlApply As RParams = RParams.DMS("https://([w\.]*)youtube.com.+", 1, RegexReturn.Replace, EDP.ReturnValue,
+                                                               CType(Function(input$) "music.", Func(Of String, String)), String.Empty)
+        <Extension> Friend Function ToMusicUrl(ByVal URL As String, ByVal IsMusic As Boolean) As String
+            Try : Return If(IsMusic And Not URL.IsEmptyString, CStr(RegexReplace(URL, MusicUrlApply)).IfNullOrEmpty(URL), URL) : Catch : Return URL : End Try
+        End Function
         Friend Function CleanFileName(ByVal f As SFile) As SFile
             If Not f.IsEmptyString And Not f.Name.IsEmptyString Then
                 Dim ff As SFile = f
-                ff.Name = ff.Name.StringRemoveWinForbiddenSymbols
-                If Not ff.Name.IsEmptyString Then ff.Name = ff.Name.Replace("%", String.Empty)
+                ff.Name = ff.Name.StringRemoveWinForbiddenSymbols.StringTrim
+                ff.Name = ff.Name.StringReplaceSymbols({vbLf, vbCr, vbCrLf}, String.Empty, EDP.ReturnValue)
+                ff.Name = ff.Name.StringTrimEnd(".")
+                If Not ff.Name.IsEmptyString And Not MyYouTubeSettings.FileRemoveCharacters.IsEmptyString Then _
+                   ff.Name = ff.Name.StringReplaceSymbols(MyYouTubeSettings.FileRemoveCharacters.Value.AsList.ListCast(Of String).ToArray, String.Empty, EDP.ReturnValue)
                 If ff.Name.IsEmptyString Then ff.Name = "file"
                 Return ff
             Else

SCrawler.YouTube/Downloader/MediaItem.vb

@@ -133,18 +133,25 @@ Namespace DownloadObjects.STDownloader
 
                 ICON_SITE.Image = .SiteIcon
                 LBL_TIME.Text = AConvert(Of String)(.Duration, TimeToStringProvider, String.Empty)
-                LBL_TITLE.Text = .ToString(True)
+                LBL_TITLE.Text = $"{If(MyYouTubeSettings.FileAddDateToFileName_VideoList.Value, $"[{ .DateAdded:yyyy-MM-dd}] ", String.Empty)}{ .ToString(True)}"
+                Dim h%, b%
+                If .Self.GetType Is GetType(YouTubeMediaContainerBase) OrElse (Not .Self.GetType.BaseType Is Nothing AndAlso .Self.GetType.BaseType Is GetType(YouTubeMediaContainerBase)) Then
+                    With DirectCast(.Self, YouTubeMediaContainerBase) : h = .HeightBase : b = .BitrateBase : End With
+                Else
+                    h = .Height
+                    b = .Bitrate
+                End If
                 If Not .SiteKey = YouTubeSiteKey And .ContentType = Plugin.UserMediaTypes.Picture Then
                     LBL_INFO.Text = .File.Extension.StringToUpper
                 ElseIf Not .IsMusic And Not (.MediaType = Plugin.UserMediaTypes.Audio Or .MediaType = Plugin.UserMediaTypes.AudioPre) Then
-                    If .Height > 0 Then
-                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{ .Height}p"
+                    If h > 0 Then
+                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{h}p"
                     Else
                         LBL_INFO.Text = .File.Extension.StringToUpper
                     End If
                 Else
-                    If .Bitrate > 0 Then
-                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{ .Bitrate}k"
+                    If b > 0 Then
+                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{b}k"
                     Else
                         LBL_INFO.Text = .File.Extension.StringToUpper
                     End If
@@ -221,7 +228,7 @@ Namespace DownloadObjects.STDownloader
                     t = 0
                 End If
 
-                LBL_TITLE.Text = MyContainer.ToString(True)
+                LBL_TITLE.Text = $"{If(MyYouTubeSettings.FileAddDateToFileName_VideoList.Value, $"[{ .DateAdded:yyyy-MM-dd}] ", String.Empty)}{ .ToString(True)}"
 
                 If Not .SiteKey = YouTubeSiteKey Then BTT_VIEW_SETTINGS.Visible = False
 

SCrawler.YouTube/Downloader/VideoListForm.vb

@@ -57,6 +57,12 @@ Namespace DownloadObjects.STDownloader
                 End If
                 MyNotificator = New YTNotificator(Me)
                 MyDownloaderSettings = MyYouTubeSettings
+                ProgramLogInitialize()
+                With ProgramLog
+                    AddHandler .TextAdded, AddressOf ProgramLog_TextAdded
+                    AddHandler .TextCleared, AddressOf ProgramLog_TextCleared
+                End With
+                UpdateLogButton()
             End If
 
             With MyView : .Import() : .SetFormSize() : End With
@@ -157,7 +163,7 @@ Namespace DownloadObjects.STDownloader
                                                    If PerformClick Then cnt.PerformClick()
                                                    If Not DisableDownload And MyDownloaderSettings.DownloadAutomatically Then AddToDownload(cnt, True)
                                                End With
-                                           End Sub, EDP.None)
+                                           End Sub, EDP.SendToLog)
         End Sub
 #Region "Controls rendering"
         Private Overloads Sub OffsetControls()
@@ -247,7 +253,7 @@ Namespace DownloadObjects.STDownloader
                 Dim useCookiesParse As Boolean? = Nothing
                 If useCookies Then useCookiesParse = True
                 Dim standardizeUrls As Boolean = MyYouTubeSettings.StandardizeURLs
-                Dim standardize As Func(Of String, String) = Function(input) If(standardizeUrls, YouTubeFunctions.StandardizeURL(input), input)
+                Dim standardize As Func(Of String, String) = Function(input) If(standardizeUrls, YouTubeFunctions.StandardizeURL(input), input.StringTrim)
 
                 Dim c As IYouTubeMediaContainer = Nothing
                 Dim url$ = String.Empty
@@ -449,12 +455,26 @@ Namespace DownloadObjects.STDownloader
             End Try
         End Sub
 #End Region
+#Region "LOG"
         Private Sub BTT_LOG_Click(sender As Object, e As EventArgs) Handles BTT_LOG.Click
             MyMainLOG_ShowForm(DesignXML,,,, AddressOf UpdateLogButton)
         End Sub
-        Friend Sub UpdateLogButton()
-            If AppMode Then MyMainLOG_UpdateLogButton(BTT_LOG, TOOLBAR_TOP)
+        Private Sub UpdateLogButton()
+            If AppMode Then
+                Try : MyMainLOG_UpdateLogButton(BTT_LOG, TOOLBAR_TOP) : Catch : End Try
+            End If
         End Sub
+        Private _LogUpdateButtonSuspended As Boolean = False
+        Private Sub ProgramLog_TextAdded(ByVal Sender As Object, ByVal e As EventArgs)
+            If Not _LogUpdateButtonSuspended Then
+                _LogUpdateButtonSuspended = True
+                Try : ControlInvokeFast(TOOLBAR_TOP, BTT_LOG, AddressOf UpdateLogButton, EDP.None) : Catch : End Try
+            End If
+        End Sub
+        Private Sub ProgramLog_TextCleared(ByVal Sender As Object, ByVal e As EventArgs)
+            _LogUpdateButtonSuspended = False
+        End Sub
+#End Region
         Private Sub BTT_BUG_REPORT_Click(sender As Object, e As EventArgs) Handles BTT_BUG_REPORT.Click
             Try
                 With MyYouTubeSettings

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2024.4.10.0")>
-<Assembly: AssemblyFileVersion("2024.4.10.0")>
+<Assembly: AssemblyVersion("2024.7.24.0")>
+<Assembly: AssemblyFileVersion("2024.7.24.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Objects/YouTubeMediaContainerBase.vb

@@ -267,12 +267,11 @@ Namespace API.YouTube.Objects
         <XMLEC(CollectionMode:=CollectionModes.String)>
         Friend ReadOnly Property PostProcessing_OutputAudioFormats As List(Of String)
         Friend Sub PostProcessing_OutputAudioFormats_Reset()
-            PostProcessing_OutputAudioFormats.Clear()
-            PostProcessing_OutputAudioFormats.ListAddList(MyYouTubeSettings.DefaultAudioCodecAddit)
-            If PostProcessing_OutputAudioFormats.Count > 0 Then
-                PostProcessing_OutputAudioFormats.Sort()
-                PostProcessing_OutputAudioFormats.RemoveAll(Function(s) s = -1)
-            End If
+            With PostProcessing_OutputAudioFormats
+                .Clear()
+                .ListAddList(MyYouTubeSettings.DefaultAudioCodecAddit)
+                If .Count > 0 Then .Sort()
+            End With
         End Sub
         <XMLEC("OutputAudioBitrate")> Protected _OutputAudioBitrate As Integer = -1
         Friend Property OutputAudioBitrate As Integer
@@ -322,21 +321,19 @@ Namespace API.YouTube.Objects
         <XMLEC(CollectionMode:=CollectionModes.String)>
         Friend ReadOnly Property PostProcessing_OutputSubtitlesFormats As List(Of String)
         Friend Sub PostProcessing_OutputSubtitlesFormats_Reset()
-            PostProcessing_OutputSubtitlesFormats.Clear()
-            PostProcessing_OutputSubtitlesFormats.ListAddList(MyYouTubeSettings.DefaultSubtitlesFormatAddit)
-            If PostProcessing_OutputSubtitlesFormats.Count > 0 Then
-                PostProcessing_OutputSubtitlesFormats.Sort()
-                PostProcessing_OutputSubtitlesFormats.RemoveAll(Function(s) s = -1)
-            End If
+            With PostProcessing_OutputSubtitlesFormats
+                .Clear()
+                .ListAddList(MyYouTubeSettings.DefaultSubtitlesFormatAddit)
+                If .Count > 0 Then .Sort()
+            End With
         End Sub
         Friend Sub SubtitlesSelectedIndexesReset()
-            SubtitlesSelectedIndexes.Clear()
-            Dim subs As List(Of Subtitles) = Subtitles
-            SubtitlesSelectedIndexes.ListAddList(MyYouTubeSettings.DefaultSubtitles.Select(Function(s) subs.FindIndex(Function(ss) ss.ID = s)))
-            If SubtitlesSelectedIndexes.Count > 0 Then
-                SubtitlesSelectedIndexes.Sort()
-                SubtitlesSelectedIndexes.RemoveAll(Function(s) s = -1)
-            End If
+            With SubtitlesSelectedIndexes
+                .Clear()
+                Dim subs As List(Of Subtitles) = Subtitles
+                .ListAddList(MyYouTubeSettings.DefaultSubtitles.Select(Function(s) subs.FindIndex(Function(ss) ss.ID = s)))
+                If .Count > 0 Then .Sort() : .RemoveAll(Function(s) s = -1)
+            End With
         End Sub
         Private Sub SetElementsSubtitles(ByVal Source As YouTubeMediaContainerBase)
             If Not Source Is Nothing And HasElements Then
@@ -442,6 +439,19 @@ Namespace API.YouTube.Objects
             End Get
         End Property
         <XMLEC> Public Property Height As Integer Implements IYouTubeMediaContainer.Height
+        Friend ReadOnly Property HeightBase As Integer
+            Get
+                If Height > 0 Then
+                    Return Height
+                ElseIf SelectedVideoIndex.ValueBetween(0, MediaObjects.Count - 1) Then
+                    Return SelectedVideo.Height
+                ElseIf SelectedAudioIndex.ValueBetween(0, MediaObjects.Count - 1) Then
+                    Return SelectedAudio.Height
+                Else
+                    Return 0
+                End If
+            End Get
+        End Property
         Protected _Bitrate As Integer = 0
         <XMLEC> Public Overridable Property Bitrate As Integer Implements IYouTubeMediaContainer.Bitrate
             Get
@@ -459,6 +469,20 @@ Namespace API.YouTube.Objects
                 Me._Bitrate = _Bitrate
             End Set
         End Property
+        Friend ReadOnly Property BitrateBase As Integer
+            Get
+                If Bitrate > 0 Then
+                    Return Bitrate
+                ElseIf OutputAudioBitrate > 0 Then
+                    Return OutputAudioBitrate
+                ElseIf HasElements Then
+                    Try : Return Elements.Average(Function(e) DirectCast(e, YouTubeMediaContainerBase).BitrateBase) : Catch : End Try
+                ElseIf SelectedAudioIndex.ValueBetween(0, MediaObjects.Count - 1) Then
+                    Return SelectedAudio.Bitrate
+                End If
+                Return 0
+            End Get
+        End Property
         <XMLEC> Public Property DateCreated As Date = Now Implements IYouTubeMediaContainer.DateCreated
         <XMLEC> Public Property DateAdded As Date Implements IYouTubeMediaContainer.DateAdded
         Private Property IUserMedia_PostDate As Date? Implements IUserMedia.PostDate
@@ -656,6 +680,15 @@ Namespace API.YouTube.Objects
                 End If
             End Set
         End Property
+        Friend Sub FileDateUpdate()
+            Dim n$ = _File.Name.StringTrim
+            Dim s$ = IIf(n.IsEmptyString, String.Empty, " ")
+            Select Case MyYouTubeSettings.FileAddDateToFileName.Value
+                Case FileDateMode.Before : n = $"[{DateAdded:yyyy-MM-dd}]{s}{n}"
+                Case FileDateMode.After : n = $"{n}{s}[{DateAdded:yyyy-MM-dd}]"
+            End Select
+            _File.Name = n
+        End Sub
         Public Property FileSettings As SFile
         Private Property IUserMedia_File As String Implements IUserMedia.File
             Get
@@ -709,7 +742,7 @@ Namespace API.YouTube.Objects
                     Bitrate = 0
                     _MediaType = UMTypes.Undefined
                     If SelectedVideoIndex >= 0 Then
-                        'URGENT: 2023.3.4 -> 2023.7.6
+                        '2023.3.4 -> 2023.7.6
                         'cmd.StringAppend($"bv*[format_id={SelectedVideo.ID}]")
                         cmd.StringAppend(SelectedVideo.ID)
                         _Size = SelectedVideo.Size
@@ -726,7 +759,7 @@ Namespace API.YouTube.Objects
                     End If
                     If SelectedAudioIndex >= 0 Then
                         Dim atCodec$
-                        'URGENT: 2023.3.4 -> 2023.7.6
+                        '2023.3.4 -> 2023.7.6
                         'cmd.StringAppend($"ba*[format_id={SelectedAudio.ID}]", "+")
                         cmd.StringAppend(SelectedAudio.ID, "+")
                         If OutputAudioCodec.StringToLower = ac3 Then
@@ -769,7 +802,7 @@ Namespace API.YouTube.Objects
                         subs = $"--write-subs --write-auto-subs --sub-format {OutputSubtitlesFormat.StringToLower} --sub-langs ""{subs}"" --convert-subs {OutputSubtitlesFormat.StringToLower}"
                     End If
                     If Not cmd.IsEmptyString Then
-                        'URGENT: 2023.3.4 -> 2023.7.6
+                        '2023.3.4 -> 2023.7.6
                         'cmd = $"yt-dlp -f ""{cmd}"""
                         'cmd = $"yt-dlp -f {cmd}"
                         cmd = $"{YTDLP_NAME} -f {cmd}"
@@ -866,10 +899,13 @@ Namespace API.YouTube.Objects
                 Return Nothing
             End Try
         End Function
-        Private Function GetPlaylistRow(ByVal Element As YouTubeMediaContainerBase, Optional ByVal __file As SFile = Nothing) As String
+        Private Function GetPlaylistRow(ByVal Element As YouTubeMediaContainerBase, Optional ByVal __file As SFile = Nothing,
+                                        Optional ByVal Mode As M3U8CreationMode = M3U8CreationMode.Absolute) As String
             Const m3u8DataRow$ = "#EXTINF:{0},{1}" & vbCrLf & "{2}"
             With Element
                 Dim f As SFile = __file.IfNullOrEmpty(.File)
+                Dim __f$ = SymbolsConverter.ASCII.EncodeSymbolsOnly(If(Mode = M3U8CreationMode.Absolute, f.ToString, f.File))
+                If Mode = M3U8CreationMode.Absolute Then __f = $"file:///{__f}"
                 Dim fName$ = .Title.IfNullOrEmpty(f.Name)
                 If MyYouTubeSettings.MusicPlaylistCreate_M3U8_AppendNumber And .PlaylistIndex > 0 Then fName = $"{ .PlaylistIndex}. {fName}"
                 If Not .UserTitle.IsEmptyString Then
@@ -877,10 +913,7 @@ Namespace API.YouTube.Objects
                     If MyYouTubeSettings.MusicPlaylistCreate_M3U8_AppendArtist Then fName = $"{ .UserTitle} - {fName}"
                 End If
                 If MyYouTubeSettings.MusicPlaylistCreate_M3U8_AppendExt Then fName &= $".{f.Extension}"
-                Return String.Format(m3u8DataRow,
-                                     CInt(.Duration.TotalSeconds),
-                                     fName,
-                                     $"file:///{SymbolsConverter.ASCII.EncodeSymbolsOnly(f)}")
+                Return String.Format(m3u8DataRow, CInt(.Duration.TotalSeconds), fName, __f)
             End With
         End Function
         Private ReadOnly DownloadProgressPattern As RParams = RParams.DMS("\[download\]\s*([\d\.,]+)", 1, EDP.ReturnValue)
@@ -921,23 +954,41 @@ Namespace API.YouTube.Objects
                     Dim t As TextSaver = Nothing
                     Try
                         Dim f As SFile
-                        If MyYouTubeSettings.MusicPlaylistCreate_M3U8 Then
-                            t = New TextSaver
-                            t.AppendLine("#EXTM3U")
-                            Elements.ForEach(Sub(e) t.AppendLine(GetPlaylistRow(e)))
-                            f = $"{Elements(0).File.PathWithSeparator}Playlist.m3u8"
-                            t.SaveAs(f, EDP.SendToLog)
-                            If f.Exists Then AddFile(f)
-                            t.Dispose()
-                        End If
-                        If MyYouTubeSettings.MusicPlaylistCreate_M3U Then
-                            t = New TextSaver
-                            Elements.ForEach(Sub(e) t.AppendLine(e.File))
-                            f = $"{Elements(0).File.PathWithSeparator}Playlist.m3u"
-                            t.SaveAs(f, EDP.SendToLog)
-                            If f.Exists Then AddFile(f)
-                            t.Dispose()
-                        End If
+                        Dim arr As M3U8CreationMode() = If(MyYouTubeSettings.MusicPlaylistCreate_CreationMode.Value = M3U8CreationMode.Both,
+                                                           {M3U8CreationMode.Relative, M3U8CreationMode.Absolute},
+                                                           {MyYouTubeSettings.MusicPlaylistCreate_CreationMode.Value})
+                        Dim postfix$
+                        Dim added As Boolean
+                        Dim checkFile As Func(Of IYouTubeMediaContainer, Boolean) = Function(ByVal e As IYouTubeMediaContainer) As Boolean
+                                                                                        If e.File.Exists Then
+                                                                                            added = True
+                                                                                            Return True
+                                                                                        Else
+                                                                                            Return False
+                                                                                        End If
+                                                                                    End Function
+                        For Each cm As M3U8CreationMode In arr
+                            If arr.Length > 1 AndAlso cm = M3U8CreationMode.Absolute Then postfix = "Abs" Else postfix = String.Empty
+                            added = False
+                            If MyYouTubeSettings.MusicPlaylistCreate_M3U8 Then
+                                t = New TextSaver
+                                t.AppendLine("#EXTM3U")
+                                Elements.ForEach(Sub(e) If checkFile(e) Then t.AppendLine(GetPlaylistRow(e,, cm)))
+                                f = $"{Elements(0).File.PathWithSeparator}Playlist{postfix}.m3u8"
+                                If added Then t.SaveAs(f, EDP.SendToLog)
+                                If f.Exists Then AddFile(f)
+                                t.Dispose()
+                            End If
+                            added = False
+                            If MyYouTubeSettings.MusicPlaylistCreate_M3U Then
+                                t = New TextSaver
+                                Elements.ForEach(Sub(e) If checkFile(e) Then t.AppendLine(If(cm = M3U8CreationMode.Relative, e.File.File, e.File.ToString)))
+                                f = $"{Elements(0).File.PathWithSeparator}Playlist{postfix}.m3u"
+                                If added Then t.SaveAs(f, EDP.SendToLog)
+                                If f.Exists Then AddFile(f)
+                                t.Dispose()
+                            End If
+                        Next
                     Catch ex As Exception
                         ErrorsDescriber.Execute(EDP.SendToLog, ex, "[YouTubeMediaContainerBase.Download.CreatePlaylist]")
                     End Try
@@ -966,7 +1017,7 @@ Namespace API.YouTube.Objects
                             .Visible = True
                             .Value = 0
                             .Maximum = DownloadGetElemCountSingle()
-                            .Information = $"Download {ObjectType}"
+                            .Information = "Downloading"
                         End With
                     End If
 
@@ -1111,7 +1162,7 @@ Namespace API.YouTube.Objects
                                 .Value = 0
                                 .Maximum = 100
                                 .Provider = ProgressProvider
-                                .Information = $"Download {MediaType}"
+                                .Information = "Downloading"
                             End With
                         End If
                         .MainProcessName = MyYouTubeSettings.YTDLP.Name '"yt-dlp"
@@ -1143,15 +1194,25 @@ Namespace API.YouTube.Objects
                                 If fileUrl.Exists Then AddFile(fileUrl)
                             End If
 
-                            If MyYouTubeSettings.CreateDescriptionFiles And Not Description.IsEmptyString Then
-                                Dim fileDesr As SFile = File
-                                fileDesr.Extension = "txt"
-                                TextSaver.SaveTextToFile(Description, fileDesr,,, EDP.None)
-                                If fileDesr.Exists Then AddFile(fileDesr)
-                            End If
+                            With MyYouTubeSettings
+                                If .CreateDescriptionFiles And (Not Description.IsEmptyString Or
+                                                                (.CreateDescriptionFiles_CreateWithNoDescription And .CreateDescriptionFiles_AddUploadDate)) Then
+                                    Dim fileDesr As SFile = File
+                                    fileDesr.Extension = "txt"
+                                    Using fileDesrText As New TextSaver(fileDesr)
+                                        If .CreateDescriptionFiles_AddUploadDate Then fileDesrText.Append($"Uploaded: {DateAdded:yyyy-MM-dd HH:mm:ss}")
+                                        If Not Description.IsEmptyString Then
+                                            If Not fileDesrText.IsEmptyString Then fileDesrText.AppendLine.AppendLine()
+                                            fileDesrText.Append(Description)
+                                        End If
+                                        fileDesrText.Save(EDP.None)
+                                    End Using
+                                    If fileDesr.Exists Then AddFile(fileDesr)
+                                End If
+                            End With
 
                             If PlaylistCount > 0 And Not CoverDownloaded And Not PlaylistID.IsEmptyString Then DownloadPlaylistCover(PlaylistID, File, UseCookies)
-                            If prExists Then Progress.InformationTemporary = $"Download {MediaType}: post processing"
+                            If prExists Then Progress.InformationTemporary = "Downloading: post processing"
                             _ThumbnailFile = File
                             _ThumbnailFile.Name &= "_thumb"
                             _ThumbnailFile.Extension = "jpg"
@@ -1219,19 +1280,21 @@ Namespace API.YouTube.Objects
 
                                 'Subtitles
                                 ThrowAny(Token)
-                                If PostProcessing_OutputSubtitlesFormats.Count > 0 Then
+                                If SubtitlesSelectedIndexes.Count > 0 And Not OutputSubtitlesFormat.IsEmptyString Then
                                     files = SFile.GetFiles(File, String.Format(fPatternFiles, OutputSubtitlesFormat.StringToLower),, EDP.ReturnValue)
-                                    AddFile(files)
                                     If files.ListExists Then
-                                        For Each f In files
-                                            For Each format In PostProcessing_OutputSubtitlesFormats
-                                                format = format.StringToLower
-                                                commandFile = $"{f.PathWithSeparator}{f.Name}.{format}"
-                                                AddFile(commandFile)
-                                                ThrowAny(Token)
-                                                .Execute($"ffmpeg -i ""{f}"" ""{commandFile}""")
+                                        AddFile(files)
+                                        If PostProcessing_OutputSubtitlesFormats.Count > 0 Then
+                                            For Each f In files
+                                                For Each format In PostProcessing_OutputSubtitlesFormats
+                                                    format = format.StringToLower
+                                                    commandFile = $"{f.PathWithSeparator}{f.Name}.{format}"
+                                                    AddFile(commandFile)
+                                                    ThrowAny(Token)
+                                                    .Execute($"ffmpeg -i ""{f}"" ""{commandFile}""")
+                                                Next
                                             Next
-                                        Next
+                                        End If
                                     End If
                                 End If
 
@@ -1284,6 +1347,7 @@ Namespace API.YouTube.Objects
                                                 If format = mp3 And Not mp3ThumbEmbedded And MyYouTubeSettings.DefaultAudioEmbedThumbnail_ExtractedFiles Then _
                                                    embedThumbTo.Invoke(f) : mp3ThumbEmbedded = True
                                                 If Not M3U8_PlaylistFiles.ListExists AndAlso f.Exists Then M3U8_Append(f)
+                                                If format = mp3 AndAlso f.Exists AndAlso MyYouTubeSettings.VideoPlaylist_AddExtractedMP3.Value Then M3U8_Append(f)
                                             End If
                                         Next
                                     End If
@@ -1308,15 +1372,29 @@ Namespace API.YouTube.Objects
                                 'Delete unrequsted files
                                 If tempFilesList.Count > 0 Then tempFilesList.ForEach(Sub(tfr) If Not tfr.Requested Then tfr.File.Delete(,, EDP.None)) : tempFilesList.Clear()
 
-                                'Update video FPS
-                                If SelectedVideoIndex >= 0 AndAlso OutputVideoFPS > 0 AndAlso SelectedVideo.Bitrate <> OutputVideoFPS Then
-                                    f = File
-                                    f.Name &= "tmp00"
-                                    .Execute($"ffmpeg -i ""{File}"" -filter:v fps={OutputVideoFPS.ToString.Replace(",", ".")} -c:a copy ""{f}""")
-                                    If f.Exists Then
-                                        File.Delete()
-                                        SFile.Rename(f, File,, EDP.LogMessageValue)
-                                    End If
+                                If SelectedVideoIndex >= 0 Then
+                                    Dim reencodeFile As Action(Of String) =
+                                        Sub(ByVal ffmpegCommand As String)
+                                            f = File
+                                            f.Name &= "tmp00"
+                                            .Execute(String.Format(ffmpegCommand, File.ToString, f.ToString))
+                                            If f.Exists Then
+                                                If f.Size > 0 Then
+                                                    File.Delete()
+                                                    SFile.Rename(f, File,, EDP.LogMessageValue)
+                                                Else
+                                                    f.Delete(, SFODelete.DeletePermanently, EDP.None)
+                                                End If
+                                            End If
+                                        End Sub
+                                    'Change video codec to AVC
+                                    If MyYouTubeSettings.DefaultVideoConvertNonAVC.Value AndAlso
+                                       Not SelectedVideo.Codec.IsEmptyString AndAlso Not SelectedVideo.Codec.Trim.ToLower.StartsWith("avc") Then _
+                                       reencodeFile("ffmpeg -i ""{0}"" -c:a copy -c:v libx264 ""{1}""")
+
+                                    'Update video FPS
+                                    If OutputVideoFPS > 0 AndAlso SelectedVideo.Bitrate <> OutputVideoFPS Then _
+                                       reencodeFile("ffmpeg -i ""{0}"" -filter:v fps=" & OutputVideoFPS.ToString.Replace(", ", ".") & " -c:a copy ""{1}""")
                                 End If
                             End If
                         End If
@@ -1532,7 +1610,7 @@ Namespace API.YouTube.Objects
                         ID = .Value("id")
                         Title = TitleHtmlConverter.Invoke(.Value("title"))
                         Description = .Value("description")
-                        URL = .Value("webpage_url")
+                        URL = .Value("webpage_url").ToMusicUrl(IsMusic)
 
                         PlaylistID = .Value("playlist_id")
                         PlaylistCount = .Value("n_entries").IfNullOrEmpty(.Value("playlist_count")).FromXML(Of Integer)(0)
@@ -1570,6 +1648,7 @@ Namespace API.YouTube.Objects
                             If tValue.HasValue Then Duration = TimeSpan.FromSeconds(tValue.Value)
                         End If
                         DateAdded = AConvert(Of Date)(.Value("release_date").IfNullOrEmpty(.Value("upload_date")), DateAddedProvider, New Date)
+                        If Not IsMusic Then FileDateUpdate()
 
                         ParseFormats(.Self)
 

SCrawler.YouTube/SCrawler.YouTube.vbproj

@@ -115,6 +115,9 @@
   <ItemGroup>
     <Compile Include="Attributes\GridVisibleAttribute.vb" />
     <Compile Include="Base\TableControlsProcessor.vb" />
+    <Compile Include="Controls\ButtonRC.vb">
+      <SubType>Component</SubType>
+    </Compile>
     <Compile Include="Controls\ChannelTabsChooserForm.Designer.vb">
       <DependentUpon>ChannelTabsChooserForm.vb</DependentUpon>
     </Compile>

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2024.4.10.0")>
-<Assembly: AssemblyFileVersion("2024.4.10.0")>
+<Assembly: AssemblyVersion("2024.7.24.0")>
+<Assembly: AssemblyFileVersion("2024.7.24.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/API/Base/Declarations.vb

@@ -6,6 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Runtime.CompilerServices
 Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Base
@@ -72,5 +73,12 @@ Namespace API.Base
                             $"Current query: [{CurrentQuery}]{vbCr}New query: [{NewQuery}]",
                             "Changing a query"}, vbExclamation,,, {"Process", "Cancel"}) = 0
         End Function
+        <Extension> Friend Function GetCookieValue(ByVal Cookies As IEnumerable(Of System.Net.Cookie), ByVal CookieName As String) As String
+            If Cookies.ListExists Then Return If(Cookies.FirstOrDefault(Function(c) c.Name.ToLower = CookieName.ToLower)?.Value, String.Empty) Else Return String.Empty
+        End Function
+        <Extension> Friend Function GetCookieValue(ByVal Cookies As IEnumerable(Of System.Net.Cookie), ByVal CookieName As String,
+                                                   ByVal PropName As String, ByVal PropNameComp As String) As String
+            Return If(PropName = PropNameComp, Cookies.GetCookieValue(CookieName), String.Empty)
+        End Function
     End Module
 End Namespace

SCrawler/API/Base/DeclaredNames.vb

@@ -11,6 +11,9 @@ Namespace API.Base
         Friend Const Header_Authorization As String = "authorization"
         Friend Const Header_CSRFToken As String = "x-csrf-token"
 
+        Friend Const CAT_UserDefs As String = "New user defaults"
+        Friend Const CAT_Timers As String = "Timers"
+
         Friend Const ConcurrentDownloadsCaption As String = "Concurrent downloads"
         Friend Const ConcurrentDownloadsToolTip As String = "The number of concurrent downloads."
         Friend Const SavedPostsUserNameCaption As String = "Saved posts user"

SCrawler/API/Base/IUserData.vb

@@ -77,7 +77,7 @@ Namespace API.Base
         ''' </summary>
         Function Delete(Optional ByVal Multiple As Boolean = False, Optional ByVal CollectionValue As Integer = -1) As Integer
         Function EraseData(ByVal Mode As EraseMode) As Boolean
-        Function MoveFiles(ByVal CollectionName As String, ByVal SpecialCollectionPath As SFile) As Boolean
+        Function MoveFiles(ByVal CollectionName As String, ByVal SpecialCollectionPath As SFile, Optional ByVal NewUser As SplitCollectionUserInfo? = Nothing) As Boolean
         Function CopyFiles(ByVal DestinationPath As SFile, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean
         Sub OpenFolder()
         Property DownloadTopCount As Integer?

SCrawler/API/Base/ProfileSaved.vb

@@ -54,9 +54,8 @@ Namespace API.Base
             Dim aStr$ = String.Empty
             If Count > 1 Then aStr = $" ({Number}/{Count})"
             Try
-                If Host.Source.ReadyToDownload(PDownload.SavedPosts) Then
-                    If Host.Available(PDownload.SavedPosts, Multiple Or Count > 1) Then
-                        Host.DownloadStarted(PDownload.SavedPosts)
+                If Host.Available(PDownload.SavedPosts, Multiple Or Count > 1) Then
+                    If Host.Source.ReadyToDownload(PDownload.SavedPosts) Then
                         If Count > 1 Then Progress.Information = $"{Host.Name} - {Host.AccountName.IfNullOrEmpty(SettingsHost.NameAccountNameDefault)}"
                         Using user As IUserData = Host.GetInstance(PDownload.SavedPosts, Nothing, False, False)
                             If Not user Is Nothing Then
@@ -83,11 +82,11 @@ Namespace API.Base
                         End Using
                     Else
                         _Unavailable += 1
-                        Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is unavailable"
+                        Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is not ready"
                     End If
                 Else
                     _NotReady += 1
-                    Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is not ready"
+                    Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is unavailable"
                 End If
             Catch oex As OperationCanceledException When Token.IsCancellationRequested
                 _ErrorCount += 1
@@ -96,9 +95,6 @@ Namespace API.Base
                 _ErrorCount += 1
                 Progress.InformationTemporary = $"{Host.Name}{aStr} downloading error"
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, $"[API.Base.ProfileSaved.Download({Host.Key}{aStr})]")
-            Finally
-                Host.DownloadDone(PDownload.SavedPosts)
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
     End Class

SCrawler/API/Base/SiteSettingsBase.vb

@@ -34,6 +34,16 @@ Namespace API.Base
         Friend Property AccountName As String Implements ISiteSettings.AccountName
         Friend Property Temporary As Boolean = False Implements ISiteSettings.Temporary
         Friend Property DefaultInstance As ISiteSettings = Nothing Implements ISiteSettings.DefaultInstance
+        Protected _UserAgentDefault As String = String.Empty
+        Friend Overridable Property UserAgentDefault As String Implements ISiteSettings.UserAgentDefault
+            Get
+                Return _UserAgentDefault
+            End Get
+            Set(ByVal _UserAgentDefault As String)
+                Me._UserAgentDefault = _UserAgentDefault
+                If _AllowUserAgentUpdate And Not Responser Is Nothing And Not _UserAgentDefault.IsEmptyString Then Responser.UserAgent = _UserAgentDefault
+            End Set
+        End Property
         Protected _AllowUserAgentUpdate As Boolean = True
         Protected _SubscriptionsAllowed As Boolean = False
         Friend ReadOnly Property SubscriptionsAllowed As Boolean Implements ISiteSettings.SubscriptionsAllowed
@@ -138,7 +148,6 @@ Namespace API.Base
         Friend Overridable Sub BeginInit() Implements ISiteSettings.BeginInit
         End Sub
         Friend Overridable Sub EndInit() Implements ISiteSettings.EndInit
-            If _AllowUserAgentUpdate And Not DefaultUserAgent.IsEmptyString And Not Responser Is Nothing Then Responser.UserAgent = DefaultUserAgent
             If CheckNetscapeCookiesOnEndInit Then Update_SaveCookiesNetscape(, True)
         End Sub
 #End Region

SCrawler/API/Base/UserDataBase.vb

@@ -143,7 +143,7 @@ Namespace API.Base
         Protected Const Name_UserID As String = "UserID"
         Protected Const Name_Options As String = "Options"
         Protected Const Name_Description As String = "Description"
-        Private Const Name_ParseUserMediaOnly As String = "ParseUserMediaOnly"
+        Protected Const Name_ParseUserMediaOnly As String = "ParseUserMediaOnly"
         Private Const Name_IsSubscription As String = UserInfo.Name_IsSubscription
         Private Const Name_Temporary As String = "Temporary"
         Private Const Name_Favorite As String = "Favorite"
@@ -950,7 +950,10 @@ BlockNullPicture:
                 LogError(ex, "user information loading error")
             End Try
         End Sub
-        Friend Overridable Sub UpdateUserInformation() Implements IUserData.UpdateUserInformation
+        Friend Overridable Overloads Sub UpdateUserInformation() Implements IUserData.UpdateUserInformation
+            UpdateUserInformation(False)
+        End Sub
+        Friend Overridable Overloads Sub UpdateUserInformation(ByVal DisableUserInfoUpdate As Boolean)
             Try
                 UpdateDataFiles()
                 MyFileSettings.Exists(SFO.Path)
@@ -1001,7 +1004,7 @@ BlockNullPicture:
 
                     x.Save(MyFileSettings)
                 End Using
-                If Not IsSavedPosts Then Settings.UpdateUsersList(User)
+                If Not IsSavedPosts And Not DisableUserInfoUpdate Then Settings.UpdateUsersList(User, True)
             Catch ex As Exception
                 LogError(ex, "user information saving error")
             End Try
@@ -1894,7 +1897,9 @@ BlockNullPicture:
                         If m.Contains(IUserData.EraseMode.History) Then
                             If MyFilePosts.Delete(SFO.File, SFODelete.DeleteToRecycleBin, e) Then result = True
                             If MyFileData.Delete(SFO.File, SFODelete.DeleteToRecycleBin, e) Then result = True
+                            LastUpdated = Nothing
                             EraseData_AdditionalDataFiles()
+                            UpdateUserInformation()
                         End If
                         If m.Contains(IUserData.EraseMode.Data) Then
                             Dim files As List(Of SFile) = SFile.GetFiles(DownloadContentDefault_GetRootDir.CSFileP,, SearchOption.AllDirectories, e)
@@ -1916,7 +1921,7 @@ BlockNullPicture:
                 Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"EraseData({CInt(Mode)}): {ToStringForLog()}", False)
             End Try
         End Function
-        Protected Overridable Sub EraseData_AdditionalDataFiles()
+        Protected Overridable Sub EraseData_AdditionalDataFiles() Implements IPluginContentProvider.ResetHistoryData
         End Sub
         Friend Overridable Function Delete(Optional ByVal Multiple As Boolean = False, Optional ByVal CollectionValue As Integer = -1) As Integer Implements IUserData.Delete
             Dim f As SFile = SFile.GetPath(MyFile.CutPath.Path)
@@ -1932,7 +1937,18 @@ BlockNullPicture:
                 Return 0
             End If
         End Function
-        Friend Overridable Function MoveFiles(ByVal __CollectionName As String, ByVal __SpecialCollectionPath As SFile) As Boolean Implements IUserData.MoveFiles
+        Friend Function SplitCollectionGetNewUserInfo() As SplitCollectionUserInfo
+            Dim u As New SplitCollectionUserInfo With {.UserOrig = User, .UserNew = User}
+            With u.UserNew
+                .CollectionName = String.Empty
+                .SpecialCollectionPath = Nothing
+                .UserModel = UsageModel.Default
+                .CollectionModel = UsageModel.Default
+                .UpdateUserFile()
+            End With
+            Return u
+        End Function
+        Friend Overridable Function MoveFiles(ByVal __CollectionName As String, ByVal __SpecialCollectionPath As SFile, Optional ByVal NewUser As SplitCollectionUserInfo? = Nothing) As Boolean Implements IUserData.MoveFiles
             Dim UserBefore As UserInfo = User
             Dim Removed As Boolean = True
             Dim _TurnBack As Boolean = False
@@ -1948,6 +1964,7 @@ BlockNullPicture:
                     User.SpecialCollectionPath = String.Empty
                     User.UserModel = UsageModel.Default
                     User.CollectionModel = UsageModel.Default
+                    If NewUser.HasValue Then User.SpecialPath = NewUser.Value.UserNew.SpecialPath
                 Else
                     Settings.Users.Remove(Me)
                     Removed = True

SCrawler/API/BaseObjects/SplitCollectionUserInfo.vb

@@ -0,0 +1,28 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.Base
+    Friend Structure SplitCollectionUserInfo
+        Friend UserOrig As UserInfo
+        Friend UserNew As UserInfo
+        Friend Changed As Boolean
+        Friend ReadOnly Property SameDrive As Boolean
+            Get
+                Return GetUserDrive(UserOrig) = GetUserDrive(UserNew)
+            End Get
+        End Property
+        Private Shared Function GetUserDrive(ByVal User As UserInfo) As String
+            Dim u As UserInfo = User
+            If u.File.IsEmptyString Then u.UpdateUserFile()
+            Return u.File.Segments.FirstOrDefault.StringToLower
+        End Function
+        Public Overrides Function ToString() As String
+            Return $"[{UserOrig.File.CutPath.PathWithSeparator}] -> [{UserNew.File.CutPath.PathWithSeparator}]"
+        End Function
+    End Structure
+End Namespace

SCrawler/API/BaseObjects/SplitCollectionUserInfoChangePathsForm.Designer.vb

@@ -0,0 +1,111 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.Base
+    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
+    Partial Friend Class SplitCollectionUserInfoChangePathsForm : Inherits System.Windows.Forms.Form
+        <System.Diagnostics.DebuggerNonUserCode()>
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            Try
+                If disposing AndAlso components IsNot Nothing Then
+                    components.Dispose()
+                End If
+            Finally
+                MyBase.Dispose(disposing)
+            End Try
+        End Sub
+        Private components As System.ComponentModel.IContainer
+        <System.Diagnostics.DebuggerStepThrough()>
+        Private Sub InitializeComponent()
+            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
+            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
+            Dim LBL_INFO As System.Windows.Forms.Label
+            Me.LIST_USERS = New System.Windows.Forms.ListBox()
+            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
+            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
+            LBL_INFO = New System.Windows.Forms.Label()
+            CONTAINER_MAIN.ContentPanel.SuspendLayout()
+            CONTAINER_MAIN.SuspendLayout()
+            TP_MAIN.SuspendLayout()
+            Me.SuspendLayout()
+            '
+            'CONTAINER_MAIN
+            '
+            '
+            'CONTAINER_MAIN.ContentPanel
+            '
+            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(384, 261)
+            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            CONTAINER_MAIN.LeftToolStripPanelVisible = False
+            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
+            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
+            CONTAINER_MAIN.RightToolStripPanelVisible = False
+            CONTAINER_MAIN.Size = New System.Drawing.Size(384, 261)
+            CONTAINER_MAIN.TabIndex = 0
+            CONTAINER_MAIN.TopToolStripPanelVisible = False
+            '
+            'TP_MAIN
+            '
+            TP_MAIN.ColumnCount = 1
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
+            TP_MAIN.Controls.Add(LBL_INFO, 0, 0)
+            TP_MAIN.Controls.Add(Me.LIST_USERS, 0, 1)
+            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_MAIN.Location = New System.Drawing.Point(0, 0)
+            TP_MAIN.Name = "TP_MAIN"
+            TP_MAIN.RowCount = 2
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 50.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.Size = New System.Drawing.Size(384, 261)
+            TP_MAIN.TabIndex = 0
+            '
+            'LBL_INFO
+            '
+            LBL_INFO.Dock = System.Windows.Forms.DockStyle.Fill
+            LBL_INFO.Location = New System.Drawing.Point(3, 0)
+            LBL_INFO.Name = "LBL_INFO"
+            LBL_INFO.Size = New System.Drawing.Size(378, 50)
+            LBL_INFO.TabIndex = 0
+            LBL_INFO.Text = "Check the user destination paths and change them if necessary." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Double-click to c" &
+    "hange."
+            LBL_INFO.TextAlign = System.Drawing.ContentAlignment.MiddleCenter
+            '
+            'LIST_USERS
+            '
+            Me.LIST_USERS.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.LIST_USERS.FormattingEnabled = True
+            Me.LIST_USERS.Location = New System.Drawing.Point(3, 53)
+            Me.LIST_USERS.Name = "LIST_USERS"
+            Me.LIST_USERS.Size = New System.Drawing.Size(378, 205)
+            Me.LIST_USERS.TabIndex = 1
+            '
+            'SplitCollectionUserInfoChangePathsForm
+            '
+            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
+            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
+            Me.ClientSize = New System.Drawing.Size(384, 261)
+            Me.Controls.Add(CONTAINER_MAIN)
+            Me.Icon = Global.SCrawler.My.Resources.Resources.UsersIcon_32
+            Me.KeyPreview = True
+            Me.MinimumSize = New System.Drawing.Size(400, 300)
+            Me.Name = "SplitCollectionUserInfoChangePathsForm"
+            Me.ShowInTaskbar = False
+            Me.Text = "Collection users"
+            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
+            CONTAINER_MAIN.ResumeLayout(False)
+            CONTAINER_MAIN.PerformLayout()
+            TP_MAIN.ResumeLayout(False)
+            Me.ResumeLayout(False)
+
+        End Sub
+
+        Private WithEvents LIST_USERS As ListBox
+    End Class
+End Namespace

SCrawler/API/BaseObjects/SplitCollectionUserInfoChangePathsForm.resx

@@ -0,0 +1,129 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" use="required" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="LBL_INFO.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+</root>

SCrawler/API/BaseObjects/SplitCollectionUserInfoChangePathsForm.vb

@@ -0,0 +1,78 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Functions.Messaging
+Namespace API.Base
+    Friend Class SplitCollectionUserInfoChangePathsForm
+        Private WithEvents MyDefs As DefaultFormOptions
+        Friend ReadOnly Property Users As List(Of SplitCollectionUserInfo)
+        ''' <summary>
+        ''' Cancel = use initial<br/>
+        ''' Abort = abort operation<br/>
+        ''' OK = use changes
+        ''' </summary>
+        Friend Sub New(ByVal _Users As IEnumerable(Of SplitCollectionUserInfo))
+            InitializeComponent()
+            MyDefs = New DefaultFormOptions(Me, Settings.Design)
+            Users = New List(Of SplitCollectionUserInfo)(_Users)
+        End Sub
+        Private Sub SplitCollectionUserInfoChangePathsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
+            With MyDefs
+                .MyViewInitialize()
+                .AddOkCancelToolbar()
+                LIST_USERS.Items.AddRange(Users.Cast(Of Object).ToArray)
+                .EndLoaderOperations()
+                .MyOkCancel.EnableOK = True
+            End With
+        End Sub
+        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
+            MyDefs.CloseForm()
+        End Sub
+        Private Sub MyDefs_ButtonCancelClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonCancelClick
+            Dim m As New MMessage("You have canceled the change. Do you want to process user(s) as is or cancel the operation?", "Change user paths",
+                                  {New MsgBoxButton("Initial", "Process users as is (IGNORE changes to this form)") With {.CallBackObject = DialogResult.Cancel},
+                                   New MsgBoxButton("Process", "Process users as is (INCLUDE changes here)") With {.CallBackObject = DialogResult.OK},
+                                   New MsgBoxButton("Abort", "Abort operation") With {.CallBackObject = DialogResult.Abort},
+                                   New MsgBoxButton("Cancel", "Continue editing here") With {.CallBackObject = DialogResult.Retry}},
+                                  vbExclamation) With {.ButtonsPerRow = 4}
+            Dim result As DialogResult = CInt(MsgBoxE(m).Button.CallBackObject)
+            If result = DialogResult.Retry Then
+                e.Handled = True
+                Exit Sub
+            Else
+                MyDefs.CloseForm(result)
+            End If
+        End Sub
+        Private Sub SplitCollectionUserInfoChangePathsForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed
+            Users.Clear()
+        End Sub
+        Private Sub LIST_USERS_MouseDoubleClick(sender As Object, e As MouseEventArgs) Handles LIST_USERS.MouseDoubleClick
+            Try
+                With LIST_USERS
+                    If .SelectedIndex >= 0 Then
+                        Dim obj As SplitCollectionUserInfo = .Items(.SelectedIndex)
+                        Using f As New SplitCollectionUserInfoPathForm(obj)
+                            f.ShowDialog()
+                            If f.DialogResult = DialogResult.OK Then
+                                obj = f.User
+                                If obj.Changed Then
+                                    Users(.SelectedIndex) = obj
+                                    .Items(.SelectedIndex) = obj
+                                    .Refresh()
+                                End If
+                            End If
+                        End Using
+                    End If
+                End With
+            Catch ex As Exception
+                ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "Change user paths")
+            End Try
+        End Sub
+    End Class
+End Namespace

SCrawler/API/BaseObjects/SplitCollectionUserInfoPathForm.Designer.vb

@@ -0,0 +1,134 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.Base
+    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
+    Partial Friend Class SplitCollectionUserInfoPathForm : Inherits System.Windows.Forms.Form
+        <System.Diagnostics.DebuggerNonUserCode()>
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            Try
+                If disposing AndAlso components IsNot Nothing Then
+                    components.Dispose()
+                End If
+            Finally
+                MyBase.Dispose(disposing)
+            End Try
+        End Sub
+        Private components As System.ComponentModel.IContainer
+        <System.Diagnostics.DebuggerStepThrough()>
+        Private Sub InitializeComponent()
+            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
+            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
+            Dim ActionButton1 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(SplitCollectionUserInfoPathForm))
+            Dim ActionButton2 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Me.TXT_PATH_CURR = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            Me.TXT_PATH_NEW = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
+            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
+            CONTAINER_MAIN.ContentPanel.SuspendLayout()
+            CONTAINER_MAIN.SuspendLayout()
+            TP_MAIN.SuspendLayout()
+            CType(Me.TXT_PATH_CURR, System.ComponentModel.ISupportInitialize).BeginInit()
+            CType(Me.TXT_PATH_NEW, System.ComponentModel.ISupportInitialize).BeginInit()
+            Me.SuspendLayout()
+            '
+            'CONTAINER_MAIN
+            '
+            '
+            'CONTAINER_MAIN.ContentPanel
+            '
+            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(484, 84)
+            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            CONTAINER_MAIN.LeftToolStripPanelVisible = False
+            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
+            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
+            CONTAINER_MAIN.RightToolStripPanelVisible = False
+            CONTAINER_MAIN.Size = New System.Drawing.Size(484, 84)
+            CONTAINER_MAIN.TabIndex = 0
+            CONTAINER_MAIN.TopToolStripPanelVisible = False
+            '
+            'TP_MAIN
+            '
+            TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
+            TP_MAIN.ColumnCount = 1
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
+            TP_MAIN.Controls.Add(Me.TXT_PATH_CURR, 0, 0)
+            TP_MAIN.Controls.Add(Me.TXT_PATH_NEW, 0, 1)
+            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_MAIN.Location = New System.Drawing.Point(0, 0)
+            TP_MAIN.Name = "TP_MAIN"
+            TP_MAIN.RowCount = 3
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.Size = New System.Drawing.Size(484, 84)
+            TP_MAIN.TabIndex = 0
+            '
+            'TXT_PATH_CURR
+            '
+            Me.TXT_PATH_CURR.CaptionText = "Current"
+            Me.TXT_PATH_CURR.CaptionWidth = 50.0R
+            Me.TXT_PATH_CURR.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_PATH_CURR.Location = New System.Drawing.Point(4, 4)
+            Me.TXT_PATH_CURR.Name = "TXT_PATH_CURR"
+            Me.TXT_PATH_CURR.Size = New System.Drawing.Size(476, 22)
+            Me.TXT_PATH_CURR.TabIndex = 0
+            Me.TXT_PATH_CURR.TextBoxReadOnly = True
+            '
+            'TXT_PATH_NEW
+            '
+            ActionButton1.BackgroundImage = CType(resources.GetObject("ActionButton1.BackgroundImage"), System.Drawing.Image)
+            ActionButton1.Name = "Refresh"
+            ActionButton1.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton2.BackgroundImage = CType(resources.GetObject("ActionButton2.BackgroundImage"), System.Drawing.Image)
+            ActionButton2.Name = "Open"
+            ActionButton2.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            Me.TXT_PATH_NEW.Buttons.Add(ActionButton1)
+            Me.TXT_PATH_NEW.Buttons.Add(ActionButton2)
+            Me.TXT_PATH_NEW.CaptionText = "New"
+            Me.TXT_PATH_NEW.CaptionWidth = 50.0R
+            Me.TXT_PATH_NEW.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_PATH_NEW.Location = New System.Drawing.Point(4, 33)
+            Me.TXT_PATH_NEW.Name = "TXT_PATH_NEW"
+            Me.TXT_PATH_NEW.Size = New System.Drawing.Size(476, 22)
+            Me.TXT_PATH_NEW.TabIndex = 1
+            '
+            'SplitCollectionUserInfoPathForm
+            '
+            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
+            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
+            Me.ClientSize = New System.Drawing.Size(484, 84)
+            Me.Controls.Add(CONTAINER_MAIN)
+            Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
+            Me.Icon = Global.SCrawler.My.Resources.Resources.UsersIcon_32
+            Me.KeyPreview = True
+            Me.MaximizeBox = False
+            Me.MaximumSize = New System.Drawing.Size(500, 123)
+            Me.MinimizeBox = False
+            Me.MinimumSize = New System.Drawing.Size(500, 123)
+            Me.Name = "SplitCollectionUserInfoPathForm"
+            Me.ShowInTaskbar = False
+            Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
+            Me.Text = "User paths"
+            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
+            CONTAINER_MAIN.ResumeLayout(False)
+            CONTAINER_MAIN.PerformLayout()
+            TP_MAIN.ResumeLayout(False)
+            CType(Me.TXT_PATH_CURR, System.ComponentModel.ISupportInitialize).EndInit()
+            CType(Me.TXT_PATH_NEW, System.ComponentModel.ISupportInitialize).EndInit()
+            Me.ResumeLayout(False)
+
+        End Sub
+
+        Private WithEvents TXT_PATH_CURR As PersonalUtilities.Forms.Controls.TextBoxExtended
+        Private WithEvents TXT_PATH_NEW As PersonalUtilities.Forms.Controls.TextBoxExtended
+    End Class
+End Namespace

SCrawler/API/BaseObjects/SplitCollectionUserInfoPathForm.resx

@@ -0,0 +1,154 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" use="required" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
+  <data name="ActionButton1.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
+        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
+        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
+        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
+        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
+        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
+        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
+        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
+        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
+        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
+        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
+        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
+</value>
+  </data>
+  <data name="ActionButton2.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
+        WQwhyWIyJIUW5NqyPb7oCVtIlhVTwYf8nv7/t2zJagel9KmqKsIACYL9RjI8UHz5zshougZr/AEvbxEP
+        aZCDBY3VslixaJvX3wzkkDiOwbZtDRGA5vdNAg+TL27qgmt5XkBG/gTdAG7Gt+3PP9oOaEGFCVEC6rp+
+        5g9MfM/c5e4OsEZMZkQEtGL5H2DdZ5JRArDwPA+iKII0TfkC9vroC9j5vq8JTWw3WzWgLMtZGIaa0MR8
+        vlAD8PYlSaIJTTiOowY0p0Bc19XEJo6HE59FAPuMzyAINKGJ1XLFZxHALtMrnkBXOIQIIIQ8YvF/KrgB
+        cMaRN0UdBBkAAAAASUVORK5CYII=
+</value>
+  </data>
+</root>

SCrawler/API/BaseObjects/SplitCollectionUserInfoPathForm.vb

@@ -0,0 +1,68 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Forms.Controls.Base
+Imports SCrawler.DownloadObjects.STDownloader
+Namespace API.Base
+    Friend Class SplitCollectionUserInfoPathForm
+        Private WithEvents MyDefs As DefaultFormOptions
+        Friend User As SplitCollectionUserInfo
+        Private ReadOnly UserNewPathDef As String
+        Friend Sub New(ByVal _User As SplitCollectionUserInfo)
+            InitializeComponent()
+            MyDefs = New DefaultFormOptions(Me, Settings.Design)
+            User = _User
+            UserNewPathDef = User.UserNew.File.CutPath.PathWithSeparator
+        End Sub
+        Private Sub SplitCollectionUserInfoPathForm_Load(sender As Object, e As EventArgs) Handles Me.Load
+            With MyDefs
+                .MyViewInitialize()
+                .AddOkCancelToolbar()
+
+                TXT_PATH_CURR.Text = User.UserOrig.File.CutPath.PathWithSeparator
+                TXT_PATH_NEW.Text = UserNewPathDef
+
+                .MyFieldsCheckerE = New FieldsChecker
+                .MyFieldsCheckerE.AddControl(Of String)(TXT_PATH_NEW, "New path")
+                .MyFieldsCheckerE.EndLoaderOperations()
+
+                .EndLoaderOperations()
+            End With
+        End Sub
+        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
+            If MyDefs.MyFieldsChecker.AllParamsOK Then MyDefs.CloseForm()
+        End Sub
+        Private Sub TXT_PATH_NEW_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As ActionButtonEventArgs) Handles TXT_PATH_NEW.ActionOnButtonClick
+            Select Case e.DefaultButton
+                Case ActionButton.DefaultButtons.Refresh : TXT_PATH_NEW.Text = UserNewPathDef
+                Case ActionButton.DefaultButtons.Open
+                    Using ff As New Editors.GlobalLocationsChooserForm With {.MyInitialLocation = TXT_PATH_NEW.Text}
+                        ff.ShowDialog()
+                        If ff.DialogResult = DialogResult.OK Then
+                            Dim dest As DownloadLocation = ff.MyDestination
+                            If Not dest.Path.IsEmptyString Then
+                                Dim ph As PathMoverHandler = Editors.GlobalLocationsChooserForm.ModelHandler(dest.Model)
+                                If Not ph Is Nothing Then TXT_PATH_NEW.Text = ph.Invoke(User.UserNew, dest.Path.CSFileP).ToString
+                            End If
+                        End If
+                    End Using
+            End Select
+        End Sub
+        Private Sub TXT_PATH_NEW_ActionOnTextChanged(sender As Object, e As EventArgs) Handles TXT_PATH_NEW.ActionOnTextChanged
+            If Not MyDefs.Initializing Then
+                Dim f As SFile = TXT_PATH_NEW.Text.CSFileP
+                If Not f.IsEmptyString Then
+                    User.UserNew.SpecialPath = f
+                    User.UserNew.UpdateUserFile()
+                    User.Changed = Not User.UserNew.File.CutPath.PathWithSeparator = UserNewPathDef
+                End If
+            End If
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Facebook/Declarations.vb

@@ -11,9 +11,8 @@ Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Facebook
     Friend Module Declarations
-        Friend ReadOnly Regex_UserToken_dtsg As RParams = RParams.DMS("DTSGInitialData.:.?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
-        Friend ReadOnly Regex_UserToken_lsd As RParams = RParams.DMS("LSD.:.?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
         Friend ReadOnly Regex_UserID As RParams = RParams.DMS("userid.:.(\d+)", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly Regex_AppID As RParams = RParams.DMS("APP_ID.:.(\d+)", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
 
         Friend ReadOnly Regex_Photos_by As RParams = RParams.DMS("photos_by"",""id"":""([^""]+)", 1, EDP.ReturnValue)
         Friend ReadOnly Regex_FileName As RParams = RParams.DM("([^/\?]+\..{3,4})(?=(\?|\Z))", 0, EDP.ReturnValue)

SCrawler/API/Facebook/SiteSettings.vb

@@ -11,6 +11,7 @@ Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.Facebook
     <Manifest("AndyProgram_Facebook"), SavedPosts, SeparatedTasks(1), SpecialForm(False)>
     Friend Class SiteSettings : Inherits ThreadsNet.SiteSettings
@@ -18,7 +19,7 @@ Namespace API.Facebook
 #Region "Auth"
         <PropertyOption(AllowNull:=False, ControlText:="Accept", ControlToolTip:="Header 'Accept'", IsAuth:=True), ControlNumber(21), PXML, PClonable>
         Friend ReadOnly Property Header_Accept As PropertyValue
-        <PropertyOption(ControlText:="x-ig-app-id", AllowNull:=True, IsAuth:=True)>
+        <PropertyOption(ControlText:="x-ig-app-id", AllowNull:=True, IsAuth:=True), HiddenControl>
         Friend Overrides ReadOnly Property HH_IG_APP_ID As PropertyValue
             Get
                 Return __HH_IG_APP_ID
@@ -31,11 +32,11 @@ Namespace API.Facebook
         End Property
 #End Region
 #Region "Defaults"
-        <PropertyOption(ControlText:="Download photos", IsAuth:=False), PXML, PClonable>
+        <PropertyOption(ControlText:="Download photos", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property ParsePhotoBlock As PropertyValue
-        <PropertyOption(ControlText:="Download videos", IsAuth:=False), PXML, PClonable>
+        <PropertyOption(ControlText:="Download videos", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property ParseVideoBlock As PropertyValue
-        <PropertyOption(ControlText:="Download stories", IsAuth:=False), PXML, PClonable>
+        <PropertyOption(ControlText:="Download stories", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property ParseStoriesBlock As PropertyValue
 #End Region
 #End Region
@@ -74,7 +75,7 @@ Namespace API.Facebook
 #End Region
 #Region "BaseAuthExists, GetUserUrl, GetUserPostUrl, IsMyUser, IsMyImageVideo"
         Friend Overrides Function BaseAuthExists() As Boolean
-            Return Responser.CookiesExists And ACheck(HH_IG_APP_ID.Value) And CBool(DownloadData_Impl.Value)
+            Return Responser.CookiesExists And CBool(DownloadData_Impl.Value) 'And ACheck(HH_IG_APP_ID.Value)
         End Function
         Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
             Return DirectCast(User, UserData).GetProfileUrl

SCrawler/API/Facebook/UserData.vb

@@ -136,6 +136,7 @@ Namespace API.Facebook
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             If CBool(MySettings.DownloadData_Impl.Value) Then
                 Try
+                    If Responser.Headers.Value(IG.Header_IG_APP_ID).IsEmptyString Then Responser.Headers.Remove(IG.Header_IG_APP_ID)
                     ResetBaseTokens()
                     GetUserTokens(Token)
                     LoadSavePostsKV(True)
@@ -529,8 +530,14 @@ Namespace API.Facebook
                 Dim r$ = resp.GetResponse(URL)
                 If Not r.IsEmptyString Then
                     If Responser.CookiesExists Then Responser.Cookies.Update(resp.Cookies)
-                    Token_dtsg = RegexReplace(r, Regex_UserToken_dtsg)
-                    Token_lsd = RegexReplace(r, Regex_UserToken_lsd)
+                    ParseTokens(r, 0)
+                    Dim app_id$ = RegexReplace(r, Regex_AppID)
+                    If Not app_id.IsEmptyString Then
+                        If Not AEquals(Of String)(MySettings.HH_IG_APP_ID.Value, app_id) Then
+                            MySettings.HH_IG_APP_ID.Value = app_id
+                            Responser.Headers.Add(IG.Header_IG_APP_ID, app_id)
+                        End If
+                    End If
                     Token_Photosby = RegexReplace(r, Regex_Photos_by)
                     If StoryBucket.IsEmptyString Then StoryBucket = RegexReplace(r, Regex_StoryBucket)
                     If ID.IsEmptyString Then
@@ -568,14 +575,14 @@ Namespace API.Facebook
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchSite, "none"))
                     .Add("Sec-Fetch-User", "?1")
                     .Add("Upgrade-Insecure-Requests", 1)
-                    Dim h$ = Responser.Headers.Value(IG.Header_Browser)
-                    If Not h.IsEmptyString Then .Add(IG.Header_Browser, h)
-                    h = Responser.Headers.Value(IG.Header_BrowserExt)
-                    If Not h.IsEmptyString Then .Add(IG.Header_BrowserExt, h)
-                    h = Responser.Headers.Value(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform))
-                    If Not h.IsEmptyString Then .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform, h))
-                    h = Responser.Headers.Value(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatformVersion))
-                    If Not h.IsEmptyString Then .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatformVersion, h))
+                    Dim cloneHeader As Action(Of String) = Sub(ByVal hName As String)
+                                                               Dim hValue$ = Responser.Headers.Value(hName)
+                                                               If Not hValue.IsEmptyString Then .Add(hName, hValue)
+                                                           End Sub
+                    cloneHeader.Invoke(IG.Header_Browser)
+                    cloneHeader.Invoke(IG.Header_BrowserExt)
+                    cloneHeader.Invoke(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform).Name)
+                    cloneHeader.Invoke(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatformVersion).Name)
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaMobile, "?0"))
                     .Add("Sec-Ch-Ua-Model", "")
                 End With

SCrawler/API/Instagram/Declarations.vb

@@ -18,6 +18,8 @@ Namespace API.Instagram
         Friend ReadOnly ObtainMedia_SizeFuncPic_RegexP As RParams = RParams.DMS("_p(\d+)x(\d+)", 1, EDP.ReturnValue)
         Friend ReadOnly ObtainMedia_SizeFuncPic_RegexS As RParams = RParams.DMS("_s(\d+)x(\d+)", 1, EDP.ReturnValue)
         Friend Const PageTokenRegexPatternDefault As String = "\[\],{""token"":""(.*?)""},\d+\]"
+        Friend ReadOnly Regex_UserToken_dtsg As RParams = RParams.DMS("DTSGInitialData["":,.\[\]]*?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_UserToken_lsd As RParams = RParams.DMS("LSD["":,.\[\]]*?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
         Friend Sub UpdateResponser(ByVal Source As IResponse, ByRef Destination As Responser, ByVal UpdateWwwClaim As Boolean)
             Const r_wwwClaimName$ = "x-ig-set-www-claim"
             Const r_tokenName$ = SiteSettings.Header_CSRF_TOKEN_COOKIE

SCrawler/API/Instagram/SiteSettings.vb

@@ -14,6 +14,7 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Cookies
 Imports Download = SCrawler.Plugin.ISiteSettings.Download
+Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.Instagram
     <Manifest(InstagramSiteKey), SeparatedTasks(1), SavedPosts, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
@@ -54,6 +55,9 @@ Namespace API.Instagram
             End Function
         End Class
 #End Region
+#Region "Categories"
+        Private Const CAT_DOWN As String = "Download data"
+#End Region
 #Region "Authorization properties"
         Friend Const Header_IG_APP_ID As String = "x-ig-app-id"
         Friend Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim"
@@ -65,6 +69,10 @@ Namespace API.Instagram
         Friend Const Header_Platform_Verion As String = "Sec-Ch-Ua-Platform-Version"
         <PropertyOption(ControlText:="x-csrftoken", ControlToolTip:="Can be automatically extracted from cookies", IsAuth:=True, AllowNull:=True), ControlNumber(2), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_CSRF_TOKEN As PropertyValue
+        <CookieValueExtractor(NameOf(HH_CSRF_TOKEN))>
+        Private Function GetValueFromCookies(ByVal PropName As String, ByVal c As CookieKeeper) As String
+            Return c.GetCookieValue(Header_CSRF_TOKEN_COOKIE, PropName, NameOf(HH_CSRF_TOKEN))
+        End Function
         <PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_IG_APP_ID As PropertyValue
         <PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=True), ControlNumber(4), PClonable(Clone:=False)>
@@ -139,10 +147,19 @@ Namespace API.Instagram
         Friend ReadOnly Property USE_GQL As PropertyValue
 #End Region
 #Region "Download properties"
+        <PropertyOption(ControlText:="DownDetector",
+                        ControlToolTip:="Use 'DownDetector' to determine if the site is accessible. -1 to disable." & vbCr &
+                                        "The value represents the average number of error reports over the last 4 hours"),
+            PClonable, PXML, ControlNumber(17)>
+        Private ReadOnly Property DownDetectorValue As PropertyValue
+        <Provider(NameOf(DownDetectorValue), FieldsChecker:=True)>
+        Private ReadOnly Property DownDetectorValueProvider As IFormatProvider
+        <PropertyOption(ControlText:="Add 'DownDetector' information to the log."), PClonable, PXML, ControlNumber(18), HiddenControl>
+        Private ReadOnly Property DownDetectorValueAddToLog As PropertyValue
         Friend Const TimersUrgentTip As String = vbCr & "It is highly recommended not to change the default value."
         <PropertyOption(ControlText:="Request timer (any)",
                         ControlToolTip:="The timer (in milliseconds) that SCrawler should wait before executing the next request." &
-                        vbCr & "The default value is 1'000." & vbCr & "The minimum value is 0." & TimersUrgentTip, AllowNull:=False),
+                        vbCr & "The default value is 1'000." & vbCr & "The minimum value is 0." & TimersUrgentTip, AllowNull:=False, Category:=DN.CAT_Timers),
                         PXML, ControlNumber(19), PClonable>
         Friend ReadOnly Property RequestsWaitTimer_Any As PropertyValue
         <Provider(NameOf(RequestsWaitTimer_Any), FieldsChecker:=True)>
@@ -150,33 +167,33 @@ Namespace API.Instagram
         <PropertyOption(ControlText:="Request timer",
                         ControlToolTip:="The time value (in milliseconds) that the program will wait before processing the next 'Request time counter' request." &
                                         vbCr & "The default value is 1'000." & vbCr & "The minimum value is 100." & TimersUrgentTip,
-                        AllowNull:=False), PXML, ControlNumber(20), PClonable>
+                        AllowNull:=False, Category:=DN.CAT_Timers), PXML, ControlNumber(20), PClonable>
         Friend ReadOnly Property RequestsWaitTimer As PropertyValue
         <Provider(NameOf(RequestsWaitTimer), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerProvider As IFormatProvider
         <PropertyOption(ControlText:="Request timer counter",
                         ControlToolTip:="How many requests will be sent to Instagram before the program waits 'Request timer'." &
                                         vbCr & "The default value is 1." & vbCr & "The minimum value is 1." & TimersUrgentTip,
-                        AllowNull:=False, LeftOffset:=120), PXML, ControlNumber(21), PClonable>
+                        AllowNull:=False, LeftOffset:=120, Category:=DN.CAT_Timers), PXML, ControlNumber(21), PClonable>
         Friend ReadOnly Property RequestsWaitTimerTaskCount As PropertyValue
         <Provider(NameOf(RequestsWaitTimerTaskCount), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerTaskCountProvider As IFormatProvider
         <PropertyOption(ControlText:="Posts limit timer",
                         ControlToolTip:="The time value (in milliseconds) the program will wait before processing the next request after 195 requests." &
                                         vbCr & "The default value is 60'000." & vbCr & "The minimum value is 10'000." & TimersUrgentTip,
-                        AllowNull:=False), PXML, ControlNumber(22), PClonable>
+                        AllowNull:=False, Category:=DN.CAT_Timers), PXML, ControlNumber(22), PClonable>
         Friend ReadOnly Property SleepTimerOnPostsLimit As PropertyValue
         <Provider(NameOf(SleepTimerOnPostsLimit), FieldsChecker:=True)>
         Private ReadOnly Property SleepTimerOnPostsLimitProvider As IFormatProvider
-        <PropertyOption(ControlText:="Get timeline", ControlToolTip:="Default value for new users"), PXML, ControlNumber(23), PClonable>
+        <PropertyOption(ControlText:="Get timeline", ControlToolTip:="Default value for new users", Category:=DN.CAT_UserDefs), PXML, ControlNumber(23), PClonable>
         Friend ReadOnly Property GetTimeline As PropertyValue
-        <PropertyOption(ControlText:="Get reels", ControlToolTip:="Default value for new users"), PXML, ControlNumber(24), PClonable>
+        <PropertyOption(ControlText:="Get reels", ControlToolTip:="Default value for new users", Category:=DN.CAT_UserDefs), PXML, ControlNumber(24), PClonable>
         Friend ReadOnly Property GetReels As PropertyValue
-        <PropertyOption(ControlText:="Get stories", ControlToolTip:="Default value for new users"), PXML, ControlNumber(25), PClonable>
+        <PropertyOption(ControlText:="Get stories", ControlToolTip:="Default value for new users", Category:=DN.CAT_UserDefs), PXML, ControlNumber(25), PClonable>
         Friend ReadOnly Property GetStories As PropertyValue
-        <PropertyOption(ControlText:="Get stories: user", ControlToolTip:="Default value for new users"), PXML, ControlNumber(26), PClonable>
+        <PropertyOption(ControlText:="Get stories: user", ControlToolTip:="Default value for new users", Category:=DN.CAT_UserDefs), PXML, ControlNumber(26), PClonable>
         Friend ReadOnly Property GetStoriesUser As PropertyValue
-        <PropertyOption(ControlText:="Get tagged photos", ControlToolTip:="Default value for new users"), PXML, ControlNumber(27), PClonable>
+        <PropertyOption(ControlText:="Get tagged photos", ControlToolTip:="Default value for new users", Category:=DN.CAT_UserDefs), PXML, ControlNumber(27), PClonable>
         Friend ReadOnly Property GetTagged As PropertyValue
         <PropertyOption(ControlText:="Tagged notify limit",
                         ControlToolTip:="If the number of tagged posts exceeds this number you will be notified." & vbCr &
@@ -186,19 +203,19 @@ Namespace API.Instagram
         Private ReadOnly Property TaggedNotifyLimitProvider As IFormatProvider
 #End Region
 #Region "Download ready"
-        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download timeline"), PXML, ControlNumber(10), PClonable>
+        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download timeline", Category:=CAT_DOWN), PXML, ControlNumber(10), PClonable>
         Friend ReadOnly Property DownloadTimeline As PropertyValue
         <PXML> Private ReadOnly Property DownloadTimeline_Def As PropertyValue
-        <PropertyOption(ControlText:="Download reels", ControlToolTip:="Download reels"), PXML, ControlNumber(11), PClonable>
+        <PropertyOption(ControlText:="Download reels", ControlToolTip:="Download reels", Category:=CAT_DOWN), PXML, ControlNumber(11), PClonable>
         Friend ReadOnly Property DownloadReels As PropertyValue
         <PXML> Private ReadOnly Property DownloadReels_Def As PropertyValue
-        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download stories"), PXML, ControlNumber(12), PClonable>
+        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download stories", Category:=CAT_DOWN), PXML, ControlNumber(12), PClonable>
         Friend ReadOnly Property DownloadStories As PropertyValue
         <PXML> Private ReadOnly Property DownloadStories_Def As PropertyValue
-        <PropertyOption(ControlText:="Download stories: user", ControlToolTip:="Download stories (user)"), PXML, ControlNumber(13), PClonable>
+        <PropertyOption(ControlText:="Download stories: user", ControlToolTip:="Download stories (user)", Category:=CAT_DOWN), PXML, ControlNumber(13), PClonable>
         Friend ReadOnly Property DownloadStoriesUser As PropertyValue
         <PXML> Private ReadOnly Property DownloadStoriesUser_Def As PropertyValue
-        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged posts"), PXML, ControlNumber(14), PClonable>
+        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged posts", Category:=CAT_DOWN), PXML, ControlNumber(14), PClonable>
         Friend ReadOnly Property DownloadTagged As PropertyValue
         <PXML> Private ReadOnly Property DownloadTagged_Def As PropertyValue
 #End Region
@@ -348,8 +365,11 @@ Namespace API.Instagram
                         platform = .Value(Header_Platform_Verion)
                     End If
                     '.Add(Header_IG_WWW_CLAIM, 0)
+                    .Add("Origin", "https://www.instagram.com")
+                    .Add("authority", "www.instagram.com")
                     .Add("Dnt", 1)
-                    .Add("Dpr", 1)
+                    '.Add("Dpr", 1)
+                    .Remove("Dpr")
                     .Add("Sec-Ch-Ua-Mobile", "?0")
                     .Add("Sec-Ch-Ua-Model", """""")
                     .Add("Sec-Ch-Ua-Platform", """Windows""")
@@ -392,6 +412,9 @@ Namespace API.Instagram
             DownloadTagged = New PropertyValue(False)
             DownloadTagged_Def = New PropertyValue(DownloadTagged.Value, GetType(Boolean))
 
+            DownDetectorValue = New PropertyValue(20)
+            DownDetectorValueProvider = New TimersChecker(-1)
+            DownDetectorValueAddToLog = New PropertyValue(False)
             RequestsWaitTimer_Any = New PropertyValue(1000)
             RequestsWaitTimer_AnyProvider = New TimersChecker(0)
             RequestsWaitTimer = New PropertyValue(1000)
@@ -409,7 +432,7 @@ Namespace API.Instagram
             TaggedNotifyLimit = New PropertyValue(200)
             TaggedNotifyLimitProvider = New TaggedNotifyLimitChecker
 
-            DownloadingErrorDate = New PropertyValue(Now.AddYears(10), GetType(Date))
+            DownloadingErrorDate = New PropertyValue(Now.AddYears(-10), GetType(Date))
             LastDownloadDate = New PropertyValue(Now.AddDays(-1))
             LastRequestsCount = New PropertyValue(0)
             LastRequestsCountLabel = New PropertyValue(String.Empty, GetType(String))
@@ -420,9 +443,19 @@ Namespace API.Instagram
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "instagram.com/"), 1)
             ImageVideoContains = "instagram.com"
         End Sub
+        Private Const SettingsVersionCurrent As Integer = 2
         Friend Overrides Sub EndInit()
             Try : MyLastRequests.Add(LastDownloadDate.Value, LastRequestsCount.Value) : Catch : End Try
             If Not CBool(HH_IG_WWW_CLAIM_USE.Value) Then Responser.Headers.Remove(Header_IG_WWW_CLAIM)
+            If CInt(SettingsVersion.Value) < SettingsVersionCurrent Then
+                SettingsVersion.Value = SettingsVersionCurrent
+                HH_IG_WWW_CLAIM_UPDATE_INTERVAL.Value = 120
+                HH_IG_WWW_CLAIM_ALWAYS_ZERO.Value = False
+                HH_IG_WWW_CLAIM_RESET_EACH_SESSION.Value = True
+                HH_IG_WWW_CLAIM_RESET_EACH_TARGET.Value = True
+                HH_IG_WWW_CLAIM_USE.Value = True
+                HH_IG_WWW_CLAIM_USE_DEFAULT_ALGO.Value = True
+            End If
             MyBase.EndInit()
         End Sub
 #End Region
@@ -452,17 +485,101 @@ Namespace API.Instagram
         End Function
 #End Region
 #Region "Downloading"
+        Private ____DownloadStarted As Boolean = False
+        Private ____AvailableRequested As Boolean = False
+        Private ____AvailableSilent As Boolean = True
+        Private ____AvailableChecked As Boolean = False
+        Private ____AvailableResult As Boolean = False
+        Private Sub ResetDownloadOptions()
+            If ActiveJobs < 1 Then
+                ____DownloadStarted = False
+                ____AvailableRequested = False
+                ____AvailableChecked = False
+                ____AvailableSilent = True
+                ____AvailableResult = False
+                If ActiveSessionRequestsExists Then RefreshMyLastRequests(Now)
+                ActiveSessionRequestsExists = False
+                _NextWNM = UserData.WNM.Notify
+                _NextTagged = True
+                SkipUntilNextSession = False
+                AvailableText = String.Empty
+                ActiveJobs = 0
+            End If
+        End Sub
+        Friend Overrides Function Available(ByVal What As Download, ByVal Silent As Boolean) As Boolean
+            If MyBase.Available(What, Silent) And ActiveJobs < 2 Then
+                If CInt(DownDetectorValue.Value) >= 0 Then
+                    If ____DownloadStarted Then
+                        ____AvailableRequested = True
+                        ____AvailableSilent = Silent
+                        Return True
+                    Else
+                        Return AvailableImpl(What, Silent)
+                    End If
+                Else
+                    Return True
+                End If
+            Else
+                Return False
+            End If
+        End Function
+#Disable Warning IDE0060
+        Private Function AvailableImpl(ByVal What As Download, ByVal Silent As Boolean) As Boolean
+#Enable Warning
+            Try
+                AvailableText = String.Empty
+                If CInt(DownDetectorValue.Value) = -1 Then
+                    Return True
+                Else
+                    Dim dl As List(Of DownDetector.Data) = DownDetector.GetData("instagram")
+                    If dl.ListExists Then
+                        dl = dl.Take(4).ToList
+                        Dim avg% = dl.Average(Function(d) d.Value)
+                        If avg > CInt(DownDetectorValue.Value) Then
+                            AvailableText = "Over the past hour, Instagram has received an average of " &
+                                            avg.NumToString(New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral}) & " outage reports:" & vbCr &
+                                            dl.ListToString(vbCr)
+                            If CBool(DownDetectorValueAddToLog.Value) Then MyMainLOG = AvailableText
+                            If Silent Then
+                                Return False
+                            Else
+                                Return MsgBoxE({$"{AvailableText}{vbCr}{vbCr}Do you want to continue parsing Instagram data?",
+                                               "There are outage reports on Instagram"}, vbYesNo) = vbYes
+                            End If
+                        End If
+                    End If
+                    Return True
+                End If
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, "[API.Instagram.SiteSettings.Available]", True)
+            End Try
+        End Function
         Friend Property SkipUntilNextSession As Boolean = False
         Friend Overrides Function ReadyToDownload(ByVal What As Download) As Boolean
-            Return ActiveJobs < 2 AndAlso Not SkipUntilNextSession AndAlso ReadyForDownload AndAlso BaseAuthExists() AndAlso DownloadTimeline.Value
+            If ActiveJobs < 2 AndAlso Not SkipUntilNextSession AndAlso ReadyForDownload AndAlso BaseAuthExists() AndAlso CBool(DownloadTimeline.Value) Then
+                If ____DownloadStarted And ____AvailableRequested Then
+                    ____AvailableResult = AvailableImpl(What, ____AvailableSilent)
+                    ____AvailableChecked = True
+                    ____AvailableRequested = False
+                    Return ____AvailableResult
+                ElseIf ____AvailableChecked Then
+                    Return ____AvailableResult
+                Else
+                    Return True
+                End If
+            Else
+                Return False
+            End If
         End Function
         Private ActiveJobs As Integer = 0
         Private ActiveSessionDate As Date
+        Private ActiveSessionRequestsExists As Boolean = False
         Private _NextWNM As UserData.WNM = UserData.WNM.Notify
         Private _NextTagged As Boolean = True
         Friend Overrides Sub DownloadStarted(ByVal What As Download)
+            ResetDownloadOptions()
             ActiveJobs += 1
-            If ActiveJobs = 1 Then ActiveSessionDate = Now
+            If ActiveJobs = 1 Then ____DownloadStarted = True : ActiveSessionDate = Now
             If Not HH_IG_WWW_CLAIM_IS_ZERO AndAlso
                (
                     (CBool(HH_IG_WWW_CLAIM_USE_DEFAULT_ALGO.Value) AndAlso MyLastRequestsDate.AddMinutes(HH_IG_WWW_CLAIM_UPDATE_INTERVAL.Value) < Now) Or
@@ -494,6 +611,7 @@ Namespace API.Instagram
                 If _NextWNM = UserData.WNM.SkipTemp Or _NextWNM = UserData.WNM.SkipCurrent Then _NextWNM = UserData.WNM.Notify
                 _NextTagged = .TaggedCheckSession
                 MyLastRequestsCount = .RequestsCountSession
+                If .RequestsCountSession > 0 Then ActiveSessionRequestsExists = True
                 _FieldsChangerSuspended = True
                 HH_IG_WWW_CLAIM.Value = Responser.Headers.Value(Header_IG_WWW_CLAIM)
                 HH_CSRF_TOKEN.Value = Responser.Headers.Value(Header_CSRF_TOKEN)
@@ -501,11 +619,8 @@ Namespace API.Instagram
             End With
         End Sub
         Friend Overrides Sub DownloadDone(ByVal What As Download)
-            _NextWNM = UserData.WNM.Notify
-            _NextTagged = True
-            RefreshMyLastRequests(Now)
             ActiveJobs -= 1
-            SkipUntilNextSession = False
+            ResetDownloadOptions()
         End Sub
 #End Region
 #Region "Settings"
@@ -557,7 +672,7 @@ Namespace API.Instagram
                 If vals.Any(Function(v) Not v.ValueOld = v.ValueNew) OrElse
                    Not Responser.Cookies.ListEquals(____Cookies) Then HH_IG_WWW_CLAIM.Value = 0 : credentialsUpdated = True
                 If Responser.CookiesExists Then
-                    Dim csrf$ = If(Responser.Cookies.FirstOrDefault(Function(c) c.Name.StringToLower = Header_CSRF_TOKEN_COOKIE)?.Value, String.Empty)
+                    Dim csrf$ = GetValueFromCookies(NameOf(HH_CSRF_TOKEN), Responser.Cookies)
                     If Not csrf.IsEmptyString Then
                         If Not AEquals(Of String)(CStr(HH_CSRF_TOKEN.Value), csrf) Then credentialsUpdated = True
                         HH_CSRF_TOKEN.Value = csrf

SCrawler/API/Instagram/UserData.GQL.vb

@@ -297,37 +297,51 @@ Namespace API.Instagram
                     End With
                 End With
                 Dim r$ = Responser.GetResponse(MySiteSettings.GetUserUrl(Me))
-                If Not r.IsEmptyString Then
-                    Dim rr As RParams = RParams.DM(PageTokenRegexPatternDefault, 0, RegexReturn.List, EDP.ReturnValue)
-                    Dim tokens As List(Of String) = RegexReplace(r, rr)
-                    Dim tt$, ttVal$
-                    If tokens.ListExists Then
-                        With rr
-                            .Match = Nothing
-                            .MatchSub = 1
-                            .WhatGet = RegexReturn.Value
-                        End With
-                        For Each tt In tokens
-                            If Not Token_lsd.IsEmptyString And Not Token_dtsg.IsEmptyString Then
-                                Exit For
-                            Else
-                                ttVal = RegexReplace(tt, rr)
-                                If Not ttVal.IsEmptyString Then
-                                    If ttVal.Contains(":") Then
-                                        If Token_dtsg.IsEmptyString Then Token_dtsg = ttVal
-                                    Else
-                                        If Token_lsd.IsEmptyString Then Token_lsd = ttVal
-                                    End If
-                                End If
-                            End If
-                        Next
-                    End If
-                End If
+                ParseTokens(r, 0)
             Catch ex As Exception
             Finally
                 ChangeResponserMode(_UseGQL, Not _UseGQL)
             End Try
         End Sub
+        Protected Sub ParseTokens(ByVal r As String, ByVal Attempt As Integer)
+            Try
+                If Not r.IsEmptyString Then
+                    ResetBaseTokens()
+                    Select Case Attempt
+                        Case 0
+                            Dim rr As RParams = RParams.DM(PageTokenRegexPatternDefault, 0, RegexReturn.List, EDP.ReturnValue)
+                            Dim tokens As List(Of String) = RegexReplace(r, rr)
+                            Dim tt$, ttVal$
+                            If tokens.ListExists Then
+                                With rr
+                                    .Match = Nothing
+                                    .MatchSub = 1
+                                    .WhatGet = RegexReturn.Value
+                                End With
+                                For Each tt In tokens
+                                    If Not Token_lsd.IsEmptyString And Not Token_dtsg.IsEmptyString Then
+                                        Exit For
+                                    Else
+                                        ttVal = RegexReplace(tt, rr)
+                                        If Not ttVal.IsEmptyString Then
+                                            If ttVal.Contains(":") Then
+                                                If Token_dtsg.IsEmptyString Then Token_dtsg = ttVal
+                                            Else
+                                                If Token_lsd.IsEmptyString Then Token_lsd = ttVal
+                                            End If
+                                        End If
+                                    End If
+                                Next
+                            End If
+                        Case 1
+                            Token_dtsg = RegexReplace(r, Regex_UserToken_dtsg)
+                            Token_lsd = RegexReplace(r, Regex_UserToken_lsd)
+                    End Select
+                    If Not ValidateBaseTokens() And Attempt = 0 Then ParseTokens(r, Attempt + 1)
+                End If
+            Catch
+            End Try
+        End Sub
 #End Region
     End Class
 End Namespace

SCrawler/API/Instagram/UserData.vb

@@ -862,15 +862,20 @@ NextPageBlock:
         Protected DefaultParser_IgnorePass As Boolean = False
         Private ReadOnly DefaultParser_PostUrlCreator_Default As Func(Of PostKV, String) = Function(post) $"https://www.instagram.com/p/{post.Code}/"
         Protected DefaultParser_PostUrlCreator As Func(Of PostKV, String) = Function(post) $"https://www.instagram.com/p/{post.Code}/"
+        Protected DefaultParser_Pinned As Func(Of IEnumerable(Of EContainer), Integer, Boolean) = Nothing
+        Protected DefaultParser_SkipPost As Func(Of IEnumerable(Of EContainer), Integer, PostKV, Boolean) = Nothing
         Protected Function DefaultParser(ByVal Items As IEnumerable(Of EContainer), ByVal Section As Sections, ByVal Token As CancellationToken,
                                          Optional ByVal SpecFolder As String = Nothing, Optional ByVal State As UStates = UStates.Unknown,
                                          Optional ByVal Attempts As Integer = 0) As Boolean
             ThrowAny(Token)
-            If Items.Count > 0 Then
+            If Items.ListExists Then
                 Dim PostIDKV As PostKV
                 Dim Pinned As Boolean
                 Dim PostDate$, PostOriginUrl$
-                Dim before%
+                Dim i%, before%
+                Dim usePinFunc As Boolean = Not DefaultParser_Pinned Is Nothing
+                Dim skipPostFuncExists As Boolean = Not DefaultParser_SkipPost Is Nothing
+                Dim nn As EContainer
                 If SpecFolder.IsEmptyString Then
                     Select Case Section
                         Case Sections.Tagged : SpecFolder = TaggedFolder
@@ -879,15 +884,22 @@ NextPageBlock:
                     End Select
                 End If
                 ProgressPre.ChangeMax(Items.Count)
-                For Each nn In Items
+                For i = 0 To Items.Count - 1
+                    nn = Items(i)
                     ProgressPre.Perform()
                     With If(Not DefaultParser_ElemNode Is Nothing, nn.ItemF(DefaultParser_ElemNode), nn)
                         If .ListExists Then
                             PostIDKV = New PostKV(.Value("code"), .Value("id"), Section)
                             PostOriginUrl = DefaultParser_PostUrlCreator(PostIDKV)
-                            Pinned = .Contains("timeline_pinned_user_ids")
-                            If (Section = Sections.Timeline And Not DefaultParser_IgnorePass) AndAlso PostKvExists(PostIDKV) Then
-                                If Not Pinned Then Return False
+                            'Pinned = .Contains("timeline_pinned_user_ids")
+                            If usePinFunc Then
+                                Pinned = DefaultParser_Pinned.Invoke(Items, i)
+                            Else
+                                Pinned = If(.Item("timeline_pinned_user_ids")?.Count, 0) > 0
+                            End If
+                            If skipPostFuncExists AndAlso DefaultParser_SkipPost.Invoke(Items, i, PostIDKV) Then
+                            ElseIf Not DefaultParser_IgnorePass AndAlso PostKvExists(PostIDKV) Then
+                                If Not Section = Sections.Timeline OrElse Not Pinned Then Return False
                             Else
                                 _TempPostsList.Add(PostIDKV.ID)
                                 PostsKVIDs.ListAddValue(PostIDKV, LNC)
@@ -1233,6 +1245,7 @@ NextPageBlock:
         Protected Overrides Sub EraseData_AdditionalDataFiles()
             Dim f As SFile = MyFilePostsKV
             If f.Exists Then f.Delete(SFO.File, SFODelete.DeleteToRecycleBin, EDP.ReturnValue)
+            FirstLoadingDone = False
         End Sub
 #End Region
 #Region "Exceptions"
@@ -1268,6 +1281,9 @@ NextPageBlock:
             ElseIf Responser.StatusCode = 560 Or Responser.StatusCode = HttpStatusCode.InternalServerError Then '560, 500
                 MySiteSettings.SkipUntilNextSession = True
                 Err5xx = Responser.StatusCode
+            ElseIf Responser.StatusCode = -1 And Responser.Status = -1 Then
+                MySiteSettings.SkipUntilNextSession = True
+                Err5xx = Responser.StatusCode
             Else
                 MyMainLOG = $"Something is wrong. Your credentials may have expired [{CInt(Responser.StatusCode)}/{CInt(Responser.Status)}]: {ToString()} [{s}]"
                 DisableSection(s)

SCrawler/API/JustForFans/SiteSettings.vb

@@ -20,6 +20,10 @@ Namespace API.JustForFans
         Friend ReadOnly Property UserID As PropertyValue
         <PropertyOption, PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property UserHash4 As PropertyValue
+        <CookieValueExtractor(NameOf(UserHash4))>
+        Private Function GetValueFromCookies(ByVal PropName As String, ByVal c As CookieKeeper) As String
+            Return c.GetCookieValue(UserHash4_CookieName, PropName, NameOf(UserHash4))
+        End Function
         <PropertyOption(ControlText:="Accept", ControlToolTip:="Header 'Accept'"), PClonable>
         Friend ReadOnly Property HeaderAccept As PropertyValue
         <PropertyOption(InheritanceName:=SettingsCLS.HEADER_DEF_UserAgent), PClonable, PXML(OnlyForChecked:=True)>
@@ -61,7 +65,7 @@ Namespace API.JustForFans
         Private Sub UpdateUserHash4()
             If Responser.CookiesExists Then
                 Dim hv_current$ = UserHash4.Value
-                Dim hv_cookie$ = If(Responser.Cookies.FirstOrDefault(Function(cc) cc.Name.ToLower = UserHash4_CookieName)?.Value, String.Empty)
+                Dim hv_cookie$ = GetValueFromCookies(NameOf(UserHash4), Responser.Cookies)
                 If Not hv_cookie.IsEmptyString And Not hv_cookie = hv_current And Responser.Cookies.Changed Then UserHash4.Value = hv_cookie
             End If
         End Sub

SCrawler/API/Mastodon/EditorExchangeOptions.vb

@@ -14,6 +14,7 @@ Namespace API.Mastodon
         <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelProfile As Boolean
         <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelSearch As Boolean
         <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelForceApply As Boolean
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelLikes As Boolean
         Friend Sub New(ByVal s As SiteSettings)
             MyBase.New(s)
         End Sub

SCrawler/API/Mastodon/SiteSettings.vb

@@ -63,15 +63,15 @@ Namespace API.Mastodon
         End Sub
 #End Region
 #Region "Other properties"
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsDownloadCaption), PXML, PClonable>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsDownloadCaption, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsDownload As PropertyValue
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip), PXML, PClonable>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsSpecialFolder As PropertyValue
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip), PXML, PClonable>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsPrefix As PropertyValue
         <Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)>
         Private ReadOnly Property GifStringChecker As IFormatProvider
-        <PropertyOption(IsAuth:=False, ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip), PXML, PClonable>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property UseMD5Comparison As PropertyValue
         <PropertyOption(IsAuth:=False, ControlText:="User related to my domain",
                         ControlToolTip:="Open user profiles and user posts through my domain."), PXML, PClonable>

SCrawler/API/OnlyFans/Declarations.vb

@@ -11,5 +11,6 @@ Namespace API.OnlyFans
     Friend Module Declarations
         Friend ReadOnly DateProvider As New ADateTime("O")
         Friend ReadOnly RegExPostID As RParams = RParams.DM("(?<=onlyfans\.com/)(\d+)", 0, EDP.ReturnValue)
+        Friend Property Rules As DynamicRulesEnv
     End Module
 End Namespace

SCrawler/API/OnlyFans/DynamicRules.txt

@@ -0,0 +1,7 @@
+https://github.com/datawhores/onlyfans-dynamic-rules/blob/main/dynamicRules.json
+https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/main/dynamicRules.json
+https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/patch-1/dynamicRules.json
+https://github.com/DATAHOARDERS/dynamic-rules/blob/main/onlyfans.json
+https://github.com/DIGITALCRIMINAL/dynamic-rules/blob/main/onlyfans.json
+https://github.com/deviint/onlyfans-dynamic-rules/blob/main/dynamicRules.json
+https://github.com/rafa-9/dynamic-rules/blob/main/rules.json

SCrawler/API/OnlyFans/DynamicRulesAll.txt

@@ -0,0 +1,11 @@
+https://github.com/datawhores/onlyfans-dynamic-rules/blob/main/dynamicRules.json
+https://github.com/datawhores/onlyfans-dynamic-rules/blob/main/rules.json
+https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/main/dynamicRules.json
+https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/patch-1/dynamicRules.json
+https://github.com/DATAHOARDERS/dynamic-rules/blob/main/onlyfans.json
+https://github.com/DIGITALCRIMINAL/dynamic-rules/blob/main/onlyfans.json
+https://github.com/deviint/onlyfans-dynamic-rules/blob/main/dynamicRules.json
+https://github.com/rafa-9/dynamic-rules/blob/main/rules.json
+
+https://github.com/SneakyOvis/onlyfans-dynamic-rules/blob/main/rules.json
+https://github.com/Growik/onlyfans-dynamic-rules/blob/main/rules.json

SCrawler/API/OnlyFans/DynamicRulesEnv.vb

@@ -0,0 +1,753 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.XML.Base
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Clients.Base
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports System.Text.RegularExpressions
+Namespace API.OnlyFans
+    Friend Structure DynamicRulesValue : Implements IComparable(Of DynamicRulesValue), IEquatable(Of DynamicRulesValue), IEContainerProvider
+#Region "XML names"
+        Private Const Name_UrlRepo As String = "UrlRepo"
+        Private Const Name_UrlRaw As String = "UrlRaw"
+        Private Const Name_UrlLatestCommit As String = "UrlLatestCommit"
+        Private Const Name_UpdatedAt As String = "UpdatedAt"
+        Private Const Name_Broken As String = "Broken"
+        Private Const Name_Exists As String = "Exists"
+#End Region
+#Region "Declarations"
+        Friend UrlRepo As String
+        Friend UrlRaw As String
+        Friend UrlLatestCommit As String
+        Friend UpdatedAt As Date
+        Friend Broken As Boolean
+        Friend Exists As Boolean
+        Friend ReadOnly Property Valid As Boolean
+            Get
+                Return Not UrlRepo.IsEmptyString And Not UrlRaw.IsEmptyString
+            End Get
+        End Property
+#End Region
+#Region "Initializers"
+        Friend Sub New(ByVal e As EContainer)
+            UrlRepo = e.Value(Name_UrlRepo)
+            UrlRaw = e.Value(Name_UrlRaw)
+            UrlLatestCommit = e.Value(Name_UrlLatestCommit)
+            UpdatedAt = e.Value(Name_UpdatedAt).ToDateDef(Now.AddYears(-10))
+            Broken = e.Value(Name_Broken).FromXML(Of Boolean)(False)
+            Exists = e.Value(Name_Exists).FromXML(Of Boolean)(True)
+        End Sub
+        Public Shared Widening Operator CType(ByVal e As EContainer) As DynamicRulesValue
+            Return New DynamicRulesValue(e)
+        End Operator
+        Public Shared Widening Operator CType(ByVal rule As DynamicRulesValue) As String
+            Return rule.ToString
+        End Operator
+#End Region
+#Region "Base functions"
+        Public Overrides Function GetHashCode() As Integer
+            Return ToString.GetHashCode
+        End Function
+        Public Overrides Function ToString() As String
+            Return UrlRaw
+        End Function
+#End Region
+#Region "IComparable Support"
+        Private Function CompareTo(ByVal Other As DynamicRulesValue) As Integer Implements IComparable(Of DynamicRulesValue).CompareTo
+            Return UpdatedAt.CompareTo(Other.UpdatedAt) * -1
+        End Function
+#End Region
+#Region "IEquatable Support"
+        Public Overloads Overrides Function Equals(ByVal Obj As Object) As Boolean
+            If Not IsNothing(Obj) Then
+                If TypeOf Obj Is String Then
+                    Dim _obj$ = CStr(Obj).StringTrim.StringToLower
+                    Return UrlRepo = _obj Or UrlRaw = _obj
+                Else
+                    Return Equals(DirectCast(Obj, DynamicRulesValue))
+                End If
+            Else
+                Return False
+            End If
+        End Function
+        Friend Overloads Function Equals(ByVal Other As DynamicRulesValue) As Boolean Implements IEquatable(Of DynamicRulesValue).Equals
+            Return UrlRepo = Other.UrlRepo Or UrlRaw = Other.UrlRaw
+        End Function
+#End Region
+#Region "IEContainerProvider Support"
+        Private Function ToEContainer(Optional ByVal e As ErrorsDescriber = Nothing) As EContainer Implements IEContainerProvider.ToEContainer
+            Return New EContainer("Rule") From {
+                New EContainer(Name_UrlRepo, UrlRepo),
+                New EContainer(Name_UrlRaw, UrlRaw),
+                New EContainer(Name_UrlLatestCommit, UrlLatestCommit),
+                New EContainer(Name_UpdatedAt, UpdatedAt.ToStringDateDef),
+                New EContainer(Name_Broken, Broken.BoolToInteger),
+                New EContainer(Name_Exists, Exists.BoolToInteger)
+            }
+        End Function
+#End Region
+    End Structure
+    Friend Class DynamicRulesEnv : Implements ICopier, IEnumerable(Of DynamicRulesValue), IMyEnumerator(Of DynamicRulesValue), IDisposable
+        Friend Enum Modes As Integer
+            List = 0
+            Personal = 1
+        End Enum
+#Region "Constants"
+        Friend Const UpdateIntervalDefault As Integer = 1440 '60 * 24
+        Friend Const DynamicRulesConfigNodeName_URL As String = "DYNAMIC_GENERIC_URL"
+        Friend Const DynamicRulesConfigNodeName_RULES As String = "DYNAMIC_RULE"
+
+        Friend Const DynamicRulesConfig_Mode_NodeName As String = "dynamic-mode-default"
+        'Friend Const DynamicRulesConfig_Mode_NodeValue As String = "generic"
+
+        Friend Const DynamicRulesConfigNodeName_URL_CONST_NAME As String = "RULE_VALUE"
+#End Region
+#Region "XML names"
+        Private Const Name_LastUpdateTimeFile As String = "LastUpdateTimeFile"
+        Private Const Name_LastUpdateTimeRules As String = "LastUpdateTimeRules"
+        Private Const Name_ProtectFile As String = "ProtectFile"
+        Private Const Name_UpdateInterval As String = "UpdateInterval"
+        Private Const Name_Mode As String = "Mode"
+        Private Const Name_PersonalRule As String = "PersonalRule"
+        Private Const Name_RulesForceUpdateRequired As String = "RulesForceUpdateRequired"
+        Private Const Name_AddErrorsToLog As String = "AddErrorsToLog"
+        Private Const Name_ConfigLastDateUpdate As String = "ConfigLastDateUpdate"
+        Private Const Name_ConfigAutoUpdate As String = "ConfigAutoUpdate"
+        Private Const Name_RulesConfigManualMode As String = "RulesConfigManualMode"
+        Private Const Name_RulesUpdateConst As String = "RulesUpdateConst"
+        Private Const Name_RulesReplaceConfig As String = "RulesReplaceConfig"
+#End Region
+#Region "Declarations"
+        Private ReadOnly Rules As List(Of DynamicRulesValue)
+        Friend ReadOnly Property RulesConstants As Dictionary(Of String, String)
+#Region "Regex patterns"
+        Private ReadOnly ReplacePattern_RepoToRaw As RParams
+        Private ReadOnly ReplacePattern_RawToRepo As RParams
+        Private ReadOnly ReplacePattern_JsonInfo As RParams
+        Private ReadOnly ConfigRulesExtract As RParams
+#End Region
+#Region "Dates"
+        Private LastUpdateTimeFile As Date = Now.AddYears(-1)
+        Private LastUpdateTimeRules As Date = Now.AddYears(-1)
+#End Region
+#Region "Files"
+        Friend ReadOnly OFScraperConfigPatternFile As SFile = $"{SettingsFolderName}\OFScraperConfigPattern.json"
+        Friend ReadOnly OFScraperConfigPatternFileConst As SFile = $"{SettingsFolderName}\OFScraperConfigPatternConstants.txt"
+        Friend ReadOnly Property AuthFile As New SFile($"{SettingsFolderName}\OnlyFans_Auth.json")
+        Private ReadOnly DynamicRulesFile As SFile
+        Private ReadOnly DynamicRulesXml As SFile
+        Private Shared ReadOnly Property DynamicRulesFileImpl As SFile
+            Get
+                Return $"{SettingsFolderName}\OnlyFansDynamicRules.txt"
+            End Get
+        End Property
+        Friend Shared Sub ValidateRulesFile()
+            Dim f As SFile = DynamicRulesFileImpl
+            If Not f.Exists Then TextSaver.SaveTextToFile(My.Resources.OFResources.DynamicRules, DynamicRulesFileImpl, True)
+        End Sub
+        Friend Property ProtectFile As Boolean = False
+#End Region
+        Friend Property UpdateInterval As Integer = UpdateIntervalDefault
+        Friend Property Mode As Modes = Modes.List
+        Friend Property PersonalRule As String = String.Empty
+        Friend Property RulesForceUpdateRequired As Boolean = False
+        Friend Property RulesUpdateConst As Boolean = True
+        Friend Property RulesReplaceConfig As Boolean = True
+        Private ReadOnly Responser As New Responser With {.Accept = "application/json"}
+        Private ReadOnly RulesLinesComparer As New FComparer(Of String)(Function(x, y) x.StringToLower = y.StringToLower)
+        Private ReadOnly OFLOG As TextSaver
+        Private ReadOnly OFError As ErrorsDescriber
+        Friend Property AddErrorsToLog As Boolean = True
+        Friend Property NeedToSave As Boolean = False
+        Private ReadOnly Property ConfigAddress As DynamicRulesValue
+        Private ReadOnly Property ConfigConstAddress As DynamicRulesValue
+        Private Property ConfigLastDateUpdate As Date = Now.AddYears(-1)
+        Friend Property ConfigAutoUpdate As Boolean = True
+        Friend Property RulesConfigManualMode As Boolean = True
+#End Region
+#Region "Current, Item, Count"
+        Private _CurrentRule As DynamicRulesValue
+        Private _CurrentContainer As EContainer
+        Private _CurrentContainerRulesText As String = String.Empty
+        Friend ReadOnly Property CurrentRule As DynamicRulesValue
+            Get
+                Return _CurrentRule
+            End Get
+        End Property
+        Friend ReadOnly Property CurrentContainer As EContainer
+            Get
+                Return _CurrentContainer
+            End Get
+        End Property
+        Friend ReadOnly Property CurrentContainerRulesText As String
+            Get
+                If _CurrentContainerRulesText.IsEmptyString AndAlso AuthFile.Exists Then _
+                   _CurrentContainerRulesText = AuthFile.GetText(OFError).StringTrim
+                Return _CurrentContainerRulesText
+            End Get
+        End Property
+        Friend ReadOnly Property Exists As Boolean
+            Get
+                Return CurrentContainer.ListExists
+            End Get
+        End Property
+        Default Friend ReadOnly Property Item(ByVal Index As Integer) As DynamicRulesValue Implements IMyEnumerator(Of DynamicRulesValue).MyEnumeratorObject
+            Get
+                Return Rules(Index)
+            End Get
+        End Property
+        Friend ReadOnly Property Count As Integer Implements IMyEnumerator(Of DynamicRulesValue).MyEnumeratorCount
+            Get
+                Return Rules.Count
+            End Get
+        End Property
+#End Region
+#Region "Initializer"
+        Friend Sub New()
+            Rules = New List(Of DynamicRulesValue)
+            DynamicRulesFile = DynamicRulesFileImpl
+            DynamicRulesXml = DynamicRulesFile
+            DynamicRulesXml.Extension = "xml"
+            ReplacePattern_RepoToRaw = New RParams("(.*github.com/([^/]+)/([^/]+)/blob/(.+))", Nothing, 0,
+                                                   RegexReturn.ReplaceChangeListMatch, EDP.ReturnValue) With {
+                                                   .PatternReplacement = "https://raw.githubusercontent.com/{2}/{3}/{4}"}
+            ReplacePattern_JsonInfo = ReplacePattern_RepoToRaw.Copy
+            ReplacePattern_JsonInfo.PatternReplacement = "https://github.com/{2}/{3}/latest-commit/{4}"
+            ReplacePattern_RawToRepo = ReplacePattern_RepoToRaw.Copy
+            ReplacePattern_RawToRepo.Pattern = "(.*raw.githubusercontent.com/([^/]+)/([^/]+)/([^/]+)/(.+))"
+            ReplacePattern_RawToRepo.PatternReplacement = "https://github.com/{2}/{3}/blob/{4}/{5}"
+            ConfigRulesExtract = RParams.DMS("DYNAMIC_RULE"":(\{.+?\}[\r\n]+)", 1, RegexOptions.Singleline, EDP.ReturnValue)
+            OFLOG = New TextSaver($"LOGs\OF_{Now:yyyyMMdd_HHmmss}.txt") With {.LogMode = True, .AutoSave = True, .AutoClear = True}
+            AddHandler OFLOG.TextSaved, AddressOf OFLOG_TextSaved
+            OFError = New ErrorsDescriber(EDP.SendToLog + EDP.ReturnValue) With {.DeclaredMessage = New MMessage With {.Loggers = {OFLOG}, .Exists = True}}
+            Responser.DeclaredError = OFError
+            Responser.ProcessExceptionDecision =
+                Function(ByVal Status As IResponserStatus, ByVal NullArg As Object, ByVal CurrentError As ErrorsDescriber) As ErrorsDescriber
+                    If Status.StatusCode = Net.HttpStatusCode.NotFound Then
+                        CurrentError.SendToLogOnlyMessage = True
+                        Dim m As MMessage = CurrentError.DeclaredMessage.Clone
+                        m.Text = $"Nothing found at URL: {Responser.LatestUrlString}"
+                        CurrentError.DeclaredMessage = m
+                        Status.ErrorException = New ErrorsDescriberException(m.Text,,, Status.ErrorException) With {.ReplaceMainMessage = True}
+                    End If
+                    Return CurrentError
+                End Function
+            ConfigAddress = ParseURL("https://github.com/AAndyProgram/SCrawler/blob/main/SCrawler/API/OnlyFans/OFScraperConfigPattern.json")
+            ConfigConstAddress = ParseURL("https://github.com/AAndyProgram/SCrawler/blob/main/SCrawler/API/OnlyFans/OFScraperConfigPatternConstants.txt")
+            RulesConstants = New Dictionary(Of String, String)
+        End Sub
+#End Region
+#Region "Log handlers"
+        Private _OFLOG_ProcessNotify As Boolean = True
+        Private Sub OFLOG_TextSaved(sender As Object, e As EventArgs)
+            If _OFLOG_ProcessNotify And AddErrorsToLog Then _OFLOG_ProcessNotify = False : MyMainLOG = $"The OnlyFans log contains errors: {OFLOG.File}"
+        End Sub
+#End Region
+#Region "ParseURL"
+        Private Const SiteGitHub As String = "github.com"
+        Private Const SiteGitHubRaw As String = "raw.githubusercontent.com"
+        Friend Function ParseURL(ByVal URL As String) As DynamicRulesValue
+            URL = URL.StringTrim
+            If Not URL.IsEmptyString Then
+                Dim r As New DynamicRulesValue
+                Dim rGet As Func(Of String, RParams, String) = Function(__url, pattern) DirectCast(RegexReplace(__url, pattern), IEnumerable(Of String)).FirstOrDefault
+                If URL.ToLower.Contains(SiteGitHubRaw) Then
+                    r.UrlRaw = URL
+                    r.UrlRepo = rGet(URL, ReplacePattern_RawToRepo)
+                ElseIf URL.ToLower.Contains(SiteGitHub) Then
+                    r.UrlRepo = URL
+                    r.UrlRaw = rGet(URL, ReplacePattern_RepoToRaw)
+                End If
+
+                If r.Valid Then
+                    r.UpdatedAt = Now.AddYears(-1)
+                    r.UrlLatestCommit = rGet(r.UrlRepo, ReplacePattern_JsonInfo)
+                    r.Exists = True
+                    Return r
+                End If
+            End If
+            Return Nothing
+        End Function
+#End Region
+#Region "GetFormat"
+        Private Shared ReadOnly Property ConfigNodes As String()
+            Get
+                Return {"advanced_options", "DYNAMIC_RULE"}
+            End Get
+        End Property
+        Private Const FormatMidPart As String = ":{0}:{1:x}:"
+        Private ReadOnly FormatExtract As RParams = RParams.DM("(\S+)\s*:\s*\{\s*\d?\s*\}\s*:\s*\{\s*\d?\s*:\s*x\s*\}\s*:\s*(\S+)", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
+        Private ReadOnly ContainerStrConv As New CustomProvider(Function(input) If(ACheck(Of Integer)(input), input, $"""{input}"""))
+        Private ReadOnly ContainerConv As New CustomProvider(Function(ByVal e As Object) As Object
+                                                                 With DirectCast(e, EContainer)
+                                                                     Dim value$ = String.Empty
+                                                                     If .ListExists Then
+                                                                         value = .Select(Function(ee) ee(0).Value).ListToStringE(",", ContainerStrConv, False, String.Empty, EDP.ReturnValue)
+                                                                         If Not value.IsEmptyString Then value = $"[{value}]"
+                                                                     Else
+                                                                         value = AConvert(Of String)(.Value, ContainerStrConv, String.Empty, EDP.SendToLog, EDP.ReturnValue)
+                                                                     End If
+                                                                     If Not value.IsEmptyString Then
+                                                                         value = $"""{ .Name}"": {value}"
+                                                                     Else
+                                                                         value = $"""{ .Name}"": """""
+                                                                     End If
+                                                                     Return value
+                                                                 End With
+                                                             End Function)
+        Friend Shared Function GetFormat(ByVal j As EContainer, Optional ByVal Check As Boolean = False,
+                                         Optional ByRef CheckResult As Boolean = False,
+                                         Optional ByVal TryConfig As Boolean = False, Optional ByRef IsConfig As Boolean = False) As String
+            Dim pattern$ = String.Empty
+            With If(TryConfig, j(ConfigNodes), j)
+                If .ListExists Then
+                    If Not .Value("format").IsEmptyString Then
+                        pattern = .Value("format").Replace("{}", "{0}").Replace("{:x}", "{1:x}")
+                    ElseIf Not .Value("prefix").IsEmptyString And Not .Value("suffix").IsEmptyString Then
+                        pattern = .Value("prefix") & FormatMidPart & .Value("suffix")
+                    ElseIf Not .Value("start").IsEmptyString And Not .Value("end").IsEmptyString Then
+                        pattern = .Value("start") & FormatMidPart & .Value("end")
+                    End If
+
+                    Dim result As Boolean = Not pattern.IsEmptyString And .Item("checksum_indexes").ListExists And
+                                            Not .Value("static_param").IsEmptyString And Not .Value("checksum_constant").IsEmptyString
+                    If Check Then CheckResult = result
+                    If Not result And Not TryConfig Then Return GetFormat(j, Check, CheckResult, True, IsConfig)
+                End If
+            End With
+            Return pattern
+        End Function
+        Private Function ConvertAuthText() As String
+            Dim result$ = String.Empty
+            With CurrentContainer
+                If .ListExists Then
+                    Dim f$ = GetFormat(.Self)
+                    If Not f.IsEmptyString Then
+                        Dim l As List(Of String) = RegexReplace(f, FormatExtract)
+                        If l.ListExists(3) Then
+                            Dim s$ = l(1), e$ = l(2)
+                            .Value("format") = s & FormatMidPart & e
+                            .Value("prefix") = s
+                            .Value("suffix") = e
+                            .Value("start") = s
+                            .Value("end") = e
+                            Dim t$ = .ListToStringE(",", ContainerConv, False)
+                            If Not t.IsEmptyString Then t = "{" & t & "}"
+                            Return t
+                        End If
+                    End If
+                End If
+            End With
+            Return String.Empty
+        End Function
+#End Region
+#Region "Load, Save"
+        Private Function GetTextLines(ByVal Input As String) As List(Of String)
+            If Not Input.IsEmptyString Then
+                Return ListAddList(Nothing, Input.StringTrim.Split(vbLf), LAP.NotContainsOnly, EDP.ReturnValue,
+                                   CType(Function(inp$) inp.StringTrim, Func(Of Object, Object)))
+            Else
+                Return New List(Of String)
+            End If
+        End Function
+        Private Sub ParseConsts(ByVal Source As String)
+            If Not Source.IsEmptyString Then
+                Dim l As List(Of String) = GetTextLines(Source)
+                Dim v$()
+                If l.ListExists Then
+                    RulesConstants.Clear()
+                    For Each value$ In l
+                        If Not value.IsEmptyString Then
+                            v = value.Split("=")
+                            If v.ListExists(2) Then RulesConstants.Add(v(0), v(1))
+                        End If
+                    Next
+                End If
+            End If
+        End Sub
+        Private Const RulesNode As String = "Rules"
+        Private _InitialValuesLoaded As Boolean = False
+        Private Sub LoadInitialValues()
+            If Not _InitialValuesLoaded Then
+                _InitialValuesLoaded = True
+
+                If Not OFScraperConfigPatternFile.Exists Then
+                    Dim t$ = Text.Encoding.UTF8.GetString(My.Resources.OFResources.OFScraperConfigPattern)
+                    TextSaver.SaveTextToFile(t, OFScraperConfigPatternFile, True)
+                End If
+
+                If Not OFScraperConfigPatternFileConst.Exists Then _
+                   TextSaver.SaveTextToFile(My.Resources.OFResources.OFScraperConfigPatternConstants, OFScraperConfigPatternFileConst, True)
+
+                If OFScraperConfigPatternFileConst.Exists Then ParseConsts(OFScraperConfigPatternFileConst.GetText(OFError))
+
+                If DynamicRulesXml.Exists Then
+                    Rules.Clear()
+                    Using x As New XmlFile(DynamicRulesXml, Protector.Modes.All, False) With {.XmlReadOnly = True, .AllowSameNames = True}
+                        x.LoadData(OFError)
+                        Dim dNull As Date = Now.AddYears(-1)
+                        LastUpdateTimeFile = x.Value(Name_LastUpdateTimeFile).ToDateDef(dNull)
+                        LastUpdateTimeRules = x.Value(Name_LastUpdateTimeRules).ToDateDef(dNull)
+                        ProtectFile = x.Value(Name_ProtectFile).FromXML(Of Boolean)(False)
+                        Mode = x.Value(Name_Mode).FromXML(Of Integer)(Modes.List)
+                        UpdateInterval = x.Value(Name_UpdateInterval).FromXML(Of Integer)(UpdateIntervalDefault)
+                        PersonalRule = x.Value(Name_PersonalRule)
+                        RulesForceUpdateRequired = x.Value(Name_RulesForceUpdateRequired).FromXML(Of Boolean)(False)
+                        RulesUpdateConst = x.Value(Name_RulesUpdateConst).FromXML(Of Boolean)(True)
+                        RulesReplaceConfig = x.Value(Name_RulesReplaceConfig).FromXML(Of Boolean)(True)
+                        AddErrorsToLog = x.Value(Name_AddErrorsToLog).FromXML(Of Boolean)(False)
+                        ConfigAutoUpdate = x.Value(Name_ConfigAutoUpdate).FromXML(Of Boolean)(True)
+                        RulesConfigManualMode = x.Value(Name_RulesConfigManualMode).FromXML(Of Boolean)(True)
+                        ConfigLastDateUpdate = x.Value(Name_ConfigLastDateUpdate).ToDateDef(Now.AddYears(-1))
+                        If x.Contains(RulesNode) Then Rules.ListAddList(x({RulesNode}), LAP.IgnoreICopier, OFError)
+                    End Using
+                End If
+            End If
+        End Sub
+        Friend Sub Save()
+            Using x As New XmlFile With {.AllowSameNames = True, .Name = "DynamicRules"}
+                x.Add(Name_LastUpdateTimeFile, LastUpdateTimeFile.ToStringDateDef)
+                x.Add(Name_LastUpdateTimeRules, LastUpdateTimeRules.ToStringDateDef)
+                x.Add(Name_ProtectFile, ProtectFile.BoolToInteger)
+                x.Add(Name_Mode, CInt(Mode))
+                x.Add(Name_UpdateInterval, UpdateInterval)
+                x.Add(Name_PersonalRule, PersonalRule)
+                x.Add(Name_RulesForceUpdateRequired, RulesForceUpdateRequired.BoolToInteger)
+                x.Add(Name_RulesUpdateConst, RulesUpdateConst.BoolToInteger)
+                x.Add(Name_RulesReplaceConfig, RulesReplaceConfig.BoolToInteger)
+                x.Add(Name_AddErrorsToLog, AddErrorsToLog.BoolToInteger)
+                x.Add(Name_ConfigAutoUpdate, ConfigAutoUpdate.BoolToInteger)
+                x.Add(Name_RulesConfigManualMode, RulesConfigManualMode.BoolToInteger)
+                x.Add(Name_ConfigLastDateUpdate, ConfigLastDateUpdate.ToStringDateDef)
+                If Count > 0 Then
+                    Rules.Sort()
+                    x.Add(New EContainer(RulesNode))
+                    x.Last.AddRange(Rules)
+                End If
+                x.Save(DynamicRulesXml, OFError)
+            End Using
+            If Count > 0 Then
+                Using t As New TextSaver(DynamicRulesFile)
+                    Rules.ForEach(Sub(r) If Not r.UrlRepo.IsEmptyString Then t.AppendLine(r.UrlRepo))
+                    t.Save(OFError)
+                End Using
+            End If
+        End Sub
+#End Region
+#Region "Update"
+        Private _UpdateInProgress As Boolean = False
+        Private _ForcedUpdate As Boolean = False
+        Friend Function Update(ByVal Force As Boolean, Optional ByVal LoadListOnly As Boolean = False) As Boolean
+            Dim skip As Boolean = _UpdateInProgress
+            If skip And _ForcedUpdate Then Force = False
+            _ForcedUpdate = Force
+            While _UpdateInProgress : Threading.Thread.Sleep(200) : End While
+            If Not skip Or Force Then UpdateImpl(Force Or RulesForceUpdateRequired, LoadListOnly)
+            Return Exists
+        End Function
+        Private Sub UpdateImpl(ByVal Force As Boolean, Optional ByVal LoadListOnly As Boolean = False)
+            Try
+                If Not _UpdateInProgress Then
+                    _UpdateInProgress = True
+
+                    LoadInitialValues()
+
+                    Dim r$
+                    Dim process As Boolean = False, updated As Boolean = False
+                    Dim forceSave As Boolean = RulesForceUpdateRequired Or Not DynamicRulesFile.Exists Or Not DynamicRulesXml.Exists
+                    Dim textLocal As List(Of String)
+                    Dim i%
+                    Dim rule As DynamicRulesValue
+                    Dim e As EContainer
+                    Dim errDate As Date = Now.AddYears(-1)
+                    Dim d As Date?
+                    '2024-06-12T12:44:06.000-05:00
+                    Dim dateProvider As New ADateTime("yyyy-MM-ddTHH:mm:ss.fff%K")
+
+                    RulesForceUpdateRequired = False
+
+                    If Not DynamicRulesFile.Exists Then process = True : ValidateRulesFile()
+
+                    'update rules list
+                    If Not LoadListOnly And (LastUpdateTimeFile.AddMinutes(UpdateInterval) < Now Or process Or Force) Then
+                        LastUpdateTimeFile = Now
+                        r = Responser.GetResponse("https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/SCrawler/API/OnlyFans/DynamicRules.txt")
+                        If Not r.IsEmptyString Then
+                            Dim textWeb As List(Of String) = GetTextLines(r)
+                            Dim fileText$
+                            If textWeb.ListExists Then
+                                Using t As New TextSaver(DynamicRulesFile)
+                                    If ProtectFile Then
+                                        fileText = DynamicRulesFile.GetText(OFError)
+                                        t.Append(fileText)
+                                        textLocal = GetTextLines(fileText)
+                                        If textLocal.ListExists Then _
+                                           textLocal.ForEach(Sub(tt) If Not tt.IsEmptyString AndAlso Not textWeb.Contains(tt, RulesLinesComparer) Then _
+                                                                     t.AppendLine(tt) : updated = True) : textLocal.Clear()
+                                    Else
+                                        t.Append(r)
+                                        updated = True
+                                    End If
+                                    t.Save(OFError)
+                                End Using
+                                textWeb.Clear()
+                            End If
+                        End If
+                    End If
+
+                    'update config and consts
+                    If Not LoadListOnly AndAlso ConfigAutoUpdate AndAlso ConfigLastDateUpdate.AddMinutes(UpdateInterval) < Now Then
+                        Dim __upConf As Boolean = False
+                        Dim __dConf As Date = ConfigLastDateUpdate
+                        Dim parseConfigFiles As Action(Of DynamicRulesValue, SFile, Boolean) =
+                            Sub(ByVal __rule As DynamicRulesValue, ByVal __fileSave As SFile, ByVal isConstFile As Boolean)
+                                r = Responser.GetResponse(__rule.UrlLatestCommit)
+                                If Not r.IsEmptyString Then
+                                    e = JsonDocument.Parse(r, OFError)
+                                    If e.ListExists Then
+                                        d = AConvert(Of Date)(e.Value("date"), dateProvider, Nothing)
+                                        Dim dConf As Date = If(d, errDate)
+                                        If dConf > __dConf Then
+                                            __dConf = dConf
+                                            __upConf = True
+                                            updated = True
+                                            r = Responser.GetResponse(__rule.UrlRaw)
+                                            If Not r.IsEmptyString Then
+                                                TextSaver.SaveTextToFile(r, __fileSave, True, False, OFError)
+                                                If isConstFile Then ParseConsts(r)
+                                            End If
+                                        End If
+                                        e.Dispose()
+                                    End If
+                                End If
+                            End Sub
+                        'Update consts
+                        If RulesUpdateConst Then parseConfigFiles(ConfigConstAddress, OFScraperConfigPatternFileConst, True)
+                        'Update config
+                        parseConfigFiles(ConfigAddress, OFScraperConfigPatternFile, False)
+                        If __upConf Then ConfigLastDateUpdate = Now
+                    End If
+
+                    'generate rules, update rules dates
+                    If LastUpdateTimeRules.AddMinutes(UpdateInterval) < Now Or updated Or Force Or LoadListOnly Then
+                        process = True
+                        If Mode = Modes.Personal And Not PersonalRule.IsEmptyString Then
+                            If Not LoadListOnly Then LastUpdateTimeRules = Now : updated = True
+                        Else
+                            If Not LoadListOnly Then LastUpdateTimeRules = Now : updated = True
+                            textLocal = GetTextLines(DynamicRulesFile.GetText(OFError))
+                            If textLocal.ListExists Then
+                                If Not LoadListOnly And Count > 0 Then
+                                    For i = 0 To Count - 1
+                                        rule = Rules(i)
+                                        rule.Exists = False
+                                        Rules(i) = rule
+                                    Next
+                                End If
+                                For Each url$ In textLocal
+                                    url = url.StringTrim
+                                    If Not url.IsEmptyString Then
+                                        i = IndexOf(url)
+                                        If i >= 0 Then
+                                            rule = Rules(i)
+                                        Else
+                                            rule = ParseURL(url)
+                                            If rule.Valid Then
+                                                i = Add(rule, False, False)
+                                            Else
+                                                rule = Nothing
+                                            End If
+                                        End If
+
+                                        If Not LoadListOnly Then
+                                            If i >= 0 And rule.Valid And Not rule.UrlLatestCommit.IsEmptyString Then
+                                                rule.Exists = True
+                                                r = Responser.GetResponse(rule.UrlLatestCommit)
+                                                If Not r.IsEmptyString Then
+                                                    e = JsonDocument.Parse(r, OFError)
+                                                    If e.ListExists Then
+                                                        d = AConvert(Of Date)(e.Value("date"), dateProvider, Nothing)
+                                                        rule.UpdatedAt = If(d, errDate)
+                                                        e.Dispose()
+                                                    Else
+                                                        rule.Broken = True
+                                                    End If
+                                                Else
+                                                    rule.Broken = True
+                                                End If
+                                                Rules(i) = rule
+                                            End If
+                                            If Rules.RemoveAll(Function(rr) Not rr.Exists) > 0 Then updated = True
+                                        End If
+                                    End If
+                                Next
+                            End If
+                        End If
+                    End If
+
+                    If Count > 0 Then Rules.Sort()
+
+                    'download and load the rule
+                    If (LoadListOnly And AuthFile.Exists) Or (Not LoadListOnly And ((updated And Count > 0) Or Not AuthFile.Exists)) Then
+                        _CurrentRule = Nothing
+                        _CurrentContainer.DisposeIfReady
+                        _CurrentContainer = Nothing
+                        Dim processRule As Func(Of DynamicRulesValue, Boolean, DialogResult) =
+                            Function(ByVal __rule As DynamicRulesValue, ByVal reparseAuth As Boolean) As DialogResult
+                                Dim fromAuthFile As Boolean = (LoadListOnly Or reparseAuth) AndAlso AuthFile.Exists
+                                If fromAuthFile Then
+                                    r = AuthFile.GetText(OFError)
+                                Else
+                                    r = GetWebString(__rule.UrlRaw,, OFError)
+                                End If
+                                Dim j As EContainer = JsonDocument.Parse(r, OFError)
+                                Dim checkResult As Boolean = False
+                                Dim isConfig As Boolean = False
+                                Dim textToSave As String = r
+                                If j.ListExists AndAlso Not GetFormat(j, True, checkResult,, isConfig).IsEmptyString AndAlso checkResult Then
+                                    If isConfig Then textToSave = RegexReplace(r, ConfigRulesExtract)
+                                    If textToSave.IsEmptyString Then
+                                        Return DialogResult.Retry
+                                    Else
+                                        _CurrentRule = __rule
+                                        _CurrentContainer = If(isConfig, j(ConfigNodes), j)
+                                        textToSave = ConvertAuthText()
+                                        _CurrentContainerRulesText = textToSave
+                                        If (Not fromAuthFile Or Not textToSave.StringTrim = r.StringTrim) And Not textToSave.IsEmptyString Then
+                                            TextSaver.SaveTextToFile(textToSave, AuthFile, True, False, OFError)
+                                            If Not reparseAuth Then processRule(__rule, True)
+                                        End If
+                                        Return DialogResult.OK
+                                    End If
+                                End If
+                                Return DialogResult.No
+                            End Function
+                        If Mode = Modes.Personal And Not PersonalRule.IsEmptyString Then
+                            processRule(New DynamicRulesValue With {.UrlRepo = PersonalRule, .UrlRaw = PersonalRule}, False)
+                        Else
+                            For Each rule In Rules
+                                If rule.Valid And Not rule.Broken Then
+                                    Select Case processRule(rule, False)
+                                        Case DialogResult.Retry : Continue For
+                                        Case DialogResult.OK : Exit For
+                                    End Select
+                                End If
+                            Next
+                        End If
+                    End If
+
+                    If updated Or forceSave Then Save()
+
+                    _UpdateInProgress = False
+                End If
+            Catch ex As Exception
+                ErrorsDescriber.Execute(OFError, ex, "[OnlyFans.DynamicRulesEnv.UpdateImpl]")
+                _UpdateInProgress = False
+            End Try
+        End Sub
+#End Region
+#Region "Add, IndexOf"
+        Friend Function Add(ByVal Rule As DynamicRulesValue, Optional ByVal AutoSort As Boolean = True, Optional ByVal AutoSave As Boolean = False) As Integer
+            If Rule.Valid Then
+                Dim i% = IndexOf(Rule)
+                If i = -1 Then
+                    Rules.Add(Rule)
+                    i = Count - 1
+                    If AutoSort Then Rules.Sort() : i = IndexOf(Rule)
+                    If AutoSave Then Save()
+                End If
+                Return i
+            Else
+                Return -1
+            End If
+        End Function
+        Friend Function RemoveAt(ByVal Index As Integer) As Boolean
+            If Index.ValueBetween(0, Count - 1) Then
+                Rules.RemoveAt(Index)
+                Return True
+            Else
+                Return False
+            End If
+        End Function
+        Friend Function IndexOf(ByVal URL As String) As Integer
+            If Count > 0 Then
+                URL = URL.StringToLower.Trim
+                Return Rules.FindIndex(Function(r) r.UrlRepo.StringToLower = URL Or r.UrlRaw.StringToLower = URL Or r.UrlLatestCommit.StringToLower = URL)
+            Else
+                Return -1
+            End If
+        End Function
+#End Region
+#Region "ICopier Support"
+        Friend Overloads Function Copy() As Object Implements ICopier.Copy
+            Return (New DynamicRulesEnv).Copy(Me)
+        End Function
+        Friend Overloads Function Copy(ByVal Source As Object) As Object Implements ICopier.Copy
+            Return Copy(Source, False)
+        End Function
+        Friend Overloads Function Copy(ByVal Source As Object, ByVal UpdateForceProperty As Boolean) As Object
+            If Not Source Is Nothing Then
+                With DirectCast(Source, DynamicRulesEnv)
+                    If Not RulesForceUpdateRequired And UpdateForceProperty Then _
+                       RulesForceUpdateRequired = Not Rules.ListEquals(.Rules) Or Not Mode = .Mode Or
+                                                  (.Mode = Modes.Personal And Not PersonalRule = .PersonalRule)
+                    ProtectFile = .ProtectFile
+                    Mode = .Mode
+                    UpdateInterval = .UpdateInterval
+                    PersonalRule = .PersonalRule
+                    If Not RulesForceUpdateRequired Then RulesForceUpdateRequired = .RulesForceUpdateRequired
+                    RulesUpdateConst = .RulesUpdateConst
+                    RulesReplaceConfig = .RulesReplaceConfig
+                    AddErrorsToLog = .AddErrorsToLog
+                    ConfigAutoUpdate = .ConfigAutoUpdate
+                    RulesConfigManualMode = .RulesConfigManualMode
+                    Rules.Clear()
+                    If .Count > 0 Then Rules.AddRange(.Rules)
+                End With
+                Return Me
+            Else
+                Return Nothing
+            End If
+        End Function
+#End Region
+#Region "IEnumerable Support"
+        Private Function GetEnumerator() As IEnumerator(Of DynamicRulesValue) Implements IEnumerable(Of DynamicRulesValue).GetEnumerator
+            Return New MyEnumerator(Of DynamicRulesValue)(Me)
+        End Function
+        Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
+            Return GetEnumerator()
+        End Function
+#End Region
+#Region "IDisposable Support"
+        Private disposedValue As Boolean = False
+        Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
+            If Not disposedValue Then
+                If disposing Then
+                    Rules.Clear()
+                    _CurrentContainer.DisposeIfReady
+                    Responser.DisposeIfReady
+                End If
+                _CurrentContainer = Nothing
+                disposedValue = True
+            End If
+        End Sub
+        Protected Overrides Sub Finalize()
+            Dispose(False)
+            MyBase.Finalize()
+        End Sub
+        Friend Overloads Sub Dispose() Implements IDisposable.Dispose
+            Dispose(True)
+            GC.SuppressFinalize(Me)
+        End Sub
+#End Region
+    End Class
+End Namespace

SCrawler/API/OnlyFans/OFResources.Designer.vb

@@ -64,6 +64,20 @@ Namespace My.Resources
             End Set
         End Property
         
+        '''<summary>
+        '''  Looks up a localized string similar to https://github.com/datawhores/onlyfans-dynamic-rules/blob/main/dynamicRules.json
+        '''https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/main/dynamicRules.json
+        '''https://github.com/riley-access-labs/onlyfans-dynamic-rules-1/blob/patch-1/dynamicRules.json
+        '''https://github.com/DATAHOARDERS/dynamic-rules/blob/main/onlyfans.json
+        '''https://github.com/DIGITALCRIMINAL/dynamic-rules/blob/main/onlyfans.json
+        '''https://github.com/deviint/onlyfans-dynamic-rules/blob/main/dynamicRules.json.
+        '''</summary>
+        Friend Shared ReadOnly Property DynamicRules() As String
+            Get
+                Return ResourceManager.GetString("DynamicRules", resourceCulture)
+            End Get
+        End Property
+        
         '''<summary>
         '''  Looks up a localized resource of type System.Byte[].
         '''</summary>
@@ -73,5 +87,15 @@ Namespace My.Resources
                 Return CType(obj,Byte())
             End Get
         End Property
+        
+        '''<summary>
+        '''  Looks up a localized string similar to dynamic-mode-default=generic
+        '''RULE_VALUE=DYNAMIC_GENERIC_URL.
+        '''</summary>
+        Friend Shared ReadOnly Property OFScraperConfigPatternConstants() As String
+            Get
+                Return ResourceManager.GetString("OFScraperConfigPatternConstants", resourceCulture)
+            End Get
+        End Property
     End Class
 End Namespace

SCrawler/API/OnlyFans/OFResources.resx

@@ -118,7 +118,13 @@
     <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
   </resheader>
   <assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
+  <data name="DynamicRules" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>DynamicRules.txt;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8</value>
+  </data>
   <data name="OFScraperConfigPattern" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>OFScraperConfigPattern.json;System.Byte[], mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
   </data>
+  <data name="OFScraperConfigPatternConstants" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>OFScraperConfigPatternConstants.txt;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8</value>
+  </data>
 </root>

SCrawler/API/OnlyFans/OFScraperConfigPattern.json

@@ -1,61 +1,63 @@
 {
-    "config": {
-        "main_profile": "main_profile",
-        "metadata": "{configpath}/{profile}/.data/{model_username}_{model_id}",
-        "discord": "",
-        "file_options": {
-            "save_location": "",
-            "dir_format": "",
-            "file_format": "{filename}.{ext}",
-            "textlength": 0,
-            "space-replacer": " ",
-            "date": "YYYY-MM-DD"
-        },
-        "download_options": {
-            "file_size_limit": 0,
-            "file_size_min": 0,
-            "filter": [
-                "Images",
-                "Audios",
-                "Videos"
-            ],
-            "auto_resume": false
-        },
-        "binary_options": {
-            "mp4decrypt": "",
-            "ffmpeg": ""
-        },
-        "cdm_options": {
-            "private-key": null,
-            "client-id": null,
-            "key-mode-default": "cdrm",
-            "keydb_api": ""
-        },
-        "performance_options": {
-            "download-sems": 6,
-            "maxfile-sem": 0,
-            "threads": 5
-        },
-        "advanced_options": {
-            "code-execution": false,
-            "dynamic-mode-default": "deviint",
-            "backend": "aio",
-            "downloadbars": false,
-            "cache-mode": "sqlite",
-            "appendlog": true,
-            "custom": null,
-            "sanitize_text": false,
-            "avatar": true
-        },
-        "responsetype": {
-            "timeline": "Posts",
-            "message": "Messages",
-            "archived": "Archived",
-            "paid": "Messages",
-            "stories": "Stories",
-            "highlights": "Stories",
-            "profile": "Profile",
-            "pinned": "Posts"
+    "main_profile": "main_profile",
+    "metadata": "{configpath}/{profile}/.data/{model_username}_{model_id}",
+    "discord": "",
+    "file_options": {
+        "save_location": "",
+        "dir_format": "",
+        "file_format": "{filename}.{ext}",
+        "textlength": 0,
+        "space_replacer": " ",
+        "date": "YYYY-MM-DD"
+    },
+    "download_options": {
+        "file_size_max": 0,
+        "file_size_min": 0,
+        "filter": [
+            "Images",
+            "Audios",
+            "Videos"
+        ],
+        "auto_resume": false
+    },
+    "binary_options": {
+        "mp4decrypt": "",
+        "ffmpeg": ""
+    },
+    "cdm_options": {
+        "private-key": null,
+        "client-id": null,
+        "key-mode-default": "cdrm",
+        "keydb_api": ""
+    },
+    "performance_options": {
+        "download-sems": 6,
+        "maxfile-sem": 0,
+        "threads": 5
+    },
+    "advanced_options": {
+        "code-execution": false,
+        "dynamic-mode-default": "generic",
+        "backend": "aio",
+        "downloadbars": false,
+        "cache-mode": "sqlite",
+        "appendlog": true,
+        "custom": null,
+        "sanitize_text": false,
+        "avatar": true,
+        "custom_values": {
+            "DYNAMIC_GENERIC_URL": "https://raw.githubusercontent.com/datawhores/onlyfans-dynamic-rules/main/dynamicRules.json",
+            "CDRM": "https://old.cdrm-project.com/wv"
         }
+    },
+    "responsetype": {
+        "timeline": "Posts",
+        "message": "Messages",
+        "archived": "Archived",
+        "paid": "Messages",
+        "stories": "Stories",
+        "highlights": "Stories",
+        "profile": "Profile",
+        "pinned": "Posts"
     }
 }

SCrawler/API/OnlyFans/OFScraperConfigPatternConstants.txt

@@ -0,0 +1,2 @@
+dynamic-mode-default=generic
+RULE_VALUE=DYNAMIC_GENERIC_URL

SCrawler/API/OnlyFans/OnlyFansAdvancedSettingsForm.Designer.vb

@@ -0,0 +1,361 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.OnlyFans
+    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
+    Partial Friend Class OnlyFansAdvancedSettingsForm : Inherits System.Windows.Forms.Form
+        <System.Diagnostics.DebuggerNonUserCode()>
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            Try
+                If disposing AndAlso components IsNot Nothing Then
+                    components.Dispose()
+                End If
+            Finally
+                MyBase.Dispose(disposing)
+            End Try
+        End Sub
+        Private components As System.ComponentModel.IContainer
+        <System.Diagnostics.DebuggerStepThrough()>
+        Private Sub InitializeComponent()
+            Me.components = New System.ComponentModel.Container()
+            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
+            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
+            Dim ActionButton7 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(OnlyFansAdvancedSettingsForm))
+            Dim ActionButton8 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton9 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim TP_RULES_LIST As System.Windows.Forms.TableLayoutPanel
+            Dim TP_RULES_LIST_LEFT As System.Windows.Forms.TableLayoutPanel
+            Dim TT_MAIN As System.Windows.Forms.ToolTip
+            Me.TXT_UP_INTERVAL = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            Me.TXT_PERSONAL_RULE = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            Me.CONTAINER_LIST = New System.Windows.Forms.ToolStripContainer()
+            Me.LIST_RULES = New System.Windows.Forms.ListBox()
+            Me.OPT_RULES_LIST = New System.Windows.Forms.RadioButton()
+            Me.CH_PROTECTED = New System.Windows.Forms.CheckBox()
+            Me.CH_FORCE_UPDATE = New System.Windows.Forms.CheckBox()
+            Me.CH_LOG_ERR = New System.Windows.Forms.CheckBox()
+            Me.CH_RULES_REPLACE_CONFIG = New System.Windows.Forms.CheckBox()
+            Me.CH_UPDATE_CONF = New System.Windows.Forms.CheckBox()
+            Me.CH_UPDATE_RULES_CONST = New System.Windows.Forms.CheckBox()
+            Me.CH_CONFIG_MANUAL_MODE = New System.Windows.Forms.CheckBox()
+            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
+            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
+            TP_RULES_LIST = New System.Windows.Forms.TableLayoutPanel()
+            TP_RULES_LIST_LEFT = New System.Windows.Forms.TableLayoutPanel()
+            TT_MAIN = New System.Windows.Forms.ToolTip(Me.components)
+            CONTAINER_MAIN.ContentPanel.SuspendLayout()
+            CONTAINER_MAIN.SuspendLayout()
+            TP_MAIN.SuspendLayout()
+            CType(Me.TXT_UP_INTERVAL, System.ComponentModel.ISupportInitialize).BeginInit()
+            CType(Me.TXT_PERSONAL_RULE, System.ComponentModel.ISupportInitialize).BeginInit()
+            TP_RULES_LIST.SuspendLayout()
+            Me.CONTAINER_LIST.ContentPanel.SuspendLayout()
+            Me.CONTAINER_LIST.SuspendLayout()
+            TP_RULES_LIST_LEFT.SuspendLayout()
+            Me.SuspendLayout()
+            '
+            'CONTAINER_MAIN
+            '
+            '
+            'CONTAINER_MAIN.ContentPanel
+            '
+            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(464, 341)
+            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            CONTAINER_MAIN.LeftToolStripPanelVisible = False
+            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
+            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
+            CONTAINER_MAIN.RightToolStripPanelVisible = False
+            CONTAINER_MAIN.Size = New System.Drawing.Size(464, 341)
+            CONTAINER_MAIN.TabIndex = 1
+            CONTAINER_MAIN.TopToolStripPanelVisible = False
+            '
+            'TP_MAIN
+            '
+            TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
+            TP_MAIN.ColumnCount = 1
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.Controls.Add(Me.TXT_UP_INTERVAL, 0, 5)
+            TP_MAIN.Controls.Add(Me.TXT_PERSONAL_RULE, 0, 6)
+            TP_MAIN.Controls.Add(TP_RULES_LIST, 0, 7)
+            TP_MAIN.Controls.Add(Me.CH_LOG_ERR, 0, 0)
+            TP_MAIN.Controls.Add(Me.CH_RULES_REPLACE_CONFIG, 0, 1)
+            TP_MAIN.Controls.Add(Me.CH_UPDATE_CONF, 0, 4)
+            TP_MAIN.Controls.Add(Me.CH_UPDATE_RULES_CONST, 0, 2)
+            TP_MAIN.Controls.Add(Me.CH_CONFIG_MANUAL_MODE, 0, 3)
+            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_MAIN.Location = New System.Drawing.Point(0, 0)
+            TP_MAIN.Name = "TP_MAIN"
+            TP_MAIN.RowCount = 8
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.Size = New System.Drawing.Size(464, 341)
+            TP_MAIN.TabIndex = 0
+            '
+            'TXT_UP_INTERVAL
+            '
+            ActionButton7.BackgroundImage = CType(resources.GetObject("ActionButton7.BackgroundImage"), System.Drawing.Image)
+            ActionButton7.Name = "Refresh"
+            ActionButton7.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton8.BackgroundImage = CType(resources.GetObject("ActionButton8.BackgroundImage"), System.Drawing.Image)
+            ActionButton8.Name = "Clear"
+            ActionButton8.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            Me.TXT_UP_INTERVAL.Buttons.Add(ActionButton7)
+            Me.TXT_UP_INTERVAL.Buttons.Add(ActionButton8)
+            Me.TXT_UP_INTERVAL.CaptionText = "Dynamic rules update"
+            Me.TXT_UP_INTERVAL.CaptionToolTipEnabled = True
+            Me.TXT_UP_INTERVAL.CaptionToolTipText = "'Dynamic rules' update interval (minutes). Default: 1440"
+            Me.TXT_UP_INTERVAL.CaptionWidth = 120.0R
+            Me.TXT_UP_INTERVAL.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_UP_INTERVAL.Location = New System.Drawing.Point(4, 134)
+            Me.TXT_UP_INTERVAL.Name = "TXT_UP_INTERVAL"
+            Me.TXT_UP_INTERVAL.Size = New System.Drawing.Size(456, 22)
+            Me.TXT_UP_INTERVAL.TabIndex = 5
+            '
+            'TXT_PERSONAL_RULE
+            '
+            Me.TXT_PERSONAL_RULE.AutoShowClearButton = True
+            ActionButton9.BackgroundImage = CType(resources.GetObject("ActionButton9.BackgroundImage"), System.Drawing.Image)
+            ActionButton9.Enabled = False
+            ActionButton9.Name = "Clear"
+            ActionButton9.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton9.Visible = False
+            Me.TXT_PERSONAL_RULE.Buttons.Add(ActionButton9)
+            Me.TXT_PERSONAL_RULE.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.RadioButton
+            Me.TXT_PERSONAL_RULE.CaptionText = "Dynamic rules URL"
+            Me.TXT_PERSONAL_RULE.CaptionToolTipEnabled = True
+            Me.TXT_PERSONAL_RULE.CaptionToolTipText = "Overwrite 'Dynamic rules' with this URL." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Change this value only if you know what" &
+    " you are doing."
+            Me.TXT_PERSONAL_RULE.CaptionWidth = 120.0R
+            Me.TXT_PERSONAL_RULE.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_PERSONAL_RULE.LeaveDefaultButtons = True
+            Me.TXT_PERSONAL_RULE.Location = New System.Drawing.Point(4, 163)
+            Me.TXT_PERSONAL_RULE.Name = "TXT_PERSONAL_RULE"
+            Me.TXT_PERSONAL_RULE.Size = New System.Drawing.Size(456, 22)
+            Me.TXT_PERSONAL_RULE.TabIndex = 6
+            '
+            'TP_RULES_LIST
+            '
+            TP_RULES_LIST.ColumnCount = 2
+            TP_RULES_LIST.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 120.0!))
+            TP_RULES_LIST.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_RULES_LIST.Controls.Add(Me.CONTAINER_LIST, 1, 0)
+            TP_RULES_LIST.Controls.Add(TP_RULES_LIST_LEFT, 0, 0)
+            TP_RULES_LIST.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_RULES_LIST.Location = New System.Drawing.Point(4, 192)
+            TP_RULES_LIST.Name = "TP_RULES_LIST"
+            TP_RULES_LIST.RowCount = 1
+            TP_RULES_LIST.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_RULES_LIST.Size = New System.Drawing.Size(456, 145)
+            TP_RULES_LIST.TabIndex = 7
+            '
+            'CONTAINER_LIST
+            '
+            Me.CONTAINER_LIST.BottomToolStripPanelVisible = False
+            '
+            'CONTAINER_LIST.ContentPanel
+            '
+            Me.CONTAINER_LIST.ContentPanel.Controls.Add(Me.LIST_RULES)
+            Me.CONTAINER_LIST.ContentPanel.Size = New System.Drawing.Size(330, 114)
+            Me.CONTAINER_LIST.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CONTAINER_LIST.LeftToolStripPanelVisible = False
+            Me.CONTAINER_LIST.Location = New System.Drawing.Point(123, 3)
+            Me.CONTAINER_LIST.Name = "CONTAINER_LIST"
+            Me.CONTAINER_LIST.RightToolStripPanelVisible = False
+            Me.CONTAINER_LIST.Size = New System.Drawing.Size(330, 139)
+            Me.CONTAINER_LIST.TabIndex = 1
+            '
+            'LIST_RULES
+            '
+            Me.LIST_RULES.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.LIST_RULES.FormattingEnabled = True
+            Me.LIST_RULES.Location = New System.Drawing.Point(0, 0)
+            Me.LIST_RULES.Name = "LIST_RULES"
+            Me.LIST_RULES.Size = New System.Drawing.Size(330, 114)
+            Me.LIST_RULES.TabIndex = 0
+            '
+            'TP_RULES_LIST_LEFT
+            '
+            TP_RULES_LIST_LEFT.ColumnCount = 1
+            TP_RULES_LIST_LEFT.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_RULES_LIST_LEFT.Controls.Add(Me.OPT_RULES_LIST, 0, 0)
+            TP_RULES_LIST_LEFT.Controls.Add(Me.CH_PROTECTED, 0, 1)
+            TP_RULES_LIST_LEFT.Controls.Add(Me.CH_FORCE_UPDATE, 0, 2)
+            TP_RULES_LIST_LEFT.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_RULES_LIST_LEFT.Location = New System.Drawing.Point(0, 0)
+            TP_RULES_LIST_LEFT.Margin = New System.Windows.Forms.Padding(0)
+            TP_RULES_LIST_LEFT.Name = "TP_RULES_LIST_LEFT"
+            TP_RULES_LIST_LEFT.RowCount = 4
+            TP_RULES_LIST_LEFT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_RULES_LIST_LEFT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_RULES_LIST_LEFT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_RULES_LIST_LEFT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_RULES_LIST_LEFT.Size = New System.Drawing.Size(120, 145)
+            TP_RULES_LIST_LEFT.TabIndex = 0
+            '
+            'OPT_RULES_LIST
+            '
+            Me.OPT_RULES_LIST.AutoSize = True
+            Me.OPT_RULES_LIST.CheckAlign = System.Drawing.ContentAlignment.MiddleRight
+            Me.OPT_RULES_LIST.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.OPT_RULES_LIST.Location = New System.Drawing.Point(3, 3)
+            Me.OPT_RULES_LIST.Margin = New System.Windows.Forms.Padding(3, 3, 0, 3)
+            Me.OPT_RULES_LIST.Name = "OPT_RULES_LIST"
+            Me.OPT_RULES_LIST.Size = New System.Drawing.Size(117, 19)
+            Me.OPT_RULES_LIST.TabIndex = 0
+            Me.OPT_RULES_LIST.TabStop = True
+            Me.OPT_RULES_LIST.Text = "Dynamic rules list"
+            Me.OPT_RULES_LIST.TextAlign = System.Drawing.ContentAlignment.MiddleRight
+            TT_MAIN.SetToolTip(Me.OPT_RULES_LIST, "List of dynamic rules sources." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "If selected, the most recently updated source wil" &
+        "l be selected.")
+            Me.OPT_RULES_LIST.UseVisualStyleBackColor = True
+            '
+            'CH_PROTECTED
+            '
+            Me.CH_PROTECTED.AutoSize = True
+            Me.CH_PROTECTED.CheckAlign = System.Drawing.ContentAlignment.MiddleRight
+            Me.CH_PROTECTED.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_PROTECTED.Location = New System.Drawing.Point(3, 28)
+            Me.CH_PROTECTED.Margin = New System.Windows.Forms.Padding(3, 3, 0, 3)
+            Me.CH_PROTECTED.Name = "CH_PROTECTED"
+            Me.CH_PROTECTED.Size = New System.Drawing.Size(117, 19)
+            Me.CH_PROTECTED.TabIndex = 1
+            Me.CH_PROTECTED.Text = "Protected list"
+            Me.CH_PROTECTED.TextAlign = System.Drawing.ContentAlignment.MiddleRight
+            TT_MAIN.SetToolTip(Me.CH_PROTECTED, "If checked, the new source will be added, but the rules list will not be overwrit" &
+        "ten by the updated one.")
+            Me.CH_PROTECTED.UseVisualStyleBackColor = True
+            '
+            'CH_FORCE_UPDATE
+            '
+            Me.CH_FORCE_UPDATE.AutoSize = True
+            Me.CH_FORCE_UPDATE.CheckAlign = System.Drawing.ContentAlignment.MiddleRight
+            Me.CH_FORCE_UPDATE.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_FORCE_UPDATE.Location = New System.Drawing.Point(3, 53)
+            Me.CH_FORCE_UPDATE.Margin = New System.Windows.Forms.Padding(3, 3, 0, 3)
+            Me.CH_FORCE_UPDATE.Name = "CH_FORCE_UPDATE"
+            Me.CH_FORCE_UPDATE.Size = New System.Drawing.Size(117, 19)
+            Me.CH_FORCE_UPDATE.TabIndex = 2
+            Me.CH_FORCE_UPDATE.Text = "Force update"
+            Me.CH_FORCE_UPDATE.TextAlign = System.Drawing.ContentAlignment.MiddleRight
+            TT_MAIN.SetToolTip(Me.CH_FORCE_UPDATE, "Check this if you want to force the rules to update.")
+            Me.CH_FORCE_UPDATE.UseVisualStyleBackColor = True
+            '
+            'CH_LOG_ERR
+            '
+            Me.CH_LOG_ERR.AutoSize = True
+            Me.CH_LOG_ERR.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_LOG_ERR.Location = New System.Drawing.Point(4, 4)
+            Me.CH_LOG_ERR.Name = "CH_LOG_ERR"
+            Me.CH_LOG_ERR.Size = New System.Drawing.Size(456, 19)
+            Me.CH_LOG_ERR.TabIndex = 0
+            Me.CH_LOG_ERR.Text = "Add dynamic rules errors to the log"
+            TT_MAIN.SetToolTip(Me.CH_LOG_ERR, "OnlyFans errors will be added to a separate log." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "A checked checkbox means that e" &
+        "rror notification will be added to the main log.")
+            Me.CH_LOG_ERR.UseVisualStyleBackColor = True
+            '
+            'CH_RULES_REPLACE_CONFIG
+            '
+            Me.CH_RULES_REPLACE_CONFIG.AutoSize = True
+            Me.CH_RULES_REPLACE_CONFIG.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_RULES_REPLACE_CONFIG.Location = New System.Drawing.Point(4, 30)
+            Me.CH_RULES_REPLACE_CONFIG.Name = "CH_RULES_REPLACE_CONFIG"
+            Me.CH_RULES_REPLACE_CONFIG.Size = New System.Drawing.Size(456, 19)
+            Me.CH_RULES_REPLACE_CONFIG.TabIndex = 1
+            Me.CH_RULES_REPLACE_CONFIG.Text = "Replace rules in OF-Scraper configuration file"
+            TT_MAIN.SetToolTip(Me.CH_RULES_REPLACE_CONFIG, "If checked, the dynamic rules (in the config) will be replaced with actual values" &
+        ".")
+            Me.CH_RULES_REPLACE_CONFIG.UseVisualStyleBackColor = True
+            '
+            'CH_UPDATE_CONF
+            '
+            Me.CH_UPDATE_CONF.AutoSize = True
+            Me.CH_UPDATE_CONF.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_UPDATE_CONF.Location = New System.Drawing.Point(4, 108)
+            Me.CH_UPDATE_CONF.Name = "CH_UPDATE_CONF"
+            Me.CH_UPDATE_CONF.Size = New System.Drawing.Size(456, 19)
+            Me.CH_UPDATE_CONF.TabIndex = 4
+            Me.CH_UPDATE_CONF.Text = "Update configuration file during update"
+            TT_MAIN.SetToolTip(Me.CH_UPDATE_CONF, "Update the configuration pattern from the site during update.")
+            Me.CH_UPDATE_CONF.UseVisualStyleBackColor = True
+            '
+            'CH_UPDATE_RULES_CONST
+            '
+            Me.CH_UPDATE_RULES_CONST.AutoSize = True
+            Me.CH_UPDATE_RULES_CONST.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_UPDATE_RULES_CONST.Location = New System.Drawing.Point(4, 56)
+            Me.CH_UPDATE_RULES_CONST.Name = "CH_UPDATE_RULES_CONST"
+            Me.CH_UPDATE_RULES_CONST.Size = New System.Drawing.Size(456, 19)
+            Me.CH_UPDATE_RULES_CONST.TabIndex = 2
+            Me.CH_UPDATE_RULES_CONST.Text = "Update rules constants file during update"
+            TT_MAIN.SetToolTip(Me.CH_UPDATE_RULES_CONST, "Update rules constants from the site during update")
+            Me.CH_UPDATE_RULES_CONST.UseVisualStyleBackColor = True
+            '
+            'CH_CONFIG_MANUAL_MODE
+            '
+            Me.CH_CONFIG_MANUAL_MODE.AutoSize = True
+            Me.CH_CONFIG_MANUAL_MODE.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_CONFIG_MANUAL_MODE.Location = New System.Drawing.Point(4, 82)
+            Me.CH_CONFIG_MANUAL_MODE.Name = "CH_CONFIG_MANUAL_MODE"
+            Me.CH_CONFIG_MANUAL_MODE.Size = New System.Drawing.Size(456, 19)
+            Me.CH_CONFIG_MANUAL_MODE.TabIndex = 3
+            Me.CH_CONFIG_MANUAL_MODE.Text = "Dynamic rules 'Manual' mode"
+            TT_MAIN.SetToolTip(Me.CH_CONFIG_MANUAL_MODE, "The rules will be added to the config as is, without using a link.")
+            Me.CH_CONFIG_MANUAL_MODE.UseVisualStyleBackColor = True
+            '
+            'OnlyFansAdvancedSettingsForm
+            '
+            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
+            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
+            Me.ClientSize = New System.Drawing.Size(464, 341)
+            Me.Controls.Add(CONTAINER_MAIN)
+            Me.Icon = Global.SCrawler.My.Resources.SiteResources.OnlyFansIcon_32
+            Me.KeyPreview = True
+            Me.MinimizeBox = False
+            Me.MinimumSize = New System.Drawing.Size(480, 380)
+            Me.Name = "OnlyFansAdvancedSettingsForm"
+            Me.ShowInTaskbar = False
+            Me.Text = "Settings"
+            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
+            CONTAINER_MAIN.ResumeLayout(False)
+            CONTAINER_MAIN.PerformLayout()
+            TP_MAIN.ResumeLayout(False)
+            TP_MAIN.PerformLayout()
+            CType(Me.TXT_UP_INTERVAL, System.ComponentModel.ISupportInitialize).EndInit()
+            CType(Me.TXT_PERSONAL_RULE, System.ComponentModel.ISupportInitialize).EndInit()
+            TP_RULES_LIST.ResumeLayout(False)
+            Me.CONTAINER_LIST.ContentPanel.ResumeLayout(False)
+            Me.CONTAINER_LIST.ResumeLayout(False)
+            Me.CONTAINER_LIST.PerformLayout()
+            TP_RULES_LIST_LEFT.ResumeLayout(False)
+            TP_RULES_LIST_LEFT.PerformLayout()
+            Me.ResumeLayout(False)
+
+        End Sub
+
+        Private WithEvents TXT_UP_INTERVAL As PersonalUtilities.Forms.Controls.TextBoxExtended
+        Private WithEvents TXT_PERSONAL_RULE As PersonalUtilities.Forms.Controls.TextBoxExtended
+        Private WithEvents CONTAINER_LIST As ToolStripContainer
+        Private WithEvents LIST_RULES As ListBox
+        Private WithEvents OPT_RULES_LIST As RadioButton
+        Private WithEvents CH_PROTECTED As CheckBox
+        Private WithEvents CH_FORCE_UPDATE As CheckBox
+        Private WithEvents CH_LOG_ERR As CheckBox
+        Private WithEvents CH_RULES_REPLACE_CONFIG As CheckBox
+        Private WithEvents CH_UPDATE_CONF As CheckBox
+        Private WithEvents CH_UPDATE_RULES_CONST As CheckBox
+        Private WithEvents CH_CONFIG_MANUAL_MODE As CheckBox
+    End Class
+End Namespace

SCrawler/API/OnlyFans/OnlyFansAdvancedSettingsForm.resx

@@ -0,0 +1,186 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" use="required" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
+  <data name="ActionButton7.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
+        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
+        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
+        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
+        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
+        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
+        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
+        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
+        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
+        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
+        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
+        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
+</value>
+  </data>
+  <data name="ActionButton8.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton9.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <metadata name="TP_RULES_LIST.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TP_RULES_LIST_LEFT.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TT_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TT_MAIN.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
+    <value>17, 17</value>
+  </metadata>
+  <metadata name="TT_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TT_MAIN.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
+    <value>17, 17</value>
+  </metadata>
+  <metadata name="TP_RULES_LIST_LEFT.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TT_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="TT_MAIN.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
+    <value>17, 17</value>
+  </metadata>
+</root>

SCrawler/API/OnlyFans/OnlyFansAdvancedSettingsForm.vb

@@ -0,0 +1,165 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Forms.Controls.Base
+Imports PersonalUtilities.Forms.Toolbars
+Namespace API.OnlyFans
+    Friend Class OnlyFansAdvancedSettingsForm
+#Region "Declarations"
+        Private WithEvents MyDefs As DefaultFormOptions
+        Friend Property CurrentRulesEnv As DynamicRulesEnv
+        Private ReadOnly Property CurrentRulesEnv_LIST As DynamicRulesEnv
+#End Region
+#Region "Initializer"
+        Friend Sub New(ByVal rules As DynamicRulesEnv)
+            InitializeComponent()
+            MyDefs = New DefaultFormOptions(Me, Settings.Design)
+            CurrentRulesEnv = rules
+            CurrentRulesEnv_LIST = New DynamicRulesEnv
+            CurrentRulesEnv_LIST.Copy(rules, False)
+        End Sub
+#End Region
+#Region "Form handlers"
+        Private Sub OnlyFansAdvancedSettingsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
+            With MyDefs
+                .MyViewInitialize()
+                .AddOkCancelToolbar()
+
+                .MyEditToolbar = New EditToolbar(Me,, CONTAINER_LIST.TopToolStripPanel) With {.Buttons = Nothing}
+                .MyEditToolbar.AddThisToolbar()
+
+                With CurrentRulesEnv
+                    Select Case .Mode
+                        Case DynamicRulesEnv.Modes.List : OPT_RULES_LIST.Checked = True
+                        Case DynamicRulesEnv.Modes.Personal : TXT_PERSONAL_RULE.Checked = True
+                    End Select
+                    CH_LOG_ERR.Checked = .AddErrorsToLog
+                    CH_RULES_REPLACE_CONFIG.Checked = .RulesReplaceConfig
+                    CH_UPDATE_RULES_CONST.Checked = .RulesUpdateConst
+                    CH_CONFIG_MANUAL_MODE.Checked = .RulesConfigManualMode
+                    CH_UPDATE_CONF.Checked = .ConfigAutoUpdate
+                    TXT_UP_INTERVAL.Text = .UpdateInterval
+                    If Not .PersonalRule.IsEmptyString Then TXT_PERSONAL_RULE.Text = .PersonalRule
+                    Refill()
+                    CH_PROTECTED.Checked = .ProtectFile
+                    CH_FORCE_UPDATE.Checked = .RulesForceUpdateRequired
+                End With
+
+                .MyFieldsCheckerE = New FieldsChecker
+                .MyFieldsCheckerE.AddControl(Of Integer)(TXT_UP_INTERVAL, TXT_UP_INTERVAL.CaptionText,,
+                                                         New FieldsCheckerProviderSimple(Function(v) IIf(AConvert(Of Integer)(v, 0, EDP.ReturnValue) > 0, v, Nothing),
+                                                                                                     "The value of [{0}] field must be greater than 0"))
+                .MyFieldsCheckerE.EndLoaderOperations()
+
+                .EndLoaderOperations()
+            End With
+        End Sub
+        Private Sub OnlyFansAdvancedSettingsForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed
+            CurrentRulesEnv_LIST.Dispose()
+        End Sub
+#End Region
+#Region "Refill"
+        Private Sub Refill()
+            With CurrentRulesEnv_LIST
+                Dim ls% = _LatestSelected
+                LIST_RULES.Items.Clear()
+                If .Count > 0 Then LIST_RULES.Items.AddRange(.Select(Function(r) r.UrlRepo).Cast(Of Object).ToArray)
+                Dim lim% = LIST_RULES.Items.Count - 1
+                If (ls - 1).ValueBetween(0, lim) Then
+                    ls -= 1
+                ElseIf (ls + 1).ValueBetween(0, lim) Then
+                    ls += 1
+                End If
+                If ls.ValueBetween(0, lim) Then LIST_RULES.SelectedIndex = ls Else ls = -1
+                _LatestSelected = ls
+            End With
+        End Sub
+#End Region
+#Region "OK, Cancel"
+        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
+            If MyDefs.MyFieldsChecker.AllParamsOK Then
+                With CurrentRulesEnv
+                    .Copy(CurrentRulesEnv_LIST, True)
+                    .ProtectFile = CH_PROTECTED.Checked
+                    .UpdateInterval = AConvert(Of Integer)(TXT_UP_INTERVAL.Text, DynamicRulesEnv.UpdateIntervalDefault)
+                    .Mode = If(TXT_PERSONAL_RULE.Checked, DynamicRulesEnv.Modes.Personal, DynamicRulesEnv.Modes.List)
+                    .PersonalRule = TXT_PERSONAL_RULE.Text
+                    .RulesReplaceConfig = CH_RULES_REPLACE_CONFIG.Checked
+                    .RulesUpdateConst = CH_UPDATE_RULES_CONST.Checked
+                    .RulesConfigManualMode = CH_CONFIG_MANUAL_MODE.Checked
+                    .ConfigAutoUpdate = CH_UPDATE_CONF.Checked
+                    .AddErrorsToLog = CH_LOG_ERR.Checked
+                    If CH_FORCE_UPDATE.Checked Then .RulesForceUpdateRequired = True
+                    .NeedToSave = True
+                End With
+                MyDefs.CloseForm()
+            End If
+        End Sub
+#End Region
+#Region "Add, Delete"
+        Private Sub MyDefs_ButtonAddClick(ByVal Sender As Object, ByVal e As EditToolbarEventArgs) Handles MyDefs.ButtonAddClick
+            Const msgTitle$ = "Add a rule"
+            Dim i%
+            Dim rule As DynamicRulesValue
+            Dim r$ = InputBoxE("Enter a valid rules URL:", msgTitle)
+            If Not r.IsEmptyString Then
+                rule = Rules.ParseURL(r)
+                If rule.Valid Then
+                    i = CurrentRulesEnv_LIST.IndexOf(r)
+                    If i >= 0 Then
+                        MsgBoxE({$"The rule you entered already exists:{vbCr}{rule.UrlRepo}", msgTitle}, vbCritical)
+                    Else
+                        CurrentRulesEnv_LIST.Add(rule, False, False)
+                        Refill()
+                    End If
+                Else
+                    MsgBoxE({$"The rule you entered has an incompatible format:{vbCr}{r}", msgTitle}, vbCritical)
+                End If
+            End If
+        End Sub
+        Private Sub MyDefs_ButtonDeleteClickE(ByVal Sender As Object, ByVal e As EditToolbarEventArgs) Handles MyDefs.ButtonDeleteClickE
+            If _LatestSelected.ValueBetween(0, LIST_RULES.Items.Count - 1) Then
+                Dim r$ = LIST_RULES.Items(_LatestSelected)
+                If MsgBoxE({$"Are you sure you want to delete the following rule?{vbCr}{r}", "Delete a rule"}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then
+                    If CurrentRulesEnv_LIST.RemoveAt(_LatestSelected) Then
+                        LIST_RULES.Items.RemoveAt(_LatestSelected)
+                        Refill()
+                    Else
+                        MsgBoxE({$"The following rule cannot be deleted:{vbCr}{r}", "Delete a rule"}, vbCritical)
+                    End If
+                End If
+            End If
+        End Sub
+#End Region
+#Region "Options"
+        Private Sub TXT_UP_INTERVAL_ActionOnButtonClick(ByVal Sender As Object, ByVal e As ActionButtonEventArgs) Handles TXT_UP_INTERVAL.ActionOnButtonClick
+            If e.DefaultButton = ActionButton.DefaultButtons.Refresh Then TXT_UP_INTERVAL.Text = DynamicRulesEnv.UpdateIntervalDefault
+        End Sub
+        Private Sub TXT_PERSONAL_RULE_ActionOnCheckedChange(ByVal Sender As Object, ByVal e As EventArgs, ByVal Checked As Boolean) Handles TXT_PERSONAL_RULE.ActionOnCheckedChange
+            Mode_CheckedChanged()
+        End Sub
+        Private Sub OPT_RULES_LIST_CheckedChanged(sender As Object, e As EventArgs)
+            Mode_CheckedChanged()
+        End Sub
+        Private Sub Mode_CheckedChanged()
+            Dim e As Boolean = TXT_PERSONAL_RULE.Checked
+            TXT_PERSONAL_RULE.Enabled(False) = e
+            CONTAINER_LIST.Enabled = Not e
+            CH_PROTECTED.Enabled = Not e
+            CH_FORCE_UPDATE.Enabled = Not e
+        End Sub
+#End Region
+#Region "List handlers"
+        Private _LatestSelected As Integer = -1
+        Private Sub LIST_RULES_SelectedIndexChanged(sender As Object, e As EventArgs) Handles LIST_RULES.SelectedIndexChanged
+            _LatestSelected = LIST_RULES.SelectedIndex
+        End Sub
+#End Region
+    End Class
+End Namespace

SCrawler/API/OnlyFans/SiteSettings.vb

@@ -9,22 +9,25 @@
 Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
-Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Cookies
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.OnlyFans
-    <Manifest("AndyProgram_OnlyFans"), SavedPosts, SpecialForm(False), SeparatedTasks(1)>
+    <Manifest("AndyProgram_OnlyFans"), SavedPosts, SpecialForm(False), SpecialForm(True), SeparatedTasks(1)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
+#Region "Categories"
+        Private Const CAT_OFS As String = "OF-Scraper support"
+#End Region
 #Region "Options"
-        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download user timeline"), PXML, PClonable>
+        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download user timeline", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property DownloadTimeline As PropertyValue
-        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download profile stories if they exists"), PXML, PClonable>
+        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download profile stories if they exists", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property DownloadStories As PropertyValue
-        <PropertyOption(ControlText:="Download highlights", ControlToolTip:="Download profile highlights if they exists"), PXML, PClonable>
+        <PropertyOption(ControlText:="Download highlights", ControlToolTip:="Download profile highlights if they exists", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property DownloadHighlights As PropertyValue
-        <PropertyOption(ControlText:="Download chat", ControlToolTip:="Download unlocked chat media"), PXML, PClonable>
+        <PropertyOption(ControlText:="Download chat", ControlToolTip:="Download unlocked chat media", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property DownloadChatMedia As PropertyValue
 #End Region
 #Region "Headers"
@@ -32,16 +35,16 @@ Namespace API.OnlyFans
         Private Const HeaderUserID As String = "User-Id"
         Friend Const HeaderXBC As String = "X-Bc"
         Friend Const HeaderAppToken As String = "App-Token"
-        <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False), PClonable(Clone:=False)>
+        <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_USER_ID As PropertyValue
-        <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False), PClonable(Clone:=False)>
+        <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Private ReadOnly Property HH_X_BC As PropertyValue
-        <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False), PClonable(Clone:=False)>
+        <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Private ReadOnly Property HH_APP_TOKEN As PropertyValue
         <PropertyOption(ControlText:=HeaderBrowser, ControlToolTip:="Can be null", AllowNull:=True,
-                        InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua), PClonable, PXML(OnlyForChecked:=True)>
+                        InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua, IsAuth:=True), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_BROWSER As PropertyValue
-        <PropertyOption(AllowNull:=False, InheritanceName:=SettingsCLS.HEADER_DEF_UserAgent), PClonable, PXML(OnlyForChecked:=True)>
+        <PropertyOption(AllowNull:=False, InheritanceName:=SettingsCLS.HEADER_DEF_UserAgent, IsAuth:=True), PClonable, PXML(OnlyForChecked:=True)>
         Friend ReadOnly Property UserAgent As PropertyValue
         Private Sub UpdateHeader(ByVal PropertyName As String, ByVal Value As String)
             Dim hName$ = String.Empty
@@ -59,33 +62,20 @@ Namespace API.OnlyFans
                 Responser.UserAgent = Value
             End If
         End Sub
-#End Region
-#Region "Rules"
-        <PXML("LastDateUpdated")> Private ReadOnly Property LastDateUpdated_XML As PropertyValue
-        Friend Property LastDateUpdated As Date
-            Get
-                Return LastDateUpdated_XML.Value
-            End Get
-            Set(ByVal d As Date)
-                LastDateUpdated_XML.Value = d
-            End Set
-        End Property
-        <PropertyOption(ControlText:="Use old authorization rules",
-                        ControlToolTip:="Use old dynamic rules (from 'DATAHOARDERS') or new ones (from 'DIGITALCRIMINALS')." & vbCr &
-                                        "Change this value only if you know what you are doing."), PXML, PClonable>
-        Friend ReadOnly Property UseOldAuthRules As PropertyValue
-        <PropertyOption(ControlText:="Dynamic rules update", ControlToolTip:="'Dynamic rules' update interval (minutes). Default: 1440", LeftOffset:=110), PXML, PClonable>
-        Friend ReadOnly Property DynamicRulesUpdateInterval As PropertyValue
-        <Provider(NameOf(DynamicRulesUpdateInterval), FieldsChecker:=True)>
-        Private ReadOnly Property DynamicRulesUpdateIntervalProvider As IFormatProvider
-        <PropertyOption(ControlText:="Dynamic rules",
-                        ControlToolTip:="Overwrite 'Dynamic rules' with this URL" & vbCr &
-                                        "Change this value only if you know what you are doing."), PXML, PClonable>
-        Friend ReadOnly Property DynamicRules As PropertyValue
+        <CookieValueExtractor(NameOf(HH_USER_ID)), CookieValueExtractor(NameOf(HH_X_BC))>
+        Private Function GetValueFromCookies(ByVal PropName As String, ByVal c As CookieKeeper) As String
+            If c.ListExists Then
+                Select Case PropName
+                    Case NameOf(HH_USER_ID) : Return c.GetCookieValue("auth_id")
+                    Case NameOf(HH_X_BC) : Return c.GetCookieValue("fp")
+                End Select
+            End If
+            Return String.Empty
+        End Function
 #End Region
 #Region "OFScraper"
         <PClonable, PXML("OFScraperPath")> Private ReadOnly Property OFScraperPath_XML As PropertyValue
-        <PropertyOption(ControlText:="OF-Scraper path", ControlToolTip:="The path to the 'ofscraper.exe'")>
+        <PropertyOption(ControlText:="OF-Scraper path", ControlToolTip:="The path to the 'ofscraper.exe'", Category:=CAT_OFS)>
         Friend ReadOnly Property OFScraperPath As PropertyValue
             Get
                 If Not DefaultInstance Is Nothing Then
@@ -96,7 +86,7 @@ Namespace API.OnlyFans
             End Get
         End Property
         <PClonable, PXML("OFScraperMP4decrypt")> Private ReadOnly Property OFScraperMP4decrypt_XML As PropertyValue
-        <PropertyOption(ControlText:="mp4decrypt path", ControlToolTip:="The path to the 'mp4decrypt.exe'")>
+        <PropertyOption(ControlText:="mp4decrypt path", ControlToolTip:="The path to the 'mp4decrypt.exe'", Category:=CAT_OFS)>
         Friend ReadOnly Property OFScraperMP4decrypt As PropertyValue
             Get
                 If Not DefaultInstance Is Nothing Then
@@ -108,7 +98,7 @@ Namespace API.OnlyFans
         End Property
         Friend Const KeyModeDefault_Default As String = "cdrm"
         <PClonable, PXML("KeyModeDefault")> Private ReadOnly Property KeyModeDefault_XML As PropertyValue
-        <PropertyOption(ControlText:="key-mode-default")>
+        <PropertyOption(ControlText:="key-mode-default", Category:=CAT_OFS)>
         Friend ReadOnly Property KeyModeDefault As PropertyValue
             Get
                 If Not DefaultInstance Is Nothing Then
@@ -118,12 +108,30 @@ Namespace API.OnlyFans
                 End If
             End Get
         End Property
+        <PClonable, PXML("keydb_api")> Private ReadOnly Property Keydb_Api_XML As PropertyValue
+        <PropertyOption(ControlText:="keydb_api", Category:=CAT_OFS)>
+        Friend ReadOnly Property Keydb_Api As PropertyValue
+            Get
+                If Not DefaultInstance Is Nothing Then
+                    Return DirectCast(DefaultInstance, SiteSettings).Keydb_Api_XML
+                Else
+                    Return Keydb_Api_XML
+                End If
+            End Get
+        End Property
 #End Region
 #End Region
 #Region "Initializer"
         Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
             MyBase.New("OnlyFans", ".onlyfans.com", AccName, Temp, My.Resources.SiteResources.OnlyFansIcon_32, My.Resources.SiteResources.OnlyFansPic_32)
 
+            If Rules Is Nothing Then
+                Rules = New DynamicRulesEnv
+                Rules.Update(False, True)
+            End If
+
+            _AllowUserAgentUpdate = False
+
             With Responser
                 .Accept = "application/json, text/plain, */*"
                 .AutomaticDecompression = Net.DecompressionMethods.GZip
@@ -154,13 +162,6 @@ Namespace API.OnlyFans
             DownloadHighlights = New PropertyValue(True)
             DownloadChatMedia = New PropertyValue(True)
 
-            LastDateUpdated_XML = New PropertyValue(Now.AddYears(-1), GetType(Date))
-            'URGENT: OF [UseOldAuthRules = True]
-            UseOldAuthRules = New PropertyValue(True)
-            DynamicRulesUpdateInterval = New PropertyValue(60 * 24)
-            DynamicRulesUpdateIntervalProvider = New FieldsCheckerProviderSimple(Function(v) IIf(AConvert(Of Integer)(v, 0) > 0, v, Nothing),
-                                                                                 "The value of [{0}] field must be greater than 0")
-            DynamicRules = New PropertyValue(String.Empty, GetType(String))
             OFScraperPath_XML = New PropertyValue(String.Empty, GetType(String))
             If ACheck(OFScraperPath_XML.Value) Then
                 Dim f As SFile = OFScraperPath_XML.Value
@@ -175,6 +176,7 @@ Namespace API.OnlyFans
             End If
             OFScraperMP4decrypt_XML = New PropertyValue(String.Empty, GetType(String))
             KeyModeDefault_XML = New PropertyValue(KeyModeDefault_Default)
+            Keydb_Api_XML = New PropertyValue(String.Empty, GetType(String))
 
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "onlyfans.com/"), 1, EDP.ReturnValue)
             UrlPatternUser = "https://onlyfans.com/{0}"
@@ -187,10 +189,28 @@ Namespace API.OnlyFans
         End Function
 #End Region
 #Region "Update"
+        Friend Overrides Sub BeginUpdate()
+            _TempSettingsEnv.DisposeIfReady
+            _TempSettingsEnv = Nothing
+            MyBase.BeginUpdate()
+        End Sub
         Friend Overrides Sub Update()
-            If _SiteEditorFormOpened Then Responser.Cookies.Changed = False
+            If _SiteEditorFormOpened Then
+                Responser.Cookies.Changed = False
+                If If(_TempSettingsEnv?.NeedToSave, False) Then Rules.Copy(_TempSettingsEnv, True) : Rules.Save()
+            End If
             MyBase.Update()
         End Sub
+        Friend Overrides Sub EndUpdate()
+            _TempSettingsEnv.DisposeIfReady
+            _TempSettingsEnv = Nothing
+            MyBase.EndUpdate()
+        End Sub
+        Private _TempSettingsEnv As DynamicRulesEnv = Nothing
+        Friend Overrides Sub OpenSettingsForm()
+            If _TempSettingsEnv Is Nothing Then _TempSettingsEnv = New DynamicRulesEnv : _TempSettingsEnv.Copy(Rules, False)
+            Using f As New OnlyFansAdvancedSettingsForm(_TempSettingsEnv) : f.ShowDialog() : End Using
+        End Sub
 #End Region
 #Region "Download"
         Friend Overrides Function BaseAuthExists() As Boolean

SCrawler/API/OnlyFans/UserData.vb

@@ -91,6 +91,7 @@ Namespace API.OnlyFans
         End Sub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
+                _DownloadingException_AuthFileUpdate = False
                 If Not MySettings.SessionAborted Then
                     ValidateOFScraper()
                     _AbsMediaIndex = 0
@@ -537,36 +538,16 @@ Namespace API.OnlyFans
         End Sub
 #End Region
 #Region "Auth"
-        Private ReadOnly Property AuthFile As SFile
-            Get
-                Dim f As SFile = MySettings.Responser.File
-                f.Name &= "_Auth"
-                f.Extension = "json"
-                Return f
-            End Get
-        End Property
-        Private Function UpdateSignature(ByVal Path As String, Optional ByVal ForceUpdateAuth As Boolean = False,
-                                         Optional ByVal Round As Integer = 0) As Boolean
+        Private Function UpdateSignature(ByVal Path As String) As Boolean
             Try
-                If UpdateAuthFile(ForceUpdateAuth) Then
-                    Const nullMsg$ = "The auth parameter is null"
-                    Dim j As EContainer
-                    Try
-                        j = JsonDocument.Parse(AuthFile.GetText)
-                    Catch jex As Exception
-                        If Round = 0 Then
-                            AuthFile.Delete()
-                            UpdateAuthFile(True)
-                            Return UpdateSignature(Path, ForceUpdateAuth, Round + 1)
-                        Else
-                            MySettings.SessionAborted = True
-                            Return False
-                        End If
-                    End Try
+                If Not Rules.Update(False) Then Rules.Update(True)
+                If Rules.Exists Then
+                    Const nullMsg$ = "The auth parameter(s) is null"
+                    Dim j As EContainer = Rules.CurrentContainer
                     If Not j Is Nothing Then
-                        Dim pattern$ = j.Value("format")
+                        Dim pattern$ = DynamicRulesEnv.GetFormat(j)
+
                         If pattern.IsEmptyString Then Throw New ArgumentNullException("format", nullMsg)
-                        pattern = pattern.Replace("{}", "{0}").Replace("{:x}", "{1:x}")
 
                         Dim li%() = j("checksum_indexes").Select(Function(e) CInt(e(0).Value)).ToArray
 
@@ -590,38 +571,16 @@ Namespace API.OnlyFans
                         Responser.Headers.Add(HeaderSign, sign)
                         Responser.Headers.Add(HeaderTime, t)
 
-                        j.Dispose()
                         Return True
                     End If
+                Else
+                    MySettings.SessionAborted = True
                 End If
                 Return False
             Catch ex As Exception
                 Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"{ToStringForLog()}: UpdateSignature", False)
             End Try
         End Function
-        Private Function UpdateAuthFile(ByVal Force As Boolean) As Boolean
-            Const urlOld$ = "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json"
-            Const urlNew$ = "https://raw.githubusercontent.com/DIGITALCRIMINALS/dynamic-rules/main/onlyfans.json"
-            Try
-                If MySettings.LastDateUpdated.AddMinutes(CInt(MySettings.DynamicRulesUpdateInterval.Value)) < Now Or Not AuthFile.Exists Or Force Then
-                    Dim r$ = GetWebString(If(ACheck(Of String)(MySettings.DynamicRules.Value),
-                                             CStr(MySettings.DynamicRules.Value),
-                                             IIf(MySettings.UseOldAuthRules.Value, urlOld, urlNew)),, EDP.ReturnValue)
-                    If Not r.IsEmptyString Then
-                        Using j As EContainer = JsonDocument.Parse(r, EDP.ReturnValue)
-                            If j.ListExists Then
-                                If Not j.Value("format").IsEmptyString And j("checksum_indexes").ListExists And
-                                   Not j.Value("static_param").IsEmptyString And Not j.Value("checksum_constant").IsEmptyString Then _
-                                   TextSaver.SaveTextToFile(r, AuthFile, True, False, EDP.ThrowException) : MySettings.LastDateUpdated = Now
-                            End If
-                        End Using
-                    End If
-                End If
-                Return AuthFile.Exists
-            Catch ex As Exception
-                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"{ToStringForLog()}: UpdateAuthFile", False)
-            End Try
-        End Function
         Private Function GetHashSha1(ByVal Input As String) As String
             Dim s As New Security.Cryptography.SHA1CryptoServiceProvider
             Dim inputBytes() As Byte = System.Text.Encoding.UTF8.GetBytes(Input)
@@ -653,37 +612,51 @@ Namespace API.OnlyFans
         Private Function OFS_CreateConfig() As SFile
             Try
                 Const confMainPattern$ = "{0}"": ""([^""]*)"""
+                Const confMainPatternRulesManual$ = "DYNAMIC_RULE"": (""[^""]*"")"
                 If OFSCache Is Nothing Then OFSCache = If(IsSingleObjectDownload, Settings.Cache.NewInstance, CreateCache())
                 Dim currentCache As CacheKeeper = OFSCache.NewInstance
                 currentCache.Validate()
                 Dim cacheRoot As SFile = currentCache.NewPath
                 cacheRoot.Exists(SFO.Path, True, EDP.ThrowException)
-                Dim f As SFile = $"{SettingsFolderName}\OFScraperConfigPattern.json"
+                Dim f As SFile = Rules.OFScraperConfigPatternFile
                 Dim configText$
-                If Not f.Exists Then
-                    configText = Text.Encoding.UTF8.GetString(My.Resources.OFResources.OFScraperConfigPattern)
-                    TextSaver.SaveTextToFile(configText, f, True)
-                End If
                 If f.Exists Then
                     Dim replaceValue$ = String.Empty
                     Dim rp As RParams = RParams.DMS(String.Empty, 1, RegexReturn.Replace, RegexOptions.IgnoreCase,
                                                     CType(Function(input) replaceValue, Func(Of String, String)), String.Empty, EDP.ReturnValue)
                     Dim ff As SFile
                     configText = f.GetText
-                    Dim updateConf As Action(Of String, String) = Sub(ByVal patternValue As String, ByVal __replaceValue As String)
-                                                                      rp.Pattern = String.Format(confMainPattern, patternValue)
-                                                                      rp.Nothing = configText
-                                                                      replaceValue = __replaceValue
-                                                                      configText = RegexReplace(configText, rp)
-                                                                  End Sub
+                    Dim updateConf As Action(Of String, String, Boolean) =
+                        Sub(ByVal patternValue As String, ByVal __replaceValue As String, ByVal __isRules As Boolean)
+                            rp.Pattern = String.Format(IIf(__isRules, confMainPatternRulesManual, confMainPattern), patternValue)
+                            rp.Nothing = configText
+                            replaceValue = __replaceValue
+                            configText = RegexReplace(configText, rp)
+                        End Sub
                     If Not configText.IsEmptyString Then
-                        updateConf("save_location", cacheRoot.PathNoSeparator.Replace("\", "/"))
+                        updateConf("save_location", cacheRoot.PathNoSeparator.Replace("\", "/"), False)
                         If ACheck(MySettings.OFScraperMP4decrypt.Value) Then
                             ff = CStr(MySettings.OFScraperMP4decrypt.Value)
-                            If ff.Exists Then updateConf("mp4decrypt", ff.ToString.Replace("\", "/"))
+                            If ff.Exists Then updateConf("mp4decrypt", ff.ToString.Replace("\", "/"), False)
+                        End If
+                        If Settings.FfmpegFile.Exists Then updateConf("ffmpeg", Settings.FfmpegFile.File.ToString.Replace("\", "/"), False)
+                        updateConf("key-mode-default", CStr(MySettings.KeyModeDefault.Value).IfNullOrEmpty(SiteSettings.KeyModeDefault_Default), False)
+                        updateConf("keydb_api", CStr(MySettings.Keydb_Api.Value), False)
+                        If Rules.RulesReplaceConfig Then
+                            If Rules.RulesConfigManualMode Then
+                                updateConf(DynamicRulesEnv.DynamicRulesConfig_Mode_NodeName, "manual", False)
+                                configText = configText.Replace(DynamicRulesEnv.DynamicRulesConfigNodeName_URL, DynamicRulesEnv.DynamicRulesConfigNodeName_RULES)
+                                updateConf(DynamicRulesEnv.DynamicRulesConfigNodeName_RULES, Rules.CurrentContainerRulesText, True)
+                            Else
+                                Dim confUrlNode$ = If(Rules.RulesConstants.ContainsKey(DynamicRulesEnv.DynamicRulesConfigNodeName_URL_CONST_NAME),
+                                                      Rules.RulesConstants(DynamicRulesEnv.DynamicRulesConfigNodeName_URL_CONST_NAME),
+                                                      DynamicRulesEnv.DynamicRulesConfigNodeName_URL)
+                                updateConf(DynamicRulesEnv.DynamicRulesConfigNodeName_URL, Rules.CurrentRule.UrlRaw, False)
+                                configText = configText.Replace(DynamicRulesEnv.DynamicRulesConfigNodeName_URL, confUrlNode)
+                                If Rules.RulesConstants.ContainsKey(DynamicRulesEnv.DynamicRulesConfig_Mode_NodeName) Then _
+                                   updateConf(DynamicRulesEnv.DynamicRulesConfig_Mode_NodeName, Rules.RulesConstants(DynamicRulesEnv.DynamicRulesConfig_Mode_NodeName), False)
+                            End If
                         End If
-                        If Settings.FfmpegFile.Exists Then updateConf("ffmpeg", Settings.FfmpegFile.File.ToString.Replace("\", "/"))
-                        updateConf("key-mode-default", CStr(MySettings.KeyModeDefault.Value).IfNullOrEmpty(SiteSettings.KeyModeDefault_Default))
                         f = currentCache
                         f.Name = "config"
                         f.Extension = "json"
@@ -777,7 +750,7 @@ Namespace API.OnlyFans
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer
             If Responser.StatusCode = Net.HttpStatusCode.BadRequest Then '400
-                If Not _DownloadingException_AuthFileUpdate AndAlso UpdateAuthFile(True) Then
+                If Not _DownloadingException_AuthFileUpdate AndAlso Rules.Update(True) Then
                     _DownloadingException_AuthFileUpdate = True
                     Return 2
                 Else

SCrawler/API/Pinterest/UserData.vb

@@ -287,7 +287,7 @@ Namespace API.Pinterest
             End Function
         End Class
         Private Function GetDataFromGalleryDL(ByVal URL As String, ByVal IsBoardsRequested As Boolean, ByVal Token As CancellationToken) As List(Of String)
-            Dim command$ = $"gallery-dl --verbose --simulate "
+            Dim command$ = $"""{Settings.GalleryDLFile.File}"" --verbose --simulate "
             Try
                 If Not URL.IsEmptyString Then
                     If MySettings.CookiesNetscapeFile.Exists Then command &= $"--cookies ""{MySettings.CookiesNetscapeFile}"" "

SCrawler/API/PornHub/UserData.vb

@@ -29,7 +29,6 @@ Namespace API.PornHub
         Private Const Name_DownloadFavorite As String = "DownloadFavorite"
         Private Const Name_DownloadGifs As String = "DownloadGifs"
         Private Const Name_DownloadPhotoOnlyFromModelHub As String = "DownloadPhotoOnlyFromModelHub"
-        <Obsolete> Private Const Name_IsUser As String = "IsUser"
 #End Region
 #Region "Structures"
         Private Structure FlashVar : Implements IRegExCreator
@@ -254,14 +253,7 @@ Namespace API.PornHub
                     DownloadFavorite = .Value(Name_DownloadFavorite).FromXML(Of Boolean)(False)
                     DownloadGifs = .Value(Name_DownloadGifs).FromXML(Of Integer)(False)
                     DownloadPhotoOnlyFromModelHub = .Value(Name_DownloadPhotoOnlyFromModelHub).FromXML(Of Boolean)(True)
-                    If .Contains(Name_SiteMode) Then
-                        SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
-                    Else
-                        'TODELETE: PornHub 'IsUser' 20231113
-#Disable Warning BC40008
-                        SiteMode = IIf(.Value(Name_IsUser).FromXML(Of Boolean)(True), SiteModes.User, SiteModes.Search)
-#Enable Warning
-                    End If
+                    SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
                     UpdateUserOptions()
                 Else
                     If UpdateUserOptions() Then .Value(Name_LabelsName) = LabelsString
@@ -404,7 +396,6 @@ Namespace API.PornHub
                 Dim r$ = Responser.GetResponse(URL)
                 If Not r.IsEmptyString Then
                     Dim l As List(Of UserVideo) = RegexFields(Of UserVideo)(r, {RegexUserVideos}, {6, 7, 3, 10})
-                    'URGENT: PornHub: changed list trimming
                     'If l.ListExists And Not SiteMode = SiteModes.Playlists Then l = l.ListTake(3, l.Count).ToList
                     If l.ListExists And Not SiteMode = SiteModes.Playlists Then l = l.ListTake(1, l.Count).ToList
                     If l.ListExists Then

SCrawler/API/Reddit/M3U8.vb

@@ -8,6 +8,7 @@
 ' but WITHOUT ANY WARRANTY
 Imports System.Net
 Imports System.Threading
+Imports SCrawler.API.Base
 Imports SCrawler.API.Reddit.M3U8_Declarations
 Imports PersonalUtilities.Tools
 Imports PersonalUtilities.Tools.Web
@@ -61,15 +62,18 @@ Namespace API.Reddit
         Private ReadOnly ProgressExists As Boolean
         Private ReadOnly Property ProgressPre As PreProgress
         Private ReadOnly UsePreProgress As Boolean
+        Private ReadOnly Media As UserMedia
 #End Region
-        Private Sub New(ByVal URL As String, ByVal OutFile As SFile, ByVal Progress As MyProgress, ByVal UsePreProgress As Boolean)
+        Private Sub New(ByVal URL As String, ByVal Media As UserMedia, ByVal OutFile As SFile, ByVal Progress As MyProgress, ByVal UsePreProgress As Boolean)
             PlayListURL = URL
+            Me.Media = Media
             BaseURL = RegexReplace(URL, BaseUrlPattern)
             Video = New List(Of String)
             Audio = New List(Of String)
             Me.OutFile = OutFile
             Me.OutFile.Name = "PlayListFile"
             Me.OutFile.Extension = "mp4"
+            If Media.Post.Date.HasValue Then Me.OutFile.Name = Media.Post.Date.Value.ToString("yyyyMMdd_HHmmss")
             Me.Progress = Progress
             ProgressExists = Not Me.Progress Is Nothing
             ProgressPre = New PreProgress(Progress)
@@ -202,9 +206,9 @@ Namespace API.Reddit
         End Function
 #End Region
 #Region "Statics"
-        Friend Shared Function Download(ByVal URL As String, ByVal f As SFile, ByVal Token As CancellationToken,
+        Friend Shared Function Download(ByVal URL As String, ByVal Media As UserMedia, ByVal f As SFile, ByVal Token As CancellationToken,
                                         ByVal Progress As MyProgress, ByVal UsePreProgress As Boolean) As SFile
-            Using m As New M3U8(URL, f, Progress, UsePreProgress) : Return m.Download(Token) : End Using
+            Using m As New M3U8(URL, Media, f, Progress, UsePreProgress) : Return m.Download(Token) : End Using
         End Function
 #End Region
 #Region "IDisposable Support"

SCrawler/API/Reddit/SiteSettings.vb

@@ -10,6 +10,7 @@ Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Clients.Base
 Imports PersonalUtilities.Tools.Web.Documents.JSON
 Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Functions.RegularExpressions
@@ -34,6 +35,8 @@ Namespace API.Reddit
                                         "You can find different tokens in the responses. Make sure that bearer token belongs to Reddit and not RedGifs." & vbCr &
                                         "There is not need to add a token if you are not using cookies to download the timeline.", IsAuth:=True)>
         Friend ReadOnly Property BearerToken As PropertyValue
+        <PropertyOption(ControlText:="Use 'cUrl' to get a token", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Private ReadOnly Property BearerTokenUseCurl As PropertyValue
 #Region "TokenUpdateInterval"
         <PropertyOption(ControlText:="Token refresh interval", ControlToolTip:="Interval (in minutes) to refresh the token",
                         AllowNull:=False, LeftOffset:=120, IsAuth:=True), PXML, PClonable>
@@ -55,7 +58,6 @@ Namespace API.Reddit
                 Return {AuthUserName.Value, AuthPassword.Value, ApiClientID.Value, ApiClientSecret.Value}.All(Function(v$) Not v.IsEmptyString)
             End Get
         End Property
-
 #End Region
 #Region "Other"
         <PropertyOption(ControlText:="Use M3U8", ControlToolTip:="Use M3U8 or mp4 for Reddit videos", IsAuth:=False), PXML, PClonable>
@@ -82,7 +84,8 @@ Namespace API.Reddit
             ApiClientID = New PropertyValue(String.Empty, GetType(String))
             ApiClientSecret = New PropertyValue(String.Empty, GetType(String))
             BearerToken = New PropertyValue(token, GetType(String), Sub(v) Responser.Headers.Add(DeclaredNames.Header_Authorization, v))
-            TokenUpdateInterval = New PropertyValue(60 * 12)
+            BearerTokenUseCurl = New PropertyValue(True)
+            TokenUpdateInterval = New PropertyValue(360)
             TokenUpdateIntervalProvider = New TokenRefreshIntervalProvider
             BearerTokenDateUpdate = New PropertyValue(Now.AddYears(-1))
             UseTokenForTimelines = New PropertyValue(False)
@@ -98,6 +101,14 @@ Namespace API.Reddit
             ImageVideoContains = "reddit.com"
             UserRegex = RParams.DM("[htps:/]{7,8}.*?reddit.com/([user]{1,4})/([^/\?&]+)", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
         End Sub
+        Private Const SettingsVersionCurrent As Integer = 1
+        Friend Overrides Sub EndInit()
+            If CInt(SettingsVersion.Value) < SettingsVersionCurrent Then
+                SettingsVersion.Value = SettingsVersionCurrent
+                TokenUpdateInterval.Value = 360
+            End If
+            MyBase.EndInit()
+        End Sub
 #End Region
 #Region "GetInstance"
         Friend Overrides Function GetInstance(ByVal What As Download) As IPluginContentProvider
@@ -269,29 +280,51 @@ Namespace API.Reddit
                     result = False
                     Dim r$ = String.Empty
                     Dim c% = 0
-                    Dim _found As Boolean
+                    Dim useCurl As Boolean = Settings.CurlFile.Exists And CBool(BearerTokenUseCurl.Value)
+                    Dim curlUsed As Boolean = useCurl
                     Do
                         c += 1
                         Using resp As New Responser With {
                             .Method = "POST",
-                            .ProcessExceptionDecision = Function(status, obj, ee) If(status.StatusCode = 429, EDP.ReturnValue, ee)
+                            .ProcessExceptionDecision = Function(ByVal status As IResponserStatus, ByVal nullArg As Object, ByVal currErr As ErrorsDescriber) As ErrorsDescriber
+                                                            If status.StatusCode = 429 Then
+                                                                useCurl = False
+                                                                Return EDP.ReturnValue
+                                                            ElseIf status.StatusCode = Net.HttpStatusCode.Forbidden And Not useCurl And Settings.CurlFile.Exists Then
+                                                                useCurl = True
+                                                                Return EDP.ReturnValue
+                                                            Else
+                                                                Return currErr
+                                                            End If
+                                                        End Function
                         }
                             With resp
-                                With .PayLoadValues
-                                    .Add("grant_type", "password")
-                                    .Add("username", UserName)
-                                    .Add("password", Password)
-                                End With
-                                .CredentialsUserName = ClientID
-                                .CredentialsPassword = ClientSecret
-                                .PreAuthenticate = True
+                                If useCurl Then
+                                    If Settings.CurlFile.Exists Then
+                                        curlUsed = True
+                                        .Mode = Responser.Modes.Curl
+                                        .CurlPath = Settings.CurlFile
+                                        .CurlArgumentsLeft = $"-d ""grant_type=password&username={UserName}&password={Password}"" --user ""{ClientID}:{ClientSecret}"""
+                                    Else
+                                        Throw New ArgumentNullException("cUrl file", "The path to the cUrl file is not specified")
+                                    End If
+                                Else
+                                    .Mode = Responser.Modes.Default
+                                    With .PayLoadValues
+                                        .Add("grant_type", "password")
+                                        .Add("username", UserName)
+                                        .Add("password", Password)
+                                    End With
+                                    .CredentialsUserName = ClientID
+                                    .CredentialsPassword = ClientSecret
+                                    .PreAuthenticate = True
+                                End If
                             End With
                             r = resp.GetResponse("https://www.reddit.com/api/v1/access_token",, EDP.ThrowException)
                         End Using
                         If Not r.IsEmptyString Then
                             Using j As EContainer = JsonDocument.Parse(r)
                                 If j.ListExists Then
-                                    _found = True
                                     Dim newToken$ = j.Value("access_token")
                                     If Not newToken.IsEmptyString Then
                                         BearerToken.Value = $"Bearer {newToken}"
@@ -302,7 +335,7 @@ Namespace API.Reddit
                                 End If
                             End Using
                         End If
-                    Loop While c < 5 And Not _found
+                    Loop While c < 5 And Not result
                 End If
                 Return result
             Catch ex As Exception

SCrawler/API/Reddit/UserData.vb

@@ -225,6 +225,7 @@ Namespace API.Reddit
 #End Region
 #Region "Download Overrides"
         Friend Overrides Sub DownloadData(ByVal Token As CancellationToken)
+            Err429Count = 0
             _CrossPosts.Clear()
             If CreatedByChannel And Settings.FromChannelDownloadTopUse And Settings.FromChannelDownloadTop > 0 Then _
                DownloadTopCount = Settings.FromChannelDownloadTop.Value
@@ -287,6 +288,7 @@ Namespace API.Reddit
         End Sub
 #End Region
 #Region "Download Functions (User, Channel)"
+        Private Err429Count As Integer = 0
         Private _TotalPostsDownloaded As Integer = 0
         Private ReadOnly _CrossPosts As List(Of String)
         Private Const SiteGfycatKey As String = "gfycat"
@@ -375,6 +377,7 @@ Namespace API.Reddit
             Loop While Not _completed
         End Sub
         Private Sub DownloadDataChannel(ByVal POST As String, ByVal Token As CancellationToken)
+            Const savedPostsSleepTimer% = 2000
             Dim eObj% = 0
             Dim round% = 0
             Dim URL$ = String.Empty
@@ -392,12 +395,14 @@ Namespace API.Reddit
 
                     If IsSavedPosts Then
                         URL = $"https://www.reddit.com/user/{TrueName}/saved.json?after={POST}"
+                        If Not POST.IsEmptyString Then Thread.Sleep(savedPostsSleepTimer)
                     Else
                         URL = $"https://reddit.com/r/{TrueName}/{View}.json?allow_quarantined=true&allow_over18=1&include=identity&after={POST}&dist=25&sort={View}&t={Period}&layout=classic"
                     End If
 
                     ThrowAny(Token)
                     Dim r$ = Responser.GetResponse(URL)
+                    If IsSavedPosts Then Err429Count = 0
                     If Not r.IsEmptyString Then
                         Using w As EContainer = JsonDocument.Parse(r).XmlIfNothing
                             If w.Count > 0 Then
@@ -458,8 +463,12 @@ Namespace API.Reddit
                     End If
                     _completed = True
                 Catch ex As Exception
-                    If ProcessException(ex, Token, $"channel data downloading error [{URL}]",, eObj) = HttpStatusCode.InternalServerError Then
+                    Dim errValue% = ProcessException(ex, Token, $"{IIf(IsSavedPosts, "saved posts", "channel")} data downloading error [{URL}]",, eObj)
+                    If errValue = HttpStatusCode.InternalServerError Then
                         If round = 2 Then eObj = HttpStatusCode.InternalServerError
+                    ElseIf errValue = 429 And round = 0 Then
+                        Thread.Sleep(savedPostsSleepTimer)
+                        round += 1
                     Else
                         _completed = True
                     End If
@@ -905,19 +914,24 @@ Namespace API.Reddit
                     Dim RedGifsHost As SettingsHost = Settings(RedGifs.RedGifsSiteKey, RedGifsAccount)
                     If RedGifsHost Is Nothing Then RedGifsHost = Settings(RedGifs.RedGifsSiteKey).Default
                     RedGifsResponser = RedGifsHost.Responser.Copy
+                    Dim respNoHeaders As Responser = Responser.Copy
                     Dim m As UserMedia, m2 As UserMedia
-                    Dim r$
+                    Dim r$, url$
                     Dim j As EContainer
                     Dim lastCount%, li%
+                    Dim rv As New ErrorsDescriber(EDP.ReturnValue)
+                    respNoHeaders.Headers.Clear()
                     ProgressPre.ChangeMax(_ContentList.Count)
                     For i% = 0 To _ContentList.Count - 1
                         m = _ContentList(i)
                         ProgressPre.Perform()
                         If m.State = UStates.Missing AndAlso Not m.Post.ID.IsEmptyString Then
                             ThrowAny(Token)
-                            r = Responser.GetResponse($"https://www.reddit.com/comments/{m.Post.ID.Split("_").LastOrDefault}/.json",, EDP.ReturnValue)
+                            url = $"https://www.reddit.com/comments/{m.Post.ID.Split("_").LastOrDefault}/.json"
+                            r = Responser.GetResponse(url,, rv)
+                            If r.IsEmptyString Then r = respNoHeaders.GetResponse(url,, rv)
                             If Not r.IsEmptyString Then
-                                j = JsonDocument.Parse(r, EDP.ReturnValue)
+                                j = JsonDocument.Parse(r, rv)
                                 If Not j Is Nothing Then
                                     If j.Count > 0 Then
                                         lastCount = _TempMediaList.Count
@@ -975,7 +989,7 @@ Namespace API.Reddit
             Dim m As New UserMedia(_URL, t) With {.Post = New UserPost With {.ID = PostID, .UserID = _UserID}}
             If t = UTypes.Picture Or t = UTypes.GIF Then m.File = CreateFileFromUrl(m.URL) Else m.File = Nothing
             If ReplacePreview And m.URL.Contains("preview") And Not t = UTypes.Picture Then m.URL = $"https://i.redd.it/{m.File.File}"
-            If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, DateTrueProvider(IsChannel), Nothing) Else m.Post.Date = Nothing
+            If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, DateTrueProvider(IsChannel Or IsSavedPosts), Nothing) Else m.Post.Date = Nothing
             Return m
         End Function
         Private Function TryFile(ByVal URL As String) As Boolean
@@ -1027,7 +1041,7 @@ Namespace API.Reddit
             Return URL.Contains(SiteRedGifsKey)
         End Function
         Protected Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile, ByVal Token As CancellationToken) As SFile
-            Return M3U8.Download(URL, DestinationFile, Token, Progress, Not IsSingleObjectDownload)
+            Return M3U8.Download(URL, Media, DestinationFile, Token, Progress, Not IsSingleObjectDownload)
         End Function
         Protected Overrides Function ChangeFileNameByProvider(ByVal f As SFile, ByVal m As UserMedia) As SFile
             If Not IsChannel Or Not SaveToCache Then
@@ -1057,8 +1071,11 @@ Namespace API.Reddit
                 ElseIf .StatusCode = HttpStatusCode.InternalServerError Then '500
                     If Not IsNothing(EObj) AndAlso IsNumeric(EObj) AndAlso CInt(EObj) = HttpStatusCode.InternalServerError Then Return 1
                     Return HttpStatusCode.InternalServerError
+                ElseIf .StatusCode = 429 And IsSavedPosts And Err429Count = 0 Then
+                    Err429Count += 1
+                    Return 429
                 ElseIf .StatusCode = 429 AndAlso
-                       ((Not IsSavedPosts And CBool(MySiteSettings.UseTokenForTimelines.Value)) Or (IsSavedPosts And MySiteSettings.UseTokenForSavedPosts.Value)) AndAlso
+                       ((Not IsSavedPosts And CBool(MySiteSettings.UseTokenForTimelines.Value)) Or (IsSavedPosts And CBool(MySiteSettings.UseTokenForSavedPosts.Value))) AndAlso
                        Not MySiteSettings.CredentialsExists Then '429
                     MyMainLOG = $"{ToStringForLog()}: [{CInt(Responser.StatusCode)}] You should use OAuth authorization or disable " &
                                 IIf(IsSavedPosts, "token usage for downloading saved posts", "the use of token and cookies for downloading timelines")

SCrawler/API/Redgifs/SiteSettings.vb

@@ -18,9 +18,9 @@ Namespace API.RedGifs
     <Manifest(RedGifsSiteKey)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
-        <PropertyOption(ControlToolTip:="Bearer token", AllowNull:=False), DependentFields(NameOf(UserAgent)), ControlNumber(1), PClonable(Clone:=False)>
+        <PropertyOption(ControlToolTip:="Bearer token", AllowNull:=False), DependentFields(NameOf(UserAgent)), ControlNumber(1), PClonable(Clone:=False), HiddenControl>
         Friend ReadOnly Property Token As PropertyValue
-        <PropertyOption, ControlNumber(2), PClonable>
+        <PropertyOption, ControlNumber(2), PClonable, HiddenControl>
         Private ReadOnly Property UserAgent As PropertyValue
         <PXML> Friend ReadOnly Property TokenLastDateUpdated As PropertyValue
         Private Const TokenName As String = "authorization"
@@ -107,7 +107,9 @@ Namespace API.RedGifs
         Friend Overrides Sub Update()
             If _SiteEditorFormOpened Then
                 Dim NewToken$ = AConvert(Of String)(Token.Value, AModes.Var, String.Empty)
-                If Not _LastTokenValue = NewToken Then TokenLastDateUpdated.Value = Now
+                If Not _LastTokenValue = NewToken And Not NewToken.IsEmptyString Then TokenLastDateUpdated.Value = Now
+                If Responser.CookiesExists AndAlso MsgBoxE({"RedGifs doesn't require cookies! Do you still want to use cookies?", "RedGifs cookies"},
+                                                           vbExclamation,,, {"Use", "Don't use"}) = 1 Then Responser.Cookies.Clear()
             End If
             MyBase.Update()
         End Sub

SCrawler/API/ThreadsNet/SiteSettings.vb

@@ -13,8 +13,9 @@ Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Cookies
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports IG = SCrawler.API.Instagram.SiteSettings
+Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.ThreadsNet
-    <Manifest("AndyProgram_ThreadsNet"), SeparatedTasks(1)>
+    <Manifest("AndyProgram_ThreadsNet"), SavedPosts, SeparatedTasks(1)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
 #Region "Authorization"
@@ -25,6 +26,10 @@ Namespace API.ThreadsNet
                 Return __HH_CSRF_TOKEN
             End Get
         End Property
+        <CookieValueExtractor(NameOf(HH_CSRF_TOKEN))>
+        Private Function GetValueFromCookies(ByVal PropName As String, ByVal c As CookieKeeper) As String
+            Return c.GetCookieValue(IG.Header_CSRF_TOKEN_COOKIE, PropName, NameOf(HH_CSRF_TOKEN))
+        End Function
         <PClonable> Protected ReadOnly __HH_IG_APP_ID As PropertyValue
         <PropertyOption(ControlText:="x-ig-app-id", AllowNull:=False, IsAuth:=True), ControlNumber(10)>
         Friend Overridable ReadOnly Property HH_IG_APP_ID As PropertyValue
@@ -36,10 +41,10 @@ Namespace API.ThreadsNet
         Friend ReadOnly Property HH_ASBD_ID As PropertyValue
         <PropertyOption(ControlText:="sec-ch-ua", AllowNull:=True, IsAuth:=True,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua), ControlNumber(30), PClonable, PXML(OnlyForChecked:=True)>
-        Private ReadOnly Property HH_BROWSER As PropertyValue
+        Friend ReadOnly Property HH_BROWSER As PropertyValue
         <PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:=SettingsCLS.HEADER_DEF_sec_ch_ua_full_version_list, AllowNull:=True, IsAuth:=True,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua_full_version_list), ControlNumber(40), PClonable, PXML(OnlyForChecked:=True)>
-        Private ReadOnly Property HH_BROWSER_EXT As PropertyValue
+        Friend ReadOnly Property HH_BROWSER_EXT As PropertyValue
         <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:=SettingsCLS.HEADER_DEF_sec_ch_ua_platform_version, AllowNull:=True, IsAuth:=True, LeftOffset:=135,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua_platform_version), ControlNumber(50), PClonable, PXML(OnlyForChecked:=True)>
         Friend ReadOnly Property HH_PLATFORM_VER As PropertyValue
@@ -71,14 +76,14 @@ Namespace API.ThreadsNet
 #Region "Other properties"
         <PropertyOption(ControlText:="Request timer (any)",
                         ControlToolTip:="The timer (in milliseconds) that SCrawler should wait before executing the next request." &
-                        vbCr & "The default value is 1'000." & vbCr & "The minimum value is 0." & IG.TimersUrgentTip, AllowNull:=False),
+                        vbCr & "The default value is 1'000." & vbCr & "The minimum value is 0." & IG.TimersUrgentTip, AllowNull:=False, Category:=DN.CAT_Timers),
                         PXML, PClonable>
         Friend ReadOnly Property RequestsWaitTimer_Any As PropertyValue
         <Provider(NameOf(RequestsWaitTimer_Any), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimer_AnyProvider As IFormatProvider
         <PropertyOption(ControlText:="Download data",
                         ControlToolTip:="The internal value indicates that site data should be downloaded." & vbCr &
-                                        "It becomes unchecked when the site returns an error."), PXML>
+                                        "It becomes unchecked when the site returns an error.", Category:="Download"), PXML>
         Friend ReadOnly Property DownloadData_Impl As PropertyValue
 #End Region
 #End Region
@@ -103,8 +108,6 @@ Namespace API.ThreadsNet
 
             With Responser
                 .Accept = "*/*"
-                'URGENT: remove after debug
-                .DeclaredError = EDP.SendToLog + EDP.ThrowException
                 If .UserAgentExists Then useragent = .UserAgent
                 With .Headers
                     If .Count > 0 Then
@@ -125,7 +128,10 @@ Namespace API.ThreadsNet
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode, "cors"))
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchSite, "same-origin"))
                     .Add("Sec-Fetch-User", "?1")
+                    .Add("dnt", 1)
+                    .Add("drp", 1)
                     .Add(Instagram.UserData.GQL_HEADER_FB_FRINDLY_NAME, "BarcelonaProfileThreadsTabRefetchableQuery")
+                    .Remove("dht")
                 End With
                 .CookiesExtractMode = Responser.CookiesExtractModes.Any
                 .CookiesUpdateMode = CookieKeeper.UpdateModes.ReplaceByNameAll
@@ -195,7 +201,7 @@ Namespace API.ThreadsNet
         End Sub
         Friend Overrides Sub Update()
             If _SiteEditorFormOpened And Responser.CookiesExists Then
-                Dim csrf$ = If(Responser.Cookies.FirstOrDefault(Function(c) c.Name.StringToLower = IG.Header_CSRF_TOKEN_COOKIE)?.Value, String.Empty)
+                Dim csrf$ = GetValueFromCookies(NameOf(HH_CSRF_TOKEN), Responser.Cookies)
                 If Not csrf.IsEmptyString Then HH_CSRF_TOKEN.Value = csrf
                 If Not __Cookies Is Nothing AndAlso Not __Cookies.ListEquals(Responser.Cookies) Then DownloadData_Impl.Value = True
             End If

SCrawler/API/ThreadsNet/UserData.vb

@@ -17,6 +17,10 @@ Imports PersonalUtilities.Tools.Web.Clients.EventArguments
 Imports IGS = SCrawler.API.Instagram.SiteSettings
 Namespace API.ThreadsNet
     Friend Class UserData : Inherits Instagram.UserData
+#Region "XML names"
+        Private Const Name_MaxLastDownDate As String = "MaxLastDownDate"
+        Private Const Name_FirstLoadingDone As String = "FirstLoadingDone"
+#End Region
 #Region "Declarations"
         Private ReadOnly Property MySettings As SiteSettings
             Get
@@ -29,9 +33,20 @@ Namespace API.ThreadsNet
                 Return ValidateBaseTokens() And Not ID.IsEmptyString
             End Get
         End Property
+        Private Property MaxLastDownDate As Date? = Nothing
+        Private Property FirstLoadingDone As Boolean = False
 #End Region
 #Region "Loader"
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+            With Container
+                If Loading Then
+                    MaxLastDownDate = AConvert(Of Date)(.Value(Name_MaxLastDownDate), DateTimeDefaultProvider, Nothing)
+                    FirstLoadingDone = .Value(Name_FirstLoadingDone).FromXML(Of Boolean)(False)
+                Else
+                    .Add(Name_MaxLastDownDate, AConvert(Of String)(MaxLastDownDate, DateTimeDefaultProvider, String.Empty))
+                    .Add(Name_FirstLoadingDone, FirstLoadingDone.BoolToInteger)
+                End If
+            End With
         End Sub
 #End Region
 #Region "Exchange"
@@ -49,6 +64,7 @@ Namespace API.ThreadsNet
             DefaultParser_PostUrlCreator = Function(post) $"https://www.threads.net/@{NameTrue}/post/{post.Code}"
             _ResponserAutoUpdateCookies = True
             _ResponserAddResponseReceivedHandler = True
+            DefaultParser_Pinned = AddressOf IsPinnedPost
         End Sub
 #End Region
 #Region "Download functions"
@@ -66,7 +82,32 @@ Namespace API.ThreadsNet
                     Responser.Method = "POST"
                     LoadSavePostsKV(True)
                     ResetBaseTokens()
-                    DownloadData(String.Empty, Token)
+                    Dim setMaxPostDate As Action(Of List(Of UserMedia)) =
+                        Sub(ByVal l As List(Of UserMedia))
+                            With (From c As UserMedia In l Where c.Post.Date.HasValue Select c.Post.Date.Value)
+                                If .ListExists Then MaxLastDownDate = .Max : _ForceSaveUserInfo = True
+                            End With
+                        End Sub
+                    If FirstLoadingDone Then
+                        If Not MaxLastDownDate.HasValue And _ContentList.Count > 0 Then setMaxPostDate.Invoke(_ContentList)
+                    Else
+                        If _ContentList.Count > 0 Then
+                            FirstLoadingDone = True
+                            If Not MaxLastDownDate.HasValue Then setMaxPostDate.Invoke(_ContentList)
+                        End If
+                    End If
+                    If FirstLoadingDone Then
+                        DefaultParser_SkipPost = Nothing
+                    Else
+                        DefaultParser_SkipPost = AddressOf SkipPost
+                    End If
+                    If IsSavedPosts Then
+                        DefaultParser_ElemNode = {"node", "thread_items", 0, "post"}
+                        DownloadSavedPosts(String.Empty, Token)
+                    Else
+                        DownloadData(String.Empty, Token)
+                    End If
+                    If _TempMediaList.Count > 0 Then FirstLoadingDone = True : setMaxPostDate.Invoke(_TempMediaList)
                 Catch ex As Exception
                     errorFound = True
                     Throw ex
@@ -78,6 +119,25 @@ Namespace API.ThreadsNet
                 End Try
             End If
         End Sub
+        Private Function IsPinnedPost(ByVal Items As IEnumerable(Of EContainer), ByVal Index As Integer) As Boolean
+            Try
+                If IsSavedPosts Then
+                    Return False
+                Else
+                    If MaxLastDownDate.HasValue Then
+                        Dim d As Date? = AConvert(Of Date)(Items(Index).ItemF(DefaultParser_ElemNode_Default).Value("taken_at"), UnixDate32Provider, Nothing)
+                        If d.HasValue Then Return d.Value < MaxLastDownDate.Value
+                    End If
+                    Return Not FirstLoadingDone
+                End If
+            Catch ex As Exception
+                LogError(ex, "IsPinnedPost")
+                Return Not FirstLoadingDone
+            End Try
+        End Function
+        Private Function SkipPost(ByVal Items As IEnumerable(Of EContainer), ByVal Index As Integer, ByVal Post As PostKV) As Boolean
+            Return PostKvExists(Post)
+        End Function
         Protected Overrides Sub UpdateResponser()
             If Not Responser Is Nothing AndAlso Not Responser.Disposed Then
                 RemoveHandler Responser.ResponseReceived, AddressOf Responser_ResponseReceived
@@ -90,22 +150,32 @@ Namespace API.ThreadsNet
                    Responser.Headers.Add(IGS.Header_CSRF_TOKEN, csrf)
             End If
         End Sub
+        Private Const GQL_Q As String = "https://www.threads.net/api/graphql?lsd={0}&fb_dtsg={1}&doc_id={2}&fb_api_req_friendly_name={3}&server_timestamps=true&variables={4}"
+        Private Const GQL_P_DOC_ID As String = "6371597506283707"
+        Private Const GQL_P_NAME As String = "BarcelonaProfileThreadsTabRefetchableQuery"
+        Private Const GQL_S_DOC_ID_1 As String = "7758166704280174"
+        Private Const GQL_S_NAME_1 As String = "BarcelonaSavedPageViewerQuery"
+        Private Const GQL_S_DOC_ID_2 As String = "8617275414954442"
+        Private Const GQL_S_NAME_2 As String = "BarcelonaSavedPageRefetchableQuery"
+        Private Sub DownloadCheckCredentials()
+            If Not Valid Then
+                Dim idIsNull As Boolean = ID.IsEmptyString
+                UpdateCredentials()
+                If idIsNull And Not ID.IsEmptyString Then _ForceSaveUserInfo = True
+            End If
+            If Not Valid Then DisableDownload() : Throw New Plugin.ExitException("Some credentials are missing")
+        End Sub
         Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
-            Const urlPattern$ = "https://www.threads.net/api/graphql?lsd={0}&variables={1}&doc_id=6371597506283707&fb_api_req_friendly_name=BarcelonaProfileThreadsTabRefetchableQuery&server_timestamps=true&fb_dtsg={2}"
             Const var_init$ = """userID"":""{0}"""
             Const var_cursor$ = """after"":""{1}"",""before"":null,""first"":25,""last"":null,""userID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false"
             Dim URL$ = String.Empty
             Try
-                If Not Valid Then
-                    Dim idIsNull As Boolean = ID.IsEmptyString
-                    UpdateCredentials()
-                    If idIsNull And Not ID.IsEmptyString Then _ForceSaveUserInfo = True
-                End If
-                If Not Valid Then DisableDownload() : Throw New Plugin.ExitException("Some credentials are missing")
+                DownloadCheckCredentials()
 
                 Responser.Method = "POST"
                 Responser.Referer = $"https://www.threads.net/@{NameTrue}"
                 Responser.Headers.Add(GQL_HEADER_FB_LSD, Token_lsd)
+                Responser.Headers.Add(GQL_HEADER_FB_FRINDLY_NAME, GQL_P_NAME)
 
                 Dim nextCursor$ = String.Empty
                 Dim dataFound As Boolean = False
@@ -118,7 +188,7 @@ Namespace API.ThreadsNet
                 End If
                 vars = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & vars & "}")
 
-                URL = String.Format(urlPattern, Token_lsd, vars, Token_dtsg_Var)
+                URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_P_DOC_ID, GQL_P_NAME, vars)
 
                 Using j As EContainer = GetDocument(URL, Token)
                     If j.ListExists Then
@@ -138,6 +208,47 @@ Namespace API.ThreadsNet
                 ProcessException(ex, Token, $"data downloading error [{URL}]")
             End Try
         End Sub
+        Private Sub DownloadSavedPosts(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Const var_init$ = """__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
+            Const var_cursor$ = """after"":""{0}"",""first"":25,""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
+            Dim URL$ = String.Empty
+            Try
+                DownloadCheckCredentials()
+
+                Responser.Method = "POST"
+                Responser.Referer = "https://www.threads.net/"
+                Responser.Headers.Add(GQL_HEADER_FB_LSD, Token_lsd)
+                Responser.Headers.Add(GQL_HEADER_FB_FRINDLY_NAME, If(Cursor.IsEmptyString, GQL_S_NAME_1, GQL_S_NAME_2))
+
+                Dim nextCursor$ = String.Empty
+                Dim dataFound As Boolean = False
+
+                Dim vars$ = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & If(Cursor.IsEmptyString, var_init, String.Format(var_cursor, Cursor)) & "}")
+
+                If Cursor.IsEmptyString Then
+                    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_1, GQL_S_NAME_1, vars)
+                Else
+                    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_2, GQL_S_NAME_2, vars)
+                End If
+
+                Using j As EContainer = GetDocument(URL, Token)
+                    If j.ListExists Then
+                        With j({"data", "xdt_viewer", "text_app_saved_media"})
+                            If .ListExists Then
+                                nextCursor = .Value({"page_info"}, "end_cursor")
+                                With .Item({"edges"})
+                                    If .ListExists Then dataFound = DefaultParser(.Self, Sections.Timeline, Token)
+                                End With
+                            End If
+                        End With
+                    End If
+                End Using
+
+                If dataFound And Not nextCursor.IsEmptyString Then DownloadSavedPosts(nextCursor, Token)
+            Catch ex As Exception
+                ProcessException(ex, Token, $"saved posts downloading error [{URL}]")
+            End Try
+        End Sub
         Private Function GetDocument(ByVal URL As String, ByVal Token As CancellationToken, Optional ByVal Round As Integer = 0) As EContainer
             Try
                 ThrowAny(Token)
@@ -155,41 +266,36 @@ Namespace API.ThreadsNet
             End Try
         End Function
         Private Function UpdateCredentials(Optional ByVal e As ErrorsDescriber = Nothing) As Boolean
-            Dim URL$ = $"https://www.threads.net/@{NameTrue}"
+            Dim URL$ = If(IsSavedPosts, "https://www.threads.net/", $"https://www.threads.net/@{NameTrue}")
             ResetBaseTokens()
+            Dim headers As New HttpHeaderCollection
+            headers.AddRange(Responser.Headers)
             Try
-                Responser.Method = "GET"
-                Responser.Referer = URL
-                Responser.Headers.Remove(GQL_HEADER_FB_LSD)
+                With Responser
+                    .Method = "GET"
+                    .Referer = URL
+                    With .Headers
+                        .Clear()
+                        .Add("dnt", 1)
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Authority, "www.threads.net"))
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Origin, "https://www.threads.net"))
+                        .Add("Sec-Ch-Ua-Model", "")
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaMobile, "?0"))
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform, """Windows"""))
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchDest, "document"))
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode, "navigate"))
+                        .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchSite, "none"))
+                        .Add("Upgrade-Insecure-Requests", 1)
+                        .Add("Sec-Fetch-User", "?1")
+                        .Add(IGS.Header_Browser, MySettings.HH_BROWSER.Value)
+                        .Add(IGS.Header_BrowserExt, MySettings.HH_BROWSER_EXT.Value)
+                    End With
+                End With
                 WaitTimer()
                 Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException)
-                Dim rr As RParams
-                Dim tt$, ttVal$
                 If Not r.IsEmptyString Then
-                    rr = RParams.DM(Instagram.PageTokenRegexPatternDefault, 0, RegexReturn.List, EDP.ReturnValue)
-                    Dim tokens As List(Of String) = RegexReplace(r, rr)
-                    If tokens.ListExists Then
-                        With rr
-                            .Match = Nothing
-                            .MatchSub = 1
-                            .WhatGet = RegexReturn.Value
-                        End With
-                        For Each tt In tokens
-                            If Not Token_dtsg.IsEmptyString And Not Token_lsd.IsEmptyString Then
-                                Exit For
-                            Else
-                                ttVal = RegexReplace(tt, rr)
-                                If Not ttVal.IsEmptyString Then
-                                    If ttVal.Contains(":") Then
-                                        If Token_dtsg.IsEmptyString Then Token_dtsg = ttVal
-                                    Else
-                                        If Token_lsd.IsEmptyString Then Token_lsd = ttVal
-                                    End If
-                                End If
-                            End If
-                        Next
-                    End If
-                    If ID.IsEmptyString Then ID = RegexReplace(r, RParams.DMS("""props"":\{""user_id"":""(\d+)""\},", 1, EDP.ReturnValue))
+                    ParseTokens(r, 0)
+                    If ID.IsEmptyString Then ID = RegexReplace(r, RParams.DMS("""props"":\{""user_id"":""(\d+)""", 1, EDP.ReturnValue))
                 End If
                 Return Valid
             Catch ex As Exception
@@ -204,6 +310,12 @@ Namespace API.ThreadsNet
                 'LogError(ex, $"failed to update some{IIf(notFound.IsEmptyString, String.Empty, $" ({notFound})")} credentials", e)
                 LogError(eex, String.Empty, e)
                 Return False
+            Finally
+                If headers.ListExists Then
+                    Responser.Headers.Clear()
+                    Responser.Headers.AddRange(headers)
+                    headers.Dispose()
+                End If
             End Try
         End Function
 #End Region

SCrawler/API/TikTok/UserData.vb

@@ -359,6 +359,11 @@ Namespace API.TikTok
             _TempMediaList.Add(m)
         End Sub
 #End Region
+#Region "EraseData"
+        Protected Overrides Sub EraseData_AdditionalDataFiles()
+            LastDownloadDate = Nothing
+        End Sub
+#End Region
 #Region "Exception"
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer

SCrawler/API/Twitter/Declarations.vb

@@ -15,6 +15,7 @@ Namespace API.Twitter
         Friend Const TwitterSiteKey As String = "AndyProgram_Twitter"
         Friend ReadOnly DateProvider As ADateTime = GetDateProvider()
         Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue)
+        Friend ReadOnly StatusRegEx As RParams = RParams.DM(".*?(twitter|x)\.com/\S+/status/\d+", 0, EDP.ReturnValue)
         Private Function GetDateProvider() As ADateTime
             Dim n As DateTimeFormatInfo = CultureInfo.GetCultureInfo("en-us").DateTimeFormat.Clone
             n.FullDateTimePattern = "ddd MMM dd HH:mm:ss +ffff yyyy"

SCrawler/API/Twitter/EditorExchangeOptions.vb

@@ -28,16 +28,20 @@ Namespace API.Twitter
         Friend Overridable Property MediaModelAllowNonUserTweets As Boolean = False
         <PSetting(Address:=SettingAddress.User,
                   Caption:="Download model 'Media'",
-                  ToolTip:="Download the data using the 'https://twitter.com/UserName/media' command.", LeftOffset:=DefaultOffset)>
+                  ToolTip:="Download the data using the 'https://x.com/UserName/media' command.", LeftOffset:=DefaultOffset)>
         Friend Overridable Property DownloadModelMedia As Boolean = False
         <PSetting(Address:=SettingAddress.User,
                   Caption:="Download model 'Profile'",
-                  ToolTip:="Download the data using the 'https://twitter.com/UserName' command.", LeftOffset:=DefaultOffset)>
+                  ToolTip:="Download the data using the 'https://x.com/UserName' command.", LeftOffset:=DefaultOffset)>
         Friend Overridable Property DownloadModelProfile As Boolean = False
         <PSetting(Address:=SettingAddress.User,
                   Caption:="Download model 'Search'",
-                  ToolTip:="Download the data using the 'https://twitter.com/search?q=from:UserName+include:nativeretweets' command.", LeftOffset:=DefaultOffset)>
+                  ToolTip:="Download the data using the 'https://x.com/search?q=from:UserName+include:nativeretweets' command.", LeftOffset:=DefaultOffset)>
         Friend Overridable Property DownloadModelSearch As Boolean = False
+        <PSetting(Address:=SettingAddress.User,
+                  Caption:="Download model 'Likes'",
+                  ToolTip:="Download the data using the 'https://x.com/UserName/likes' command.", LeftOffset:=DefaultOffset)>
+        Friend Overridable Property DownloadModelLikes As Boolean = False
         <PSetting(Address:=SettingAddress.User,
                   Caption:="Force apply",
                   ToolTip:="Force overrides the default parameters for the first download." & vbCr & "Applies to first download only.", LeftOffset:=DefaultOffset)>
@@ -73,6 +77,7 @@ Namespace API.Twitter
                     DownloadModelMedia = dm.Contains(DModels.Media)
                     DownloadModelProfile = dm.Contains(DModels.Profile)
                     DownloadModelSearch = dm.Contains(DModels.Search)
+                    DownloadModelLikes = dm.Contains(DModels.Likes)
                 End If
             End If
             MySettings = u.HOST.Source

SCrawler/API/Twitter/SiteSettings.vb

@@ -16,32 +16,37 @@ Namespace API.Twitter
     <Manifest(TwitterSiteKey), SavedPosts, SeparatedTasks, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
+#Region "Categories"
+        Private Const CAT_DOWN As String = "Downloading"
+#End Region
 #Region "Other properties"
         <PropertyOption(ControlText:="Use the appropriate model",
                         ControlToolTip:="Use the appropriate model for new users." & vbCr &
                         "If disabled, all download models will be used for the first download. " &
                         "Next, the appropriate download model will be automatically selected." & vbCr &
-                        "Otherwise the appropriate download model will be selected right from the start."), PXML, PClonable>
+                        "Otherwise the appropriate download model will be selected right from the start.", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property UseAppropriateModel As PropertyValue
 #Region "End points"
-        <PropertyOption(ControlText:="New endpoint: search", ControlToolTip:="Use new endpoint argument (-o search-endpoint=graphql) for the search model."), PXML, PClonable>
+        <PropertyOption(ControlText:="New endpoint: search", ControlToolTip:="Use new endpoint argument (-o search-endpoint=graphql) for the search model.",
+                        Category:=CAT_DOWN), PXML, PClonable>
         Friend Property UseNewEndPointSearch As PropertyValue
-        <PropertyOption(ControlText:="New endpoint: profiles", ControlToolTip:="Use new endpoint argument (-o search-endpoint=graphql) for the profile models."), PXML, PClonable>
+        <PropertyOption(ControlText:="New endpoint: profiles", ControlToolTip:="Use new endpoint argument (-o search-endpoint=graphql) for the profile models.",
+                        Category:=CAT_DOWN), PXML, PClonable>
         Friend Property UseNewEndPointProfiles As PropertyValue
 #End Region
 #Region "Limits"
-        <PropertyOption(ControlText:="Abort on limit", ControlToolTip:="Abort twitter downloading when limit is reached"), PXML, PClonable>
+        <PropertyOption(ControlText:="Abort on limit", ControlToolTip:="Abort twitter downloading when limit is reached", Category:=CAT_DOWN), PXML, PClonable>
         Friend Property AbortOnLimit As PropertyValue
-        <PropertyOption(ControlText:="Download already parsed", ControlToolTip:="Download already parsed content on abort"), PXML, PClonable>
+        <PropertyOption(ControlText:="Download already parsed", ControlToolTip:="Download already parsed content on abort", Category:=CAT_DOWN), PXML, PClonable>
         Friend Property DownloadAlreadyParsed As PropertyValue
 #End Region
-        <PropertyOption(ControlText:="Media Model: allow non-user tweets", ControlToolTip:="Allow downloading non-user tweets in the media-model."), PXML, PClonable>
+        <PropertyOption(ControlText:="Media Model: allow non-user tweets", ControlToolTip:="Allow downloading non-user tweets in the media-model.", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property MediaModelAllowNonUserTweets As PropertyValue
-        <PropertyOption(ControlText:=DN.GifsDownloadCaption), PXML, PClonable>
+        <PropertyOption(ControlText:=DN.GifsDownloadCaption, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsDownload As PropertyValue
-        <PropertyOption(ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip), PXML, PClonable>
+        <PropertyOption(ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsSpecialFolder As PropertyValue
-        <PropertyOption(ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip), PXML, PClonable>
+        <PropertyOption(ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property GifsPrefix As PropertyValue
         <Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)>
         Private ReadOnly Property GifStringChecker As IFormatProvider
@@ -63,17 +68,17 @@ Namespace API.Twitter
                 Throw New NotImplementedException("[GetFormat] is not available in the context of [GifStringProvider]")
             End Function
         End Class
-        <PropertyOption(ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip), PXML, PClonable>
+        <PropertyOption(ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property UseMD5Comparison As PropertyValue
         <PropertyOption(ControlText:=DN.ConcurrentDownloadsCaption,
-                        ControlToolTip:=DN.ConcurrentDownloadsToolTip, AllowNull:=False, LeftOffset:=120), PXML, TaskCounter, PClonable>
+                        ControlToolTip:=DN.ConcurrentDownloadsToolTip, AllowNull:=False, LeftOffset:=120, Category:=CAT_DOWN), PXML, TaskCounter, PClonable>
         Friend ReadOnly Property ConcurrentDownloads As PropertyValue
         <Provider(NameOf(ConcurrentDownloads), FieldsChecker:=True)>
         Private ReadOnly Property MyConcurrentDownloadsProvider As IFormatProvider
 #End Region
 #End Region
         Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
-            MyBase.New(TwitterSite, "twitter.com", AccName, Temp, My.Resources.SiteResources.TwitterIcon_32, My.Resources.SiteResources.TwitterIcon_32.ToBitmap)
+            MyBase.New(TwitterSite, "x.com", AccName, Temp, My.Resources.SiteResources.TwitterIcon_32, My.Resources.SiteResources.TwitterIcon_32.ToBitmap)
 
             LimitSkippedUsers = New List(Of UserDataBase)
 
@@ -96,8 +101,8 @@ Namespace API.Twitter
             ConcurrentDownloads = New PropertyValue(1)
             MyConcurrentDownloadsProvider = New ConcurrentDownloadsProvider
 
-            UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "/(twitter|x).com/"), 2)
-            UrlPatternUser = "https://twitter.com/{0}"
+            UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, $"/(twitter|x).com({CommunitiesUser}|)/"), 3)
+            UrlPatternUser = "https://x.com/{0}"
             ImageVideoContains = "twitter"
             CheckNetscapeCookiesOnEndInit = True
             UseNetscapeCookies = True
@@ -105,8 +110,9 @@ Namespace API.Twitter
         Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
             Return New UserData
         End Function
+        Friend Const SinglePostPattern As String = "https://x.com/i/web/status/{0}"
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
-            Return $"https://twitter.com/{User.Name}/status/{Media.Post.ID}"
+            Return String.Format(SinglePostPattern, Media.Post.ID)
         End Function
         Friend Overrides Function BaseAuthExists() As Boolean
             Return Responser.CookiesExists
@@ -146,5 +152,18 @@ Namespace API.Twitter
             End If
             MyBase.Update()
         End Sub
+        Friend Const CommunitiesUser As String = "/i/communities"
+        Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
+            Dim e As ExchangeOptions = MyBase.IsMyUser(UserURL)
+            If Not e.UserName.IsEmptyString Then
+                If UserURL.Contains(CommunitiesUser) Then e.Options = CommunitiesUser : e.UserName &= "@c"
+                Return e
+            Else
+                Return Nothing
+            End If
+        End Function
+        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
+            Return DirectCast(User, UserData).GetUserUrl
+        End Function
     End Class
 End Namespace

SCrawler/API/Twitter/UserData.vb

@@ -26,13 +26,30 @@ Namespace API.Twitter
         Private Const Name_GifsDownload As String = "GifsDownload"
         Private Const Name_GifsSpecialFolder As String = "GifsSpecialFolder"
         Private Const Name_GifsPrefix As String = "GifsPrefix"
+        Private Const Name_IsCommunity As String = "IsCommunity"
 #End Region
 #Region "Declarations"
+        Private Const Label_Community As String = "Community"
+        Private _NameTrue As String = String.Empty
+        Friend Property NameTrue As String
+            Get
+                Return _NameTrue.IfNullOrEmpty(Name)
+            End Get
+            Set(ByVal NewName As String)
+                _NameTrue = NewName
+            End Set
+        End Property
+        Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
+            Get
+                Return {Label_Community}
+            End Get
+        End Property
         Friend Enum DownloadModels As Integer
             Undefined = 0
             Media = 1
             Profile = 2
             Search = 5
+            Likes = 10
         End Enum
         Private FirstDownloadComplete As Boolean = False
         Friend Property DownloadModelForceApply As Boolean = False
@@ -41,6 +58,8 @@ Namespace API.Twitter
         Friend Property GifsDownload As Boolean = True
         Friend Property GifsSpecialFolder As String = String.Empty
         Friend Property GifsPrefix As String = String.Empty
+        Friend Property IsCommunity As Boolean = False
+        Private ReadOnly LikesPosts As List(Of String)
         Private ReadOnly _DataNames As List(Of String)
         Private ReadOnly Property MySettings As SiteSettings
             Get
@@ -55,6 +74,9 @@ Namespace API.Twitter
         Private Function RenameGdlFile(ByVal Input As SFile, ByVal i As Integer) As SFile
             Return SFile.Rename(Input, $"{Input.PathWithSeparator}{i.NumToString(FileNameProvider)}.{Input.Extension}",, EDP.ThrowException)
         End Function
+        Friend Function GetUserUrl() As String
+            Return $"https://x.com{IIf(IsCommunity, SiteSettings.CommunitiesUser, String.Empty)}/{NameTrue}"
+        End Function
 #End Region
 #Region "Exchange options"
         Friend Overrides Function ExchangeOptionsGet() As Object
@@ -74,6 +96,7 @@ Namespace API.Twitter
                     If .DownloadModelMedia Then DownloadModel += DownloadModels.Media
                     If .DownloadModelProfile Then DownloadModel += DownloadModels.Profile
                     If .DownloadModelSearch Then DownloadModel += DownloadModels.Search
+                    If .DownloadModelLikes Then DownloadModel += DownloadModels.Likes
                 End With
             End If
         End Sub
@@ -81,6 +104,7 @@ Namespace API.Twitter
 #Region "Initializer, loader"
         Friend Sub New()
             _DataNames = New List(Of String)
+            LikesPosts = New List(Of String)
         End Sub
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
             With Container
@@ -117,7 +141,20 @@ Namespace API.Twitter
                     RemoveExistingDuplicates = .Value(Name_RemoveExistingDuplicates).FromXML(Of Boolean)(False)
                     StartMD5Checked = .Value(Name_StartMD5Checked).FromXML(Of Boolean)(False)
                     MediaModelAllowNonUserTweets = .Value(Name_MediaModelAllowNonUserTweets).FromXML(Of Boolean)(False)
+                    IsCommunity = .Value(Name_IsCommunity).FromXML(Of Boolean)(False)
+                    _NameTrue = .Value(Name_TrueName)
                 Else
+                    If Name.Contains("@") And Not IsCommunity Then
+                        IsCommunity = True
+                        _NameTrue = Name.Split("@")(0)
+                        ID = _NameTrue
+                        ParseUserMediaOnly = False
+                        Labels.ListAddValue(Label_Community, LNC)
+                        Labels.Sort()
+                        .Add(Name_UserID, ID)
+                        .Add(Name_LabelsName, LabelsString)
+                        .Add(Name_ParseUserMediaOnly, ParseUserMediaOnly.BoolToInteger)
+                    End If
                     .Add(Name_FirstDownloadComplete, FirstDownloadComplete.BoolToInteger)
                     .Add(Name_DownloadModelForceApply, DownloadModelForceApply.BoolToInteger)
                     .Add(Name_DownloadModel, CInt(DownloadModel))
@@ -128,6 +165,8 @@ Namespace API.Twitter
                     .Add(Name_RemoveExistingDuplicates, RemoveExistingDuplicates.BoolToInteger)
                     .Add(Name_StartMD5Checked, StartMD5Checked.BoolToInteger)
                     .Add(Name_MediaModelAllowNonUserTweets, MediaModelAllowNonUserTweets.BoolToInteger)
+                    .Add(Name_IsCommunity, IsCommunity.BoolToInteger)
+                    .Add(Name_TrueName, _NameTrue)
                 End If
             End With
         End Sub
@@ -142,21 +181,32 @@ Namespace API.Twitter
             }
         End Function
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
-            If MySettings.LIMIT_ABORT Then
-                Throw New TwitterLimitException(Me)
-            Else
-                If IsSavedPosts Then
-                    If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.Post.ID), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
-                    DownloadData_SavedPosts(Token)
+            Try
+                If MySettings.LIMIT_ABORT Then
+                    Throw New TwitterLimitException(Me)
                 Else
-                    If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
-                    DownloadData_Timeline(Token)
+                    If IsSavedPosts Then
+                        If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.Post.ID), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
+                        DownloadData_SavedPosts(Token)
+                    Else
+                        LikesPosts.Clear()
+                        If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
+                        DownloadData_Timeline(Token)
+                        If LikesPosts.Count > 0 Then
+                            _ReparseLikes = True
+                            ReparseMissing(Token)
+                            _ReparseLikes = False
+                        End If
+                    End If
                 End If
-            End If
+            Finally
+                _ReparseLikes = False
+            End Try
         End Sub
         Private Sub DownloadData_Timeline(ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
             Dim tCache As CacheKeeper = Nothing
+            Dim likesDetected As Boolean = False
             Try
                 Const entry$ = "entry"
                 Dim PostID$ = String.Empty
@@ -173,14 +223,15 @@ Namespace API.Twitter
                 Dim newTwitterNodes() As Object = {0, "content", "items"}
                 Dim p As Predicate(Of EContainer)
                 Dim pIndx%
+                Dim indxChanged As Boolean = False
                 Dim isOneNode As Boolean, isPins As Boolean, ExistsDetected As Boolean, userInfoParsed As Boolean = False
                 Dim j As EContainer, rootNode As EContainer, optionalNode As EContainer, workingNode As EContainer, tmpNode As EContainer, nn As EContainer = Nothing
 
                 Dim __parseContainer As Func(Of EContainer, Boolean) =
                     Function(ByVal ee As EContainer) As Boolean
                         nn = Nothing
-                        If dirIndx > 1 Then nn = ee
-                        If Not nn.ListExists Then
+                        If dirIndx > 1 Or IsCommunity Then nn = ee
+                        If Not nn.ListExists Or IsCommunity Then
                             For Each node In nodes
                                 nn = ee(node)
                                 If nn.ListExists Then Exit For
@@ -199,6 +250,7 @@ Namespace API.Twitter
 
                             If Not _TempPostsList.Contains(PostID) Then
                                 _TempPostsList.Add(PostID)
+                            ElseIf dirIndx = 3 Then
                             ElseIf isPins Then
                                 Return False
                             Else
@@ -211,9 +263,22 @@ Namespace API.Twitter
                             If tmpUserId.IsEmptyString Then tmpUserId = nn.ItemF({"extended_entities", "media", 0, sourceIdPredicate}).XmlIfNothingValue.
                                                                                 IfNullOrEmpty(nn.Value("user_id")).IfNullOrEmpty(nn.Value("user_id_str")).IfNullOrEmpty("/")
 
-                            If Not ParseUserMediaOnly OrElse
+                            If (Not ParseUserMediaOnly Or dirIndx = 3) OrElse
                                (dirIndx = 0 AndAlso MediaModelAllowNonUserTweets) OrElse
-                               (Not ID.IsEmptyString AndAlso tmpUserId = ID) Then ObtainMedia(nn, PostID, PostDate)
+                               (Not ID.IsEmptyString AndAlso tmpUserId = ID) Then
+                                If dirIndx = 3 Then
+                                    Dim lUrl$ = nn.ItemF({"content", "itemContent", "tweet_results", "result", "legacy", "entities", "media", 0}, "expanded_url").XmlIfNothingValue
+                                    If Not lUrl.IsEmptyString Then
+                                        lUrl = RegexReplace(lUrl, StatusRegEx)
+                                        If Not lUrl.IsEmptyString Then
+                                            If Not _TempPostsList.Contains(lUrl) Then _TempPostsList.Add(lUrl) Else Return False
+                                            LikesPosts.ListAddValue(lUrl, LNC)
+                                        End If
+                                    End If
+                                Else
+                                    ObtainMedia(nn, PostID, PostDate)
+                                End If
+                            End If
                         End If
                         Return True
                     End Function
@@ -225,6 +290,8 @@ Namespace API.Twitter
                     For Each dir As SFile In dirs
                         dirIndx += 1
 
+                        If dirIndx = 3 Then likesDetected = True
+
                         ExistsDetected = False
 
                         If Not dir.IsEmptyString Then
@@ -238,10 +305,22 @@ Namespace API.Twitter
                                 For i = 0 To timelineFiles.Count - 1
                                     j = JsonDocument.Parse(timelineFiles(i).GetText)
                                     If Not j Is Nothing Then
-                                        If i = 0 Then
+                                        If i = 0 And Not indxChanged Then
                                             If Not userInfoParsed Then
                                                 userInfoParsed = True
-                                                Dim resValue$ = j.Value({"data", "user", "result"}, "__typename").StringTrim.StringToLower
+                                                Dim resValue$ = j.Value({"data", IIf(IsCommunity, "communityResults", "user"), "result"}, "__typename").StringTrim.StringToLower
+                                                Dim icon$
+                                                Dim __getImage As Action(Of String) = Sub(ByVal img As String)
+                                                                                          If Not img.IsEmptyString Then
+                                                                                              Dim __imgFile As SFile = UrlFile(img, True)
+                                                                                              If Not __imgFile.Name.IsEmptyString Then
+                                                                                                  If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
+                                                                                                  __imgFile.Path = MyFile.CutPath.Path
+                                                                                                  If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
+                                                                                                  If __imgFile.Exists Then IconBannerDownloaded = True
+                                                                                              End If
+                                                                                          End If
+                                                                                      End Sub
                                                 If resValue.IsEmptyString Then
                                                     UserExists = False
                                                     j.Dispose()
@@ -250,6 +329,29 @@ Namespace API.Twitter
                                                     UserSuspended = True
                                                     j.Dispose()
                                                     Exit Sub
+                                                ElseIf IsCommunity Then
+                                                    With j({"data", "communityResults", "result", "community_media_timeline", "timeline", "instructions"})
+                                                        If .ListExists Then
+                                                            With .Find(entriesNode, True)
+                                                                If .ListExists Then
+                                                                    With .ItemF({0, "content", "items", 0, "item", "itemContent", "tweet_results", "result", "tweet", "community_results", "result"})
+                                                                        If .ListExists Then
+                                                                            If ID = .Value("id_str") Then
+                                                                                UserSiteNameUpdate(.Value("name"))
+                                                                                UserDescriptionUpdate(.Value("description"))
+
+                                                                                icon = .Value({"custom_banner_media", "media_info"}, "original_img_url").
+                                                                                       IfNullOrEmpty(.Value({"default_banner_media", "media_info"}, "original_img_url"))
+                                                                                If Not icon.IsEmptyString And DownloadIconBanner Then __getImage.Invoke(icon)
+                                                                            End If
+                                                                        End If
+                                                                    End With
+                                                                End If
+                                                            End With
+                                                        End If
+                                                    End With
+                                                    i = -1
+                                                    indxChanged = True
                                                 Else
                                                     With j({"data", "user", "result"})
                                                         If .ListExists Then
@@ -259,21 +361,11 @@ Namespace API.Twitter
                                                             End If
                                                             With .Item({"legacy"})
                                                                 If .ListExists Then
-                                                                    If .Value("screen_name").StringToLower = Name.ToLower Then
+                                                                    If .Value("screen_name").StringToLower = NameTrue.ToLower Then
                                                                         UserSiteNameUpdate(.Value("name"))
                                                                         UserDescriptionUpdate(.Value("description"))
-                                                                        Dim __getImage As Action(Of String) = Sub(ByVal img As String)
-                                                                                                                  If Not img.IsEmptyString Then
-                                                                                                                      Dim __imgFile As SFile = UrlFile(img, True)
-                                                                                                                      If Not __imgFile.Name.IsEmptyString Then
-                                                                                                                          If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
-                                                                                                                          __imgFile.Path = MyFile.CutPath.Path
-                                                                                                                          If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
-                                                                                                                          If __imgFile.Exists Then IconBannerDownloaded = True
-                                                                                                                      End If
-                                                                                                                  End If
-                                                                                                              End Sub
-                                                                        Dim icon$ = .Value("profile_image_url_https")
+
+                                                                        icon = .Value("profile_image_url_https")
                                                                         If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty)
                                                                         If DownloadIconBanner Then
                                                                             __getImage.Invoke(.Value("profile_banner_url"))
@@ -285,29 +377,55 @@ Namespace API.Twitter
                                                         End If
                                                     End With
                                                 End If
+                                            ElseIf IsCommunity Then
+                                                i = -1
+                                                indxChanged = True
                                             End If
                                         Else
-                                            For pIndx = 0 To IIf(dirIndx < 2, 1, 0)
+                                            For pIndx = 0 To IIf(dirIndx < 2 Or dirIndx = 3, 1, 0)
                                                 optionalNode = Nothing
-                                                Select Case dirIndx
-                                                    Case 0, 1
-                                                        rootNode = j({"data", "user", "result", "timeline_v2", "timeline", "instructions"})
-                                                        If rootNode.ListExists Then
-                                                            p = If(pIndx = 0, pinNode, timelineNode)
-                                                            isPins = pIndx = 0
+                                                rootNode = Nothing
+                                                If IsCommunity Then
+                                                    With j({"data", "communityResults", "result", "community_media_timeline", "timeline", "instructions"})
+                                                        If .ListExists Then
+                                                            If i = 0 Then
+                                                                rootNode = .Find(entriesNode, True)
+                                                            Else
+                                                                rootNode = .Find(moduleItemsPredicate, True)
+                                                            End If
                                                             optionalNode = rootNode
-                                                            rootNode = rootNode.Find(p, False)
-                                                            If rootNode.ListExists Then rootNode = rootNode.Find(entriesNode, False)
                                                         End If
-                                                    Case Else
-                                                        isPins = False
-                                                        rootNode = j({"globalObjects", "tweets"})
-                                                        optionalNode = rootNode
-                                                End Select
+                                                    End With
+                                                Else
+                                                    Select Case dirIndx
+                                                        Case 0, 1, 3
+                                                            rootNode = j({"data", "user", "result", "timeline_v2", "timeline", "instructions"})
+                                                            If rootNode.ListExists Then
+                                                                If dirIndx = 3 Then
+                                                                    p = entriesNode
+                                                                    isPins = False
+                                                                Else
+                                                                    p = If(pIndx = 0, pinNode, timelineNode)
+                                                                    isPins = pIndx = 0
+                                                                End If
+                                                                optionalNode = rootNode
+                                                                rootNode = rootNode.Find(p, dirIndx = 3)
+                                                                If dirIndx <> 3 And rootNode.ListExists Then rootNode = rootNode.Find(entriesNode, dirIndx = 3)
+                                                            End If
+                                                        Case Else
+                                                            isPins = False
+                                                            rootNode = j({"globalObjects", "tweets"})
+                                                            optionalNode = rootNode
+                                                    End Select
+                                                End If
 
                                                 If rootNode.ListExists Then
                                                     With rootNode
-                                                        isOneNode = dirIndx < 2 AndAlso .Name = entry
+                                                        If IsCommunity Then
+                                                            isOneNode = pIndx = 0
+                                                        Else
+                                                            isOneNode = dirIndx < 2 AndAlso .Name = entry
+                                                        End If
                                                         ProgressPre.ChangeMax(If(isOneNode, 1, .Count))
                                                         If isOneNode Then
                                                             ProgressPre.Perform()
@@ -336,7 +454,6 @@ Namespace API.Twitter
                                                 End If
                                             Next
 
-                                            'TODO: Twitter: is this line needed?
                                             If ExistsDetected And i = 1 Then Exit For Else ExistsDetected = False
                                         End If
                                         j.Dispose()
@@ -369,12 +486,12 @@ Namespace API.Twitter
                 ProcessException(ex, Token, $"data downloading error [{URL}]")
             Finally
                 If Not tCache Is Nothing Then tCache.Dispose()
-                If _TempPostsList.Count > 0 Then _TempPostsList.Sort()
+                If _TempPostsList.Count > 0 And Not likesDetected Then _TempPostsList.Sort()
             End Try
         End Sub
         Private Sub DownloadData_SavedPosts(ByVal Token As CancellationToken)
             Try
-                Dim f As SFile = GetDataFromGalleryDL("https://twitter.com/i/bookmarks", Settings.Cache, True, Token)
+                Dim f As SFile = GetDataFromGalleryDL("https://x.com/i/bookmarks", Settings.Cache, True, Token)
                 Dim files As List(Of SFile) = SFile.GetFiles(f, "*.txt")
                 If files.ListExists Then
                     ResetFileNameProvider(Math.Max(files.Count.ToString.Length, 3))
@@ -417,21 +534,24 @@ Namespace API.Twitter
 #End Region
 #Region "Obtain media"
         Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown,
-                                Optional ByVal Attempts As Integer = 0)
+                                Optional ByVal Attempts As Integer = 0, Optional ByVal SpecialFolder As String = Nothing)
             Dim s As EContainer = e({"extended_entities", "media"})
             If If(s?.Count, 0) = 0 Then s = e({"retweeted_status", "extended_entities", "media"})
             If If(s?.Count, 0) = 0 Then s = e({"retweeted_status_result", "result", "legacy", "extended_entities", "media"})
 
             If If(s?.Count, 0) > 0 Then
                 Dim mUrl$
+                Dim media As UserMedia
                 For Each m As EContainer In s
-                    If Not CheckVideoNode(m, PostID, PostDate, State) Then
+                    If Not CheckVideoNode(m, PostID, PostDate, State, SpecialFolder) Then
                         mUrl = m.Value("media_url").IfNullOrEmpty(m.Value("media_url_https"))
                         If Not mUrl.IsEmptyString Then
                             Dim dName$ = UrlFile(mUrl)
                             If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then
                                 _DataNames.Add(dName)
-                                _TempMediaList.ListAddValue(MediaFromData(mUrl, PostID, PostDate, GetPictureOption(m), State, UTypes.Picture, Attempts), LNC)
+                                media = MediaFromData(mUrl, PostID, PostDate, GetPictureOption(m), State, UTypes.Picture, Attempts)
+                                If Not SpecialFolder.IsEmptyString Then media.SpecialFolder = SpecialFolder
+                                _TempMediaList.ListAddValue(media, LNC)
                             End If
                         End If
                     End If
@@ -439,15 +559,17 @@ Namespace API.Twitter
             End If
         End Sub
         Private Function CheckVideoNode(ByVal w As EContainer, ByVal PostID As String, ByVal PostDate As String,
-                                        Optional ByVal State As UStates = UStates.Unknown) As Boolean
+                                        Optional ByVal State As UStates = UStates.Unknown, Optional ByVal SpecialFolder As String = Nothing) As Boolean
             Try
-                If CheckForGif(w, PostID, PostDate, State) Then Return True
+                If CheckForGif(w, PostID, PostDate, State, SpecialFolder) Then Return True
                 Dim URL$ = GetVideoNodeURL(w)
                 If Not URL.IsEmptyString Then
                     Dim f$ = UrlFile(URL)
                     If Not f.IsEmptyString AndAlso Not _DataNames.Contains(f) Then
                         _DataNames.Add(f)
-                        _TempMediaList.ListAddValue(MediaFromData(URL, PostID, PostDate,, State, UTypes.Video), LNC)
+                        Dim m As UserMedia = MediaFromData(URL, PostID, PostDate,, State, UTypes.Video)
+                        If Not SpecialFolder.IsEmptyString Then m.SpecialFolder = SpecialFolder
+                        _TempMediaList.ListAddValue(m, LNC)
                     End If
                     Return True
                 End If
@@ -458,7 +580,7 @@ Namespace API.Twitter
             End Try
         End Function
         Private Function CheckForGif(ByVal w As EContainer, ByVal PostID As String, ByVal PostDate As String,
-                                     Optional ByVal State As UStates = UStates.Unknown) As Boolean
+                                     Optional ByVal State As UStates = UStates.Unknown, Optional ByVal SpecialFolder As String = Nothing) As Boolean
             Try
                 Dim gifUrl As Predicate(Of EContainer) = Function(e) Not e.Value("content_type").IsEmptyString AndAlso
                                                                      e.Value("content_type").Contains("mp4") AndAlso
@@ -477,9 +599,13 @@ Namespace API.Twitter
                                         If Not ff.IsEmptyString Then
                                             If GifsDownload And Not _DataNames.Contains(ff) Then
                                                 m = MediaFromData(url, PostID, PostDate,, State, UTypes.Video)
+                                                If Not SpecialFolder.IsEmptyString Then m.SpecialFolder = SpecialFolder
                                                 f = m.File
                                                 If Not f.IsEmptyString And Not GifsPrefix.IsEmptyString Then f.Name = $"{GifsPrefix}{f.Name}" : m.File = f
-                                                If Not GifsSpecialFolder.IsEmptyString Then m.SpecialFolder = $"{GifsSpecialFolder}*"
+                                                If Not GifsSpecialFolder.IsEmptyString Then
+                                                    If Not m.SpecialFolder.IsEmptyString Then m.SpecialFolder &= "\"
+                                                    m.SpecialFolder &= $"{GifsSpecialFolder}*"
+                                                End If
                                                 _TempMediaList.ListAddValue(m, LNC)
                                             End If
                                             Return True
@@ -615,6 +741,7 @@ Namespace API.Twitter
                 Dim dir As SFile
                 Dim dm As List(Of DownloadModels) = EnumExtract(Of DownloadModels)(DownloadModel).ListIfNothing
                 Dim process As Boolean
+                Dim urlPrePattern$ = $"https://x.com{IIf(IsCommunity, SiteSettings.CommunitiesUser, String.Empty)}/"
 
                 Using tgdl As New TwitterGDL(Nothing, Token, MySettings.AbortOnLimit.Value) With {
                     .TempPostsList = _TempPostsList,
@@ -625,7 +752,7 @@ Namespace API.Twitter
                 }
                     tgdl.FileExchanger.DeleteCacheOnDispose = False
                     tgdl.FileExchanger.DeleteRootOnDispose = False
-                    For i As Byte = 0 To 2
+                    For i As Byte = 0 To IIf(IsCommunity, 0, 3)
                         dir = rootDir.NewPath
                         dir.Exists(SFO.Path, True, EDP.ThrowException)
                         outList.Add(dir)
@@ -633,9 +760,10 @@ Namespace API.Twitter
                         command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
                         command &= GdlGetIdFilterString()
                         Select Case i
-                            Case 0 : command &= $"https://twitter.com/{Name}/media" : process = dm.Contains(DownloadModels.Media)
-                            Case 1 : command &= $"https://twitter.com/{Name}" : process = dm.Contains(DownloadModels.Profile)
-                            Case 2 : command &= $"-o search-endpoint=graphql https://twitter.com/search?q=from:{Name}+include:nativeretweets" : process = dm.Contains(DownloadModels.Search)
+                            Case 0 : command &= $"{urlPrePattern}{NameTrue}/media" : process = dm.Contains(DownloadModels.Media) Or IsCommunity
+                            Case 1 : command &= $"{urlPrePattern}{NameTrue}" : process = dm.Contains(DownloadModels.Profile)
+                            Case 2 : command &= $"-o search-endpoint=graphql https://x.com/search?q=from:{NameTrue}+include:nativeretweets" : process = dm.Contains(DownloadModels.Search) And Not IsCommunity
+                            Case 3 : command &= $"{urlPrePattern}{NameTrue}/likes" : process = dm.Contains(DownloadModels.Likes)
                             Case Else : process = False
                         End Select
                         '#If DEBUG Then
@@ -687,13 +815,13 @@ Namespace API.Twitter
         End Function
 #End Region
 #Region "ReparseMissing"
+        Private _ReparseLikes As Boolean = False
         Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
-            Const SinglePostPattern$ = "https://twitter.com/{0}/status/{1}"
             Dim rList As New List(Of Integer)
             Dim URL$ = String.Empty
             Dim cache As CacheKeeper = Nothing
             Try
-                If ContentMissingExists Then
+                If ContentMissingExists Or (_ReparseLikes And LikesPosts.Count > 0) Then
                     Dim m As UserMedia
                     Dim PostDate$
                     Dim nodes As List(Of String()) = GetContainerSubnodes()
@@ -702,24 +830,24 @@ Namespace API.Twitter
                     Dim f As SFile
                     Dim i%, ii%
                     Dim files As List(Of SFile)
+                    Dim lim%
+                    Dim specFolder$ = IIf(_ReparseLikes, "Likes", String.Empty)
                     ResetFileNameProvider()
-                    If IsSingleObjectDownload Then
-                        cache = Settings.Cache
-                    Else
-                        cache = New CacheKeeper(DownloadContentDefault_GetRootDir.CSFilePS)
-                        cache.CacheDeleteError = CacheDeletionError(cache)
-                    End If
-                    ProgressPre.ChangeMax(_ContentList.Count)
-                    For i = 0 To _ContentList.Count - 1
+                    cache = If(IsSingleObjectDownload, Settings.Cache, CreateCache())
+                    If _ReparseLikes Then lim = LikesPosts.Count Else lim = _ContentList.Count
+                    ProgressPre.ChangeMax(lim)
+                    For i = 0 To lim - 1
                         ProgressPre.Perform()
-                        If _ContentList(i).State = UStates.Missing Then
-                            m = _ContentList(i)
-                            If Not m.Post.ID.IsEmptyString Or (IsSingleObjectDownload And Not m.URL_BASE.IsEmptyString) Then
+                        If _ReparseLikes OrElse _ContentList(i).State = UStates.Missing Then
+                            m = If(_ReparseLikes, Nothing, _ContentList(i))
+                            If Not m.Post.ID.IsEmptyString Or (IsSingleObjectDownload And Not m.URL_BASE.IsEmptyString) Or _ReparseLikes Then
                                 ThrowAny(Token)
                                 If IsSingleObjectDownload Then
                                     URL = m.URL_BASE
+                                ElseIf _ReparseLikes Then
+                                    URL = LikesPosts(i)
                                 Else
-                                    URL = String.Format(SinglePostPattern, Name, m.Post.ID)
+                                    URL = String.Format(SiteSettings.SinglePostPattern, m.Post.ID)
                                 End If
                                 f = GetDataFromGalleryDL(URL, cache, False, Token)
                                 If Not f.IsEmptyString Then
@@ -737,7 +865,7 @@ Namespace API.Twitter
                                                                     If .ListExists Then
                                                                         PostDate = String.Empty
                                                                         If .Contains("created_at") Then PostDate = .Value("created_at") Else PostDate = String.Empty
-                                                                        ObtainMedia(.Self, m.Post.ID, PostDate, UStates.Missing, m.Attempts)
+                                                                        ObtainMedia(.Self, m.Post.ID, PostDate, UStates.Missing, m.Attempts, specFolder)
                                                                         rList.ListAddValue(i, LNC)
                                                                     End If
                                                                 End With
@@ -759,7 +887,7 @@ Namespace API.Twitter
                 ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
             Finally
                 If Not cache Is Nothing And Not IsSingleObjectDownload Then cache.Dispose()
-                If rList.Count > 0 Then
+                If rList.Count > 0 And Not _ReparseLikes Then
                     For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
                     rList.Clear()
                 End If
@@ -856,7 +984,7 @@ Namespace API.Twitter
 #End Region
 #Region "IDisposable support"
         Protected Overrides Sub Dispose(ByVal disposing As Boolean)
-            If Not disposedValue And disposing Then _DataNames.Clear()
+            If Not disposedValue And disposing Then _DataNames.Clear() : LikesPosts.Clear()
             MyBase.Dispose(disposing)
         End Sub
 #End Region

SCrawler/API/UserDataBind.vb

@@ -558,7 +558,7 @@ Namespace API
         End Sub
 #End Region
 #Region "Move, Merge"
-        Friend Overrides Function MoveFiles(ByVal __CollectionName As String, ByVal __SpecialCollectionPath As SFile) As Boolean
+        Friend Overrides Function MoveFiles(ByVal __CollectionName As String, ByVal __SpecialCollectionPath As SFile, Optional ByVal NewUser As SplitCollectionUserInfo? = Nothing) As Boolean
             Throw New NotImplementedException("Move files is not available in the collection context")
         End Function
         Friend Overloads Sub MergeData(ByVal Merging As Boolean)
@@ -601,7 +601,19 @@ Namespace API
                         "Operation canceled", MsgBoxStyle.Critical)
                 Return False
             Else
-                _Item.MoveFiles(String.Empty, Nothing)
+                Dim uObj As SplitCollectionUserInfo? = DirectCast(_Item, UserDataBase).SplitCollectionGetNewUserInfo
+                If uObj.Value.SameDrive Then
+                    uObj = Nothing
+                Else
+                    Using f As New SplitCollectionUserInfoChangePathsForm({uObj})
+                        f.ShowDialog()
+                        Select Case f.DialogResult
+                            Case DialogResult.OK : If f.Users(0).Changed Then uObj = f.Users(0) Else uObj = Nothing
+                            Case DialogResult.Abort : Return False
+                        End Select
+                    End Using
+                End If
+                _Item.MoveFiles(String.Empty, Nothing, uObj)
                 MainFrameObj.ImageHandler(_Item)
                 AddRemoveBttDeleteHandler(_Item, False)
                 RaiseEvent OnUserRemoved(_Item)
@@ -618,7 +630,7 @@ Namespace API
                 End If
                 Dim m As New MMessage($"Collection [{CollectionName} (number of profiles: {Count})] may contain data" & vbCr &
                                       "Are you sure you want to delete the collection and all of its files?", MsgTitle,
-                                      {New MsgBoxButton("Delete") With {.ToolTip = "Delete the collection and all files", .KeyCode = Keys.Enter},
+                                      {New MsgBoxButton("Delete", "Delete the collection and all files") With {.KeyCode = Keys.Enter},
                                        New MsgBoxButton("Split") With {
                                         .ToolTip = "Users will be removed from the collection and will be displayed in the program as separate users." & vbCr &
                                                    "All user data will remain.",
@@ -653,12 +665,31 @@ Namespace API
                             MsgBoxE({$"Collection [{CollectionName}] data merged{vbCr}Unable to split merged collection{vbCr}Operation canceled", MsgTitle}, vbExclamation)
                             Return 0
                         Else
-                            Collections.ForEach(Sub(ByVal c As IUserData)
-                                                    If c.MoveFiles(String.Empty, Nothing) Then
-                                                        UserListLoader.UpdateUser(Settings.GetUser(c), True)
-                                                        MainFrameObj.ImageHandler(c)
-                                                    End If
-                                                End Sub)
+                            Dim uu As New List(Of SplitCollectionUserInfo)(Collections.Select(Function(uuu As UserDataBase) uuu.SplitCollectionGetNewUserInfo))
+                            If uu.All(Function(uuu) uuu.SameDrive) Then
+                                uu.Clear()
+                            Else
+                                Using colPaths As New SplitCollectionUserInfoChangePathsForm(uu)
+                                    colPaths.ShowDialog()
+                                    Select Case colPaths.DialogResult
+                                        Case DialogResult.OK
+                                            If colPaths.Users.Any(Function(uuu) uuu.Changed) Then
+                                                uu = New List(Of SplitCollectionUserInfo)(colPaths.Users)
+                                            Else
+                                                uu.Clear()
+                                            End If
+                                        Case DialogResult.Abort : Return 0
+                                    End Select
+                                End Using
+                            End If
+                            Collections.ListForEach(Sub(ByVal c As IUserData, ByVal indx As Integer)
+                                                        Dim uObj As SplitCollectionUserInfo? = Nothing
+                                                        If uu.Count > 0 AndAlso indx.ValueBetween(0, uu.Count - 1) AndAlso uu(indx).Changed Then uObj = uu(indx)
+                                                        If c.MoveFiles(String.Empty, Nothing, uObj) Then
+                                                            UserListLoader.UpdateUser(Settings.GetUser(c), True)
+                                                            MainFrameObj.ImageHandler(c)
+                                                        End If
+                                                    End Sub)
                             If Collections.All(Function(c) c.CollectionName.IsEmptyString) Then
                                 Settings.Users.Remove(Me)
                                 Collections.Clear()

SCrawler/API/Xhamster/UserData.vb

@@ -265,7 +265,12 @@ Namespace API.Xhamster
                 Dim checkLimit As Func(Of Boolean) = Function() limit > 0 And SearchPostsCount >= limit And IsVideo
 
                 If IsSavedPosts Then
-                    containerNodes.Add(If(IsVideo, {"favoriteVideoListComponent", "models"}, {"favoritesGalleriesAndPhotosCollection"}))
+                    If IsVideo Then
+                        containerNodes.Add({"favoriteVideoListComponent", "models"})
+                        containerNodes.Add({"favoriteVideoListComponent", "videoThumbProps"})
+                    Else
+                        containerNodes.Add({"favoritesGalleriesAndPhotosCollection"})
+                    End If
                 ElseIf Not SiteMode = SiteModes.Search Then
                     If IsVideo Then
                         containerNodes.Add({"trendingVideoListComponent", "models"})
@@ -280,32 +285,18 @@ Namespace API.Xhamster
                     End If
                 End If
 
-                'TODELETE: xHamster remove old container nodes attachments 
-
                 If IsSavedPosts Then
                     URL = $"https://xhamster.com/my/favorites/{IIf(IsVideo, "videos", "photos-and-galleries")}{IIf(Page = 1, String.Empty, $"/{Page}")}"
                     containerNodes.Add(If(IsVideo, {"favoriteVideoListComponent", "models"}, {"favoritesGalleriesAndPhotosCollection"}))
                 ElseIf IsChannel Then
                     URL = $"https://xhamster.com/channels/{TrueName}/newest{IIf(Page = 1, String.Empty, $"/{Page}")}"
-                    'containerNodes.Add({"trendingVideoListComponent", "models"})
-                    'containerNodes.Add({"pagesCategoryComponent", "trendingVideoListProps", "models"})
                 ElseIf SiteMode = SiteModes.Search Then
                     URL = GetNonUserUrl(Page)
                     containerNodes.Add({"searchResult", "models"})
                 ElseIf IsCreator Or SiteMode = SiteModes.Tags Or SiteMode = SiteModes.Categories Or SiteMode = SiteModes.Pornstars Then
                     URL = GetNonUserUrl(Page)
-                    'If SiteMode = SiteModes.Pornstars Then
-                    '    containerNodes.Add({"trendingVideoListComponent", "models"})
-                    '    containerNodes.Add({"pagesCategoryComponent", "trendingVideoListProps", "models"})
-                    'Else
-                    '    containerNodes.Add({"pagesCategoryComponent", "trendingVideoListProps", "models"})
-                    '    containerNodes.Add({"trendingVideoListComponent", "models"})
-                    'End If
-                    'containerNodes.Add({"trendingVideoSectionComponent", "videoModels"})
                 Else
                     URL = $"https://xhamster.com/users/{TrueName}/{IIf(IsVideo, "videos", "photos")}{IIf(Page = 1, String.Empty, $"/{Page}")}"
-                    'containerNodes.Add({If(IsVideo, "userVideoCollection", "userGalleriesCollection")})
-                    'containerNodes.Add(If(IsVideo, {"videoListComponent", "models"}, {"userGalleriesCollection"}))
                 End If
                 ThrowAny(Token)
 

SCrawler/API/YouTube/SiteSettings.vb

@@ -59,10 +59,10 @@ Namespace API.YouTube
                             .Cookies.Clear()
                             .Cookies.AddRange(Responser.Cookies)
                             .CookiesUpdated = True
-                            .PerformUpdate()
                         End With
                     End If
                 End With
+                DirectCast(MyYouTubeSettings, YTSettings_Internal).PerformUpdate()
             End If
             MyBase.Update()
         End Sub

SCrawler/API/YouTube/UserData.vb

@@ -467,6 +467,13 @@ Namespace API.YouTube
             _TempMediaList.Add(New UserMedia(Data))
         End Sub
 #End Region
+#Region "EraseData"
+        Protected Overrides Sub EraseData_AdditionalDataFiles()
+            LastDownloadDateVideos = Nothing
+            LastDownloadDateShorts = Nothing
+            LastDownloadDatePlaylist = Nothing
+        End Sub
+#End Region
 #Region "DownloadingException"
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer

SCrawler/Download/ActiveDownloadingProgress.vb

@@ -58,7 +58,6 @@ Namespace DownloadObjects
                 End If
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "Change 'ActiveDownloadingProgress' size")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
         Private Sub Downloader_Reconfigured()

SCrawler/Download/Automation/AutoDownloader.vb

@@ -16,11 +16,13 @@ Imports PersonalUtilities.Tools.Notifications
 Namespace DownloadObjects
     Friend Class AutoDownloader : Inherits GroupParameters : Implements IIndexable, IEContainerProvider, IComparable(Of AutoDownloader)
         Friend Event PauseChanged(ByVal Value As PauseModes)
+        Friend Event PlanChanged As Scheduler.PlanChangedEventHandler
         Friend Enum Modes As Integer
             None = 0
             Specified = 3
             Groups = 4
         End Enum
+        Friend Const NoPauseMode As Integer = -100
         Friend Enum PauseModes As Integer
             Disabled = -2
             Enabled = -1
@@ -198,6 +200,52 @@ Namespace DownloadObjects
         Private Const Name_ShowSimpleNotification As String = "ShowSimpleNotification"
 #End Region
 #Region "Declarations"
+        Private _LVIState As Integer = 0
+        Friend Function LVIStateChanged() As Boolean
+            Dim ____LVIState%
+            Dim result As Boolean
+            If Downloading Then
+                ____LVIState = 1
+            ElseIf DownloadReady(True, True) Or ForceStartRequested Then
+                ____LVIState = 2
+            ElseIf Not Working And Not IsManual Then
+                ____LVIState = 3
+            ElseIf IsPaused Then
+                ____LVIState = 4
+            ElseIf Settings.AutomationBrushUndownloadedPlansMinutes.Value > 0 AndAlso
+                   LastDownloadDate.AddMinutes(Settings.AutomationBrushUndownloadedPlansMinutes.Value) < Now Then
+                ____LVIState = 5
+            Else
+                ____LVIState = 0
+            End If
+            result = Not _LVIState = ____LVIState
+            _LVIState = ____LVIState
+            Return result
+        End Function
+        Friend ReadOnly Property LVI As ListViewItem
+            Get
+                Dim l As New ListViewItem(ToString)
+                If Downloading Then
+                    l.BackColor = MyColor.OkBack
+                    l.ForeColor = MyColor.OkFore
+                ElseIf DownloadReady(True, True) Or ForceStartRequested Then
+                    l.BackColor = MyColor.EditBack
+                    l.ForeColor = MyColor.EditFore
+                ElseIf Not Working And Not IsManual Then
+                    l.BackColor = MyColor.DeleteBack
+                    l.ForeColor = MyColor.DeleteFore
+                ElseIf Working And IsPaused And Not IsManual Then
+                    l.BackColor = MyColor.UpdateBack
+                    l.ForeColor = MyColor.UpdateFore
+                ElseIf Settings.AutomationBrushUndownloadedPlansMinutes.Value > 0 AndAlso
+                       LastDownloadDate.AddMinutes(Settings.AutomationBrushUndownloadedPlansMinutes.Value) < Now Then
+                    l.BackColor = Color.FromArgb(224, 224, 224)
+                    l.ForeColor = Color.FromArgb(64, 64, 64)
+                End If
+                LVIStateChanged()
+                Return l
+            End Get
+        End Property
         Friend Property Source As Scheduler
         Private _Mode As Modes = Modes.None
         Friend Property Mode As Modes
@@ -266,11 +314,15 @@ Namespace DownloadObjects
         End Property
         Private Function GetWorkingState() As String
             Dim OutStr$
-            If Working Then
-                If StartupDelay > 0 And _StartTime.AddMinutes(StartupDelay) > Now Then
-                    OutStr = $"delayed until {_StartTime.AddMinutes(StartupDelay).ToStringDate(ADateTime.Formats.BaseDateTime)}"
+            If Working And Not IsManual Then
+                If _Downloading Then
+                    OutStr = "downloading"
+                ElseIf _ForceStartRequested Then
+                    OutStr = "force start pending"
                 ElseIf _StopRequested Then
                     OutStr = "stopping"
+                ElseIf StartupDelay > 0 And _StartTime.AddMinutes(StartupDelay) > Now Then
+                    OutStr = $"delayed ({StartupDelay}) until {_StartTime.AddMinutes(StartupDelay).ToStringDate(ADateTime.Formats.BaseDateTime)}"
                 Else
                     OutStr = "working"
                 End If
@@ -281,14 +333,26 @@ Namespace DownloadObjects
                         Case Else : OutStr &= $", paused ([{Pause}] until {AConvert(Of String)(_PauseValue, ADateTime.Formats.BaseDateTime, "?")})"
                     End Select
                 End If
+            ElseIf IsManual Then
+                If _Downloading Then
+                    OutStr = "downloading"
+                ElseIf _ForceStartRequested Then
+                    OutStr = "force start pending"
+                Else
+                    OutStr = String.Empty
+                End If
             Else
                 OutStr = "stopped"
             End If
             Return OutStr
         End Function
+        Private Function GetWorkingState_Manual() As String
+            Dim OutStr$ = GetWorkingState()
+            If Not OutStr.IsEmptyString Then Return $" ({OutStr})" Else Return String.Empty
+        End Function
         Public Overrides Function ToString() As String
             If IsManual Then
-                Return $"{Name} (manual): last download date: {GetLastDateString()}"
+                Return $"{Name} (manual{GetWorkingState_Manual()}): last download date: {GetLastDateString()}"
             Else
                 Return $"{Name} ({GetWorkingState()}): last download date: {GetLastDateString()}; next run: {GetNextDateString()}"
             End If
@@ -315,9 +379,7 @@ Namespace DownloadObjects
         Friend Sub New(ByVal x As EContainer)
             Me.New
             Initialization = True
-            Dim m% = x.Value(Name_Mode).FromXML(Of Integer)(Modes.None)
-            If m = 1 Or m = 2 Then m = Modes.Specified
-            Mode = m
+            Mode = x.Value(Name_Mode).FromXML(Of Integer)(Modes.None)
             Import(x)
             If Name.IsEmptyString Then Name = "Default"
             Groups.ListAddList(x.Value(Name_Groups).StringToList(Of String)("|"), LAP.NotContainsOnly)
@@ -394,10 +456,10 @@ Namespace DownloadObjects
         End Function
 #End Region
 #Region "Execution"
-        Private AThread As Thread
+        Private _Working As Boolean = False
         Friend ReadOnly Property Working As Boolean
             Get
-                Return If(AThread?.IsAlive, False)
+                Return _Working
             End Get
         End Property
         Private _StartTime As Date = Now
@@ -405,11 +467,8 @@ Namespace DownloadObjects
             If Not IsManual Or Force Then
                 If Init Then _StartTime = Now
                 _IsNewPlan = False
-                If Not Working And Not Mode = Modes.None Then
-                    AThread = New Thread(New ThreadStart(AddressOf Checker))
-                    AThread.SetApartmentState(ApartmentState.MTA)
-                    AThread.Start()
-                End If
+                If Not Working And Not Mode = Modes.None Then _Working = True
+                RaiseEvent PlanChanged(Me)
             End If
         End Sub
         Private _StopRequested As Boolean = False
@@ -433,6 +492,7 @@ Namespace DownloadObjects
                     Case Else : _PauseValue = Nothing
                 End Select
                 RaiseEvent PauseChanged(p)
+                RaiseEvent PlanChanged(Me)
             End Set
         End Property
         Private ReadOnly Property IsPaused As Boolean
@@ -456,7 +516,10 @@ Namespace DownloadObjects
             End Get
         End Property
         Friend Sub [Stop]()
-            If Working Then _StopRequested = True
+            If Working Then
+                If Downloading Then _StopRequested = True Else _Working = False
+                RaiseEvent PlanChanged(Me)
+            End If
         End Sub
         Friend Overloads Sub Skip()
             If LastDownloadDate.AddMinutes(Timer) <= Now Then
@@ -464,65 +527,59 @@ Namespace DownloadObjects
             Else
                 _LastDownloadDateSkip = LastDownloadDate.AddMinutes(Timer)
             End If
+            RaiseEvent PlanChanged(Me)
         End Sub
         Friend Overloads Sub Skip(ByVal Minutes As Integer)
             _LastDownloadDateSkip = If(_LastDownloadDateSkip, Now).AddMinutes(Minutes)
+            RaiseEvent PlanChanged(Me)
         End Sub
         Friend Overloads Sub Skip(ByVal ToDate As Date)
             _LastDownloadDateSkip = ToDate
+            RaiseEvent PlanChanged(Me)
         End Sub
         Friend Sub SkipReset()
             _LastDownloadDateSkip = Nothing
+            RaiseEvent PlanChanged(Me)
         End Sub
         Friend Sub ForceStart()
             _ForceStartRequested = True
             If IsManual Then Start(False, True)
+            RaiseEvent PlanChanged(Me)
         End Sub
         Private _ForceStartRequested As Boolean = False
-        Private _SpecialDelayUse As Boolean = False
-        Private _SpecialDelayTime As Date? = Nothing
-        Private Sub Checker()
-            Try
-                Dim _StartDownload As Boolean
-                While (Not _StopRequested Or Downloader.Working) And Not Mode = Modes.None
-                    If ((IsManual And _ForceStartRequested) Or (NextExecutionDate < Now And Not IsPaused) Or _ForceStartRequested) And
-                       Not _StopRequested And Not Mode = Modes.None Then
-                        If Downloader.Working Then
-                            _SpecialDelayUse = True
-                        Else
-                            If _SpecialDelayUse And Not _SpecialDelayTime.HasValue Then _SpecialDelayTime = Now.AddSeconds(10)
-                            If Not _SpecialDelayUse OrElse (_SpecialDelayTime.HasValue AndAlso _SpecialDelayTime.Value < Now) Then
-                                _SpecialDelayUse = False
-                                _SpecialDelayTime = Nothing
-                                _StartDownload = False
-                                If Settings.Automation.Count = 1 Or _ForceStartRequested Or Index = -1 Then
-                                    _StartDownload = True
-                                Else
-                                    _StartDownload = NextExecutionDate.AddMilliseconds(1000 * (Index + 1)).Ticks <= Now.Ticks
-                                End If
-                                If _StartDownload Then
-                                    Download()
-                                    If IsManual Then Exit While
-                                End If
-                            End If
-                        End If
+        Friend ReadOnly Property ForceStartRequested As Boolean
+            Get
+                Return _ForceStartRequested
+            End Get
+        End Property
+        Friend ReadOnly Property DownloadReady(Optional ByVal IgnorePause As Boolean = False, Optional ByVal IgnoreDownloaderWorking As Boolean = False) As Boolean
+            Get
+                If _StopRequested Then _Working = False
+                Return (Working Or IsManual) And ((IsManual And _ForceStartRequested) Or (Not IsManual And NextExecutionDate < Now And (Not IsPaused Or IgnorePause)) Or _ForceStartRequested) And
+                       Not _StopRequested And Not Mode = Modes.None And (Not Downloader.Working Or IgnoreDownloaderWorking)
+            End Get
+        End Property
+        Friend ReadOnly Property NextDate As Date?
+            Get
+                If Not _StopRequested And Not Mode = Modes.None Then
+                    If IsManual Or _ForceStartRequested Then
+                        Return Now.AddYears(-10)
+                    ElseIf Not IsPaused And Not IsManual And Working Then
+                        Return NextExecutionDate
                     End If
-                    Thread.Sleep(500)
-                End While
-            Catch ex As Exception
-                ErrorsDescriber.Execute(EDP.SendToLog, ex, "[AutoDownloader.Checker]")
-            Finally
-                _StopRequested = False
-            End Try
-        End Sub
+                End If
+                Return Nothing
+            End Get
+        End Property
         Private _Downloading As Boolean = False
         Friend ReadOnly Property Downloading As Boolean
             Get
                 Return _Downloading
             End Get
         End Property
-        Private Sub Download()
+        Friend Sub Download()
             _Downloading = True
+            RaiseEvent PlanChanged(Me)
             Dim Keys As New List(Of String)
             Try
                 Dim users As New List(Of IUserData)
@@ -572,11 +629,23 @@ Namespace DownloadObjects
                     With Downloader
                         .AutoDownloaderWorking = True
                         If .Downloaded.Count > 0 Then .Downloaded.RemoveAll(Function(u) Keys.Contains(u.Key)) : .InvokeDownloadsChangeEvent()
-                        Do : Try : doRound += 1 : .AddRange(users, True) : Exit Do : Catch iex As IndexOutOfRangeException : Thread.Sleep(200) : End Try : Loop While doRound < doLim
+                        Do
+                            Try
+                                doRound += 1
+                                .AddRange(users, True)
+                                Exit Do
+                            Catch iex As Exception
+                                If doRound = doLim Then
+                                    Throw iex
+                                Else
+                                    Thread.Sleep(200)
+                                End If
+                            End Try
+                        Loop While doRound <= doLim
                         While .Working Or .Count > 0 : notify.Invoke() : Thread.Sleep(200) : End While
                         .AutoDownloaderWorking = False
                         notify.Invoke
-                        If simple And DownloadedUsersCount + DownloadedSubscriptionsCount > 0 Then
+                        If simple And (DownloadedUsersCount + DownloadedSubscriptionsCount) > 0 Then
                             Dim msg$ = String.Empty
                             If DownloadedUsersCount > 0 Then msg = $"{DownloadedUsersCount} user(s) "
                             If DownloadedSubscriptionsCount > 0 Then msg &= $"{IIf(DownloadedUsersCount > 0, "and ", String.Empty)}{DownloadedSubscriptionsCount} subscription(s) "
@@ -586,7 +655,7 @@ Namespace DownloadObjects
                     End With
                 End If
             Catch ex As Exception
-                ErrorsDescriber.Execute(EDP.SendToLog, ex, "[AutoDownloader.Download]")
+                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"[AutoDownloader.Download({Name})]")
             Finally
                 Keys.Clear()
                 LastDownloadDate = Now
@@ -594,6 +663,8 @@ Namespace DownloadObjects
                 _Downloading = False
                 _ForceStartRequested = False
                 _LastDownloadDateSkip = Nothing
+                If _StopRequested Then _Working = False
+                RaiseEvent PlanChanged(Me)
             End Try
         End Sub
         Private Sub ShowNotification(ByVal u As IUserData)

SCrawler/Download/Automation/AutoDownloaderEditorForm.Designer.vb

@@ -28,9 +28,10 @@ Namespace DownloadObjects
             Dim ActionButton1 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(AutoDownloaderEditorForm))
             Dim ActionButton2 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim TP_NOTIFY As System.Windows.Forms.TableLayoutPanel
             Dim ActionButton3 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim TP_NOTIFY As System.Windows.Forms.TableLayoutPanel
             Dim ActionButton4 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton5 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TT_MAIN As System.Windows.Forms.ToolTip
             Me.DEF_GROUP = New SCrawler.DownloadObjects.Groups.GroupDefaults()
             Me.OPT_SPEC = New System.Windows.Forms.RadioButton()
@@ -178,9 +179,14 @@ Namespace DownloadObjects
             ActionButton1.BackgroundImage = CType(resources.GetObject("ActionButton1.BackgroundImage"), System.Drawing.Image)
             ActionButton1.Name = "Edit"
             ActionButton2.BackgroundImage = CType(resources.GetObject("ActionButton2.BackgroundImage"), System.Drawing.Image)
-            ActionButton2.Name = "Clear"
+            ActionButton2.Name = "Info"
+            ActionButton2.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Info
+            ActionButton2.ToolTipText = "Open group"
+            ActionButton3.BackgroundImage = CType(resources.GetObject("ActionButton3.BackgroundImage"), System.Drawing.Image)
+            ActionButton3.Name = "Clear"
             Me.TXT_GROUPS.Buttons.Add(ActionButton1)
             Me.TXT_GROUPS.Buttons.Add(ActionButton2)
+            Me.TXT_GROUPS.Buttons.Add(ActionButton3)
             Me.TXT_GROUPS.CaptionText = "Groups"
             Me.TXT_GROUPS.CaptionWidth = 50.0R
             Me.TXT_GROUPS.Dock = System.Windows.Forms.DockStyle.Fill
@@ -260,9 +266,9 @@ Namespace DownloadObjects
             '
             'TXT_TIMER
             '
-            ActionButton3.BackgroundImage = CType(resources.GetObject("ActionButton3.BackgroundImage"), System.Drawing.Image)
-            ActionButton3.Name = "Refresh"
-            Me.TXT_TIMER.Buttons.Add(ActionButton3)
+            ActionButton4.BackgroundImage = CType(resources.GetObject("ActionButton4.BackgroundImage"), System.Drawing.Image)
+            ActionButton4.Name = "Refresh"
+            Me.TXT_TIMER.Buttons.Add(ActionButton4)
             Me.TXT_TIMER.CaptionText = "Timer"
             Me.TXT_TIMER.CaptionToolTipEnabled = True
             Me.TXT_TIMER.CaptionToolTipText = "Timer (in minutes)"
@@ -275,9 +281,9 @@ Namespace DownloadObjects
             '
             'NUM_DELAY
             '
-            ActionButton4.BackgroundImage = CType(resources.GetObject("ActionButton4.BackgroundImage"), System.Drawing.Image)
-            ActionButton4.Name = "Refresh"
-            Me.NUM_DELAY.Buttons.Add(ActionButton4)
+            ActionButton5.BackgroundImage = CType(resources.GetObject("ActionButton5.BackgroundImage"), System.Drawing.Image)
+            ActionButton5.Name = "Refresh"
+            Me.NUM_DELAY.Buttons.Add(ActionButton5)
             Me.NUM_DELAY.CaptionText = "Delay"
             Me.NUM_DELAY.CaptionToolTipEnabled = True
             Me.NUM_DELAY.CaptionToolTipText = "Startup delay"

SCrawler/Download/Automation/AutoDownloaderEditorForm.resx

@@ -189,6 +189,18 @@
 </value>
   </data>
   <data name="ActionButton2.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAAIGNIUk0AAHolAACAgwAA+f8AAIDpAAB1
+        MAAA6mAAADqYAAAXb5JfxUYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAFQSURBVFhH7ZfNDcIwFIMZoXcm
+        YBtGYRHECIgTR1ZhBsS9YoJgQ1Poi5sfqhIOWPqkqvV7dWlI0oVzriry5Dd5HSS0PFwasAEn0AJn4Dle
+        o6fpykaVHYDNwB7YG6ZgzWiQrABosAbqaXNh7bprN1AyAAp3b42msuva9ooGYIFpELA931D2FI+VxzAI
+        gTIdAEb+7KpBz+p4RclQyifoXwdKwgAwcMAl3/mEAOz9GJgokQGyR/sHr8CzlwFwgU+vCuagUQE4gSjz
+        HGxUAM5iyiyxUp4IJ5QEAYomHCvlidCiJAigjKNYKU8M6B/g9wJUH4TV/4ZFE5GV8kSQE1HRVGylPBHC
+        qbh0MbJSnhH0YtQFyFqOiZXyCOLLMQVDckNCrJRHEN+QeMGY3JJZKY8hb0vmxQLTYAplm1IvFNbblnuh
+        Qb0Pk3exGZjv06wW8uT3cIs7jQnSONrSxH0AAAAASUVORK5CYII=
+</value>
+  </data>
+  <data name="ActionButton3.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -203,22 +215,6 @@
     <value>Show a simple notification instead of a user notification.
 This means that if any user data has been downloaded with the plan, a simple notification will be shown with the number of users downloaded.
 The 'Image' and 'User icon' parameters will be ignored.</value>
-  </data>
-  <data name="ActionButton3.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
-        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
-        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
-        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
-        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
-        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
-        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
-        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
-        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
-        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
-        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
-        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
-</value>
   </data>
   <data name="ActionButton4.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
@@ -234,6 +230,22 @@ The 'Image' and 'User icon' parameters will be ignored.</value>
         HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
         qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
         VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
+</value>
+  </data>
+  <data name="ActionButton5.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
+        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
+        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
+        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
+        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
+        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
+        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
+        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
+        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
+        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
+        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
+        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
 </value>
   </data>
 </root>

SCrawler/Download/Automation/AutoDownloaderEditorForm.vb

@@ -100,7 +100,7 @@ Namespace DownloadObjects
                             users.ListAddList(DownloadGroup.GetUsers(g))
                         End Using
                     End If
-                    GroupUsersViewer.Show(users)
+                    GroupUsersViewer.Show(users, $"S {DEF_GROUP.TXT_NAME.Text}")
                     users.Clear()
                 End If
             Catch ex As Exception
@@ -134,11 +134,26 @@ Namespace DownloadObjects
         Private Sub TXT_GROUPS_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles TXT_GROUPS.ActionOnButtonClick
             Select Case Sender.DefaultButton
                 Case ActionButton.DefaultButtons.Edit
-                    Using f As New LabelsForm(MyGroups, (From g As DownloadGroup In Settings.Groups Where Not g.IsViewFilter Select g.Name)) With {.Text = "Groups", .Icon = My.Resources.GroupByIcon_16}
+                    Using f As New LabelsForm(MyGroups, (From g As DownloadGroup In Settings.Groups Where Not g.IsViewFilter Select g.Name)) With {
+                        .Text = "Groups (F3 to edit)",
+                        .Icon = My.Resources.GroupByIcon_16,
+                        .IsGroups = True
+                    }
                         f.ShowDialog()
                         If f.DialogResult = DialogResult.OK Then MyGroups.ListAddList(f.LabelsList, LAP.ClearBeforeAdd) : TXT_GROUPS.Text = MyGroups.ListToString
                     End Using
                 Case ActionButton.DefaultButtons.Clear : MyGroups.Clear()
+                Case ActionButton.DefaultButtons.Info
+                    Try
+                        If MyGroups.Count > 0 Then
+                            Dim i% = Settings.Groups.IndexOf(MyGroups(0))
+                            If i >= 0 Then
+                                Using gf As New GroupEditorForm(Settings.Groups(i)) : gf.ShowDialog() : End Using
+                            End If
+                        End If
+                    Catch ex As Exception
+                        ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "Show group")
+                    End Try
             End Select
         End Sub
         Private Sub ChangeEnabled() Handles OPT_DISABLED.CheckedChanged,

SCrawler/Download/Automation/Scheduler.vb

@@ -14,15 +14,19 @@ Imports PauseModes = SCrawler.DownloadObjects.AutoDownloader.PauseModes
 Namespace DownloadObjects
     Friend Class Scheduler : Implements IEnumerable(Of AutoDownloader), IMyEnumerator(Of AutoDownloader), IDisposable
         Friend Const Name_Plan As String = "Plan"
+        Friend Delegate Sub PlanChangedEventHandler(ByVal Plan As AutoDownloader)
         Friend Event PauseChanged As AutoDownloader.PauseChangedEventHandler
         Private Sub OnPauseChanged(ByVal Value As PauseModes)
             RaiseEvent PauseChanged(Pause)
         End Sub
+        Friend Event PlanChanged As PlanChangedEventHandler
+        Private Sub OnPlanChanged(ByVal Plan As AutoDownloader)
+            Try : RaiseEvent PlanChanged(Plan) : Catch : End Try
+        End Sub
         Private ReadOnly Plans As List(Of AutoDownloader)
         Friend Const FileNameDefault As String = "AutoDownload"
         Friend ReadOnly FileDefault As SFile = $"{SettingsFolderName}\{FileNameDefault}.xml"
         Friend File As SFile = Nothing
-        Private ReadOnly PlanWorking As Predicate(Of AutoDownloader) = Function(Plan) Plan.Working
         Private ReadOnly PlanDownloading As Predicate(Of AutoDownloader) = Function(Plan) Plan.Downloading
         Private ReadOnly PlansWaiter As Action(Of Predicate(Of AutoDownloader)) = Sub(ByVal Predicate As Predicate(Of AutoDownloader))
                                                                                       While Plans.Exists(Predicate) : Thread.Sleep(200) : End While
@@ -69,6 +73,7 @@ Namespace DownloadObjects
         Friend Sub Add(ByVal Plan As AutoDownloader)
             Plan.Source = Me
             AddHandler Plan.PauseChanged, AddressOf OnPauseChanged
+            AddHandler Plan.PlanChanged, AddressOf OnPlanChanged
             Plans.Add(Plan)
             Plans.ListReindex
             Update()
@@ -77,9 +82,9 @@ Namespace DownloadObjects
             If Index.ValueBetween(0, Count - 1) Then
                 With Plans(Index)
                     .Stop()
-                    If .Working Then
+                    If .Downloading Then
                         Await Task.Run(Sub()
-                                           While .Working : Thread.Sleep(510) : End While
+                                           While .Downloading : Thread.Sleep(510) : End While
                                        End Sub)
                     End If
                     .Dispose()
@@ -103,10 +108,10 @@ Namespace DownloadObjects
             End Try
         End Sub
         Friend Function Reset(ByVal f As SFile, ByVal IsInit As Boolean) As Boolean
-            If Plans.Count > 0 Then
-                If Not Plans.Exists(PlanWorking) Then
+            If Count > 0 Then
+                If Not Plans.Exists(PlanDownloading) Then
                     Pause = PauseModes.Unlimited
-                    If Plans.Exists(PlanWorking) Then
+                    If Plans.Exists(PlanDownloading) Then
                         MsgBoxE({$"Some plans are already being worked.{vbCr}Wait for the plans to complete their work and try again.",
                                  "Change scheduler"}, vbCritical)
                         Pause = PauseModes.Unlimited
@@ -114,6 +119,7 @@ Namespace DownloadObjects
                     End If
                 End If
                 [Stop]()
+                While Working : Thread.Sleep(200) : End While
                 If _UpdateRequired Then Update()
                 Plans.ListClearDispose(,, EDP.LogMessageValue)
             End If
@@ -127,11 +133,12 @@ Namespace DownloadObjects
                         Plans.Add(New AutoDownloader(x))
                     End If
                 End Using
-                If Plans.Count > 0 Then Plans.ForEach(Sub(ByVal p As AutoDownloader)
-                                                          p.Source = Me
-                                                          If Not IsInit Then p.Pause = PauseModes.Unlimited
-                                                          AddHandler p.PauseChanged, AddressOf OnPauseChanged
-                                                      End Sub) : Plans.ListReindex
+                If Count > 0 Then Plans.ForEach(Sub(ByVal p As AutoDownloader)
+                                                    p.Source = Me
+                                                    If Not IsInit Then p.Pause = PauseModes.Unlimited
+                                                    AddHandler p.PauseChanged, AddressOf OnPauseChanged
+                                                    AddHandler p.PlanChanged, AddressOf OnPlanChanged
+                                                End Sub) : Plans.ListReindex
             End If
             Return True
         End Function
@@ -163,38 +170,96 @@ Namespace DownloadObjects
         End Sub
 #End Region
 #Region "Execution"
+        Private AThread As Thread = Nothing
+        Private _StopRequested As Boolean = False
+        Friend ReadOnly Property Working As Boolean
+            Get
+                Return If(AThread?.IsAlive, False)
+            End Get
+        End Property
         Friend Async Function Start(ByVal Init As Boolean) As Task
             Try
+                _StopRequested = False
                 Await Task.Run(Sub()
                                    Dim r% = 0
                                    Do
                                        r += 1
                                        Try
-                                           If Count > 0 Then
-                                               If Plans.Exists(PlanDownloading) Then PlansWaiter(PlanDownloading)
-                                               For Each Plan In Plans
-                                                   Plan.Start(Init)
-                                                   PlansWaiter(PlanDownloading)
-                                                   Thread.Sleep(1000)
-                                               Next
-                                           End If
+                                           If Count > 0 Then PlansWaiter(PlanDownloading) : Plans.ForEach(Sub(p) p.Start(Init))
                                            Exit Do
                                        Catch io_ex As InvalidOperationException 'Collection was modified; enumeration operation may not execute
                                        End Try
                                    Loop While r < 10
                                End Sub)
+                If Not Working Then
+                    AThread = New Thread(New ThreadStart(AddressOf Checker))
+                    AThread.SetApartmentState(ApartmentState.MTA)
+                    AThread.Start()
+                End If
             Catch ex As Exception
                 If Init Then
                     ErrorsDescriber.Execute(EDP.SendToLog, ex, "Start automation")
-                    MainFrameObj.UpdateLogButton()
                 Else
                     Throw ex
                 End If
             End Try
         End Function
         Friend Sub [Stop]()
+            If Working Then _StopRequested = True
             If Count > 0 Then Plans.ForEach(Sub(p) p.Stop())
         End Sub
+        Private Sub Checker()
+            Do
+                Try
+                    If Count = 0 Or _StopRequested Then Exit Sub
+                    PlansWaiter.Invoke(PlanDownloading)
+                    Dim i% = Checker_GetNextPlanIndex()
+                    If i >= 0 Then Checker_DownloadPlan(i)
+                    Thread.Sleep(500)
+                Catch dex As ArgumentOutOfRangeException When disposedValue Or _StopRequested
+                Catch ex As Exception
+                    ErrorsDescriber.Execute(EDP.SendToLog, ex, "[Scheduler.Checker]")
+                End Try
+            Loop While Not _StopRequested
+            _StopRequested = False
+        End Sub
+        Private Sub Checker_DownloadPlan(ByVal PlanIndex As Integer)
+            While Downloader.Working : Thread.Sleep(200) : End While
+            With Plans(PlanIndex)
+                If .Downloading Then
+                    PlansWaiter.Invoke(PlanDownloading)
+                ElseIf .DownloadReady Then
+                    .Download()
+                    If Settings.AutomationScript.Use AndAlso Not Settings.AutomationScript.Value.IsEmptyString AndAlso
+                       (Not .IsManual Or Not Settings.AutomationScript_ExcludeManual) Then ExecuteCommand(Settings.AutomationScript)
+                End If
+            End With
+        End Sub
+        Private Function Checker_GetNextPlanIndex() As Integer
+            Try
+                Dim result% = -1
+                Dim l As New List(Of KeyValuePair(Of Integer, Date))
+                Dim d As Date?
+                If Count > 0 Then
+                    For i% = 0 To Count - 1
+                        With Plans(i)
+                            If .DownloadReady Then
+                                d = .NextDate
+                                If d.HasValue Then l.Add(New KeyValuePair(Of Integer, Date)(i, d.Value))
+                            End If
+                        End With
+                    Next
+                End If
+                If l.Count > 0 Then
+                    Dim md As Date = l.Min(Function(p) p.Value)
+                    result = l.Find(Function(p) p.Value = md).Key
+                    l.Clear()
+                End If
+                Return result
+            Catch
+                Return -1
+            End Try
+        End Function
         Friend Property Pause(Optional ByVal LimitDate As Date? = Nothing) As PauseModes
             Get
                 If Count > 0 Then Return Plans.FirstOrDefault(Function(p) p.Pause >= PauseModes.Disabled).Pause Else Return PauseModes.Disabled
@@ -218,7 +283,8 @@ Namespace DownloadObjects
             If Not disposedValue Then
                 If disposing Then
                     [Stop]()
-                    If Plans.Exists(PlanWorking) Then Task.WaitAll(Task.Run(Sub() PlansWaiter(PlanWorking)))
+                    If Plans.Exists(PlanDownloading) Then Task.WaitAll(Task.Run(Sub() PlansWaiter(PlanDownloading)))
+                    While Working : Thread.Sleep(200) : End While
                     If _UpdateRequired Then Update()
                     Plans.ListClearDispose
                 End If

SCrawler/Download/Automation/SchedulerEditorForm.Designer.vb

@@ -23,7 +23,8 @@ Namespace DownloadObjects
         <System.Diagnostics.DebuggerStepThrough()>
         Private Sub InitializeComponent()
             Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
-            Me.LIST_PLANS = New System.Windows.Forms.ListBox()
+            Me.LIST_PLANS = New System.Windows.Forms.ListView()
+            Me.COL_MAIN = CType(New System.Windows.Forms.ColumnHeader(), System.Windows.Forms.ColumnHeader)
             CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
             CONTAINER_MAIN.ContentPanel.SuspendLayout()
             CONTAINER_MAIN.SuspendLayout()
@@ -47,12 +48,25 @@ Namespace DownloadObjects
             '
             'LIST_PLANS
             '
+            Me.LIST_PLANS.Alignment = System.Windows.Forms.ListViewAlignment.Left
+            Me.LIST_PLANS.Columns.AddRange(New System.Windows.Forms.ColumnHeader() {Me.COL_MAIN})
             Me.LIST_PLANS.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.LIST_PLANS.FormattingEnabled = True
+            Me.LIST_PLANS.FullRowSelect = True
+            Me.LIST_PLANS.HeaderStyle = System.Windows.Forms.ColumnHeaderStyle.None
+            Me.LIST_PLANS.HideSelection = False
             Me.LIST_PLANS.Location = New System.Drawing.Point(0, 0)
+            Me.LIST_PLANS.MultiSelect = False
             Me.LIST_PLANS.Name = "LIST_PLANS"
+            Me.LIST_PLANS.ShowGroups = False
             Me.LIST_PLANS.Size = New System.Drawing.Size(414, 316)
             Me.LIST_PLANS.TabIndex = 0
+            Me.LIST_PLANS.UseCompatibleStateImageBehavior = False
+            Me.LIST_PLANS.View = System.Windows.Forms.View.Details
+            '
+            'COL_MAIN
+            '
+            Me.COL_MAIN.Text = "Task"
+            Me.COL_MAIN.Width = 410
             '
             'SchedulerEditorForm
             '
@@ -63,7 +77,6 @@ Namespace DownloadObjects
             Me.KeyPreview = True
             Me.MinimumSize = New System.Drawing.Size(430, 380)
             Me.Name = "SchedulerEditorForm"
-            Me.ShowIcon = True
             Me.ShowInTaskbar = False
             Me.Text = "Scheduler"
             CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
@@ -72,6 +85,7 @@ Namespace DownloadObjects
             Me.ResumeLayout(False)
 
         End Sub
-        Private WithEvents LIST_PLANS As ListBox
+        Private WithEvents LIST_PLANS As ListView
+        Private WithEvents COL_MAIN As ColumnHeader
     End Class
 End Namespace

SCrawler/Download/Automation/SchedulerEditorForm.vb

@@ -6,8 +6,12 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Threading
+Imports System.ComponentModel
 Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Forms.Toolbars
+Imports PersonalUtilities.Forms.Controls.Base
+Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Tools
 Imports ECI = PersonalUtilities.Forms.Toolbars.EditToolbar.ControlItem
 Imports ADB = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons
@@ -129,6 +133,16 @@ Namespace DownloadObjects
                 SetTitle()
                 .EndLoaderOperations(False)
             End With
+            SchedulerCheckerThread = New Thread(New ThreadStart(AddressOf SchedulerChecker))
+            SchedulerCheckerThread.SetApartmentState(ApartmentState.MTA)
+            SchedulerCheckerThread.Start()
+            Try : AddHandler Settings.Automation.PlanChanged, AddressOf Scheduler_PauseChanged : Catch : End Try
+        End Sub
+        Private _CloseRequested As Boolean = False
+        Private Sub SchedulerEditorForm_Closing(sender As Object, e As CancelEventArgs) Handles Me.Closing
+            _CloseRequested = True
+            While If(SchedulerCheckerThread?.IsAlive, False) : Thread.Sleep(200) : End While
+            Try : RemoveHandler Settings.Automation.PlanChanged, AddressOf Scheduler_PauseChanged : Catch : End Try
         End Sub
         Private Sub SchedulerEditorForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown
             If e.KeyCode = Keys.Escape Then
@@ -141,21 +155,48 @@ Namespace DownloadObjects
         Private Sub SchedulerEditorForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed
             PauseArr.Dispose()
         End Sub
+        Private Sub SchedulerEditorForm_SizeChanged(sender As Object, e As EventArgs) Handles Me.SizeChanged
+            Try : ControlInvokeFast(LIST_PLANS, Sub()
+                                                    COL_MAIN.Width = -2
+                                                    LIST_PLANS.Refresh()
+                                                End Sub, EDP.None) : Catch : End Try
+        End Sub
 #End Region
+        Private Property ListSelectedIndex As Integer
+            Get
+                Return If(LIST_PLANS.SelectedIndices.Count > 0, LIST_PLANS.SelectedIndices(0), -1)
+            End Get
+            Set(ByVal indx As Integer)
+                Try : ControlInvokeFast(LIST_PLANS, Sub()
+                                                        With LIST_PLANS
+                                                            .SelectedItems.Clear()
+                                                            .Items(indx).Selected = True
+                                                            .FocusedItem = .Items(indx)
+                                                            .Refresh()
+                                                            .EnsureVisible(indx)
+                                                        End With
+                                                    End Sub, EDP.None) : Catch : End Try
+            End Set
+        End Property
         Private _RefillInProgress As Boolean = False
         Private Sub Refill() Handles MyDefs.ButtonUpdateClick
             Try
-                If Not _RefillInProgress Then
-                    _RefillInProgress = True
-                    LIST_PLANS.Items.Clear()
-                    If Settings.Automation.Count > 0 Then
-                        LIST_PLANS.Items.AddRange(Settings.Automation.Select(Function(a) a.ToString()).Cast(Of Object).ToArray)
-                        If _LatestSelected.ValueBetween(0, LIST_PLANS.Items.Count - 1) Then LIST_PLANS.SelectedIndex = _LatestSelected
-                    Else
-                        _LatestSelected = -1
+                ControlInvokeFast(LIST_PLANS,
+                Sub()
+                    If Not _RefillInProgress Then
+                        _RefillInProgress = True
+                        With LIST_PLANS.Items
+                            .Clear()
+                            If Settings.Automation.Count > 0 Then
+                                .AddRange(Settings.Automation.Select(Function(a) a.LVI).ToArray)
+                                If _LatestSelected.ValueBetween(0, .Count - 1) Then ListSelectedIndex = _LatestSelected
+                            Else
+                                _LatestSelected = -1
+                            End If
+                        End With
+                        _RefillInProgress = False
                     End If
-                    _RefillInProgress = False
-                End If
+                End Sub)
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[DownloadObjects.SchedulerEditorForm.Refill]")
             End Try
@@ -194,7 +235,7 @@ Namespace DownloadObjects
         Private Sub Edit() Handles MyDefs.ButtonEditClick
             If _LatestSelected.ValueBetween(0, LIST_PLANS.Items.Count - 1) Then
                 Using f As New AutoDownloaderEditorForm(Settings.Automation(_LatestSelected)) : f.ShowDialog() : End Using
-                Refill()
+                LIST_PLANS_Refresh(_LatestSelected)
             Else
                 MsgBoxE("You have not selected a plan to edit.", vbExclamation)
             End If
@@ -222,19 +263,83 @@ Namespace DownloadObjects
 #End Region
 #Region "List handlers"
         Private _LatestSelected As Integer = -1
+        Private _LatestSelectedChangeEnabled As Boolean = True
         Private Sub LIST_PLANS_SelectedIndexChanged(sender As Object, e As EventArgs) Handles LIST_PLANS.SelectedIndexChanged
-            _LatestSelected = LIST_PLANS.SelectedIndex
-            PauseArr.PlanIndex = _LatestSelected
-            PauseArr.UpdatePauseButtons(False)
+            If _LatestSelectedChangeEnabled Then
+                _LatestSelected = ListSelectedIndex
+                PauseArr.PlanIndex = _LatestSelected
+                PauseArr.UpdatePauseButtons(False)
+            End If
         End Sub
         Private Sub LIST_PLANS_MouseDoubleClick(sender As Object, e As MouseEventArgs) Handles LIST_PLANS.MouseDoubleClick
             Edit()
         End Sub
+        Private Sub LIST_PLANS_Refresh(Optional ByVal PlanIndex As Integer = -1)
+            _LatestSelectedChangeEnabled = False
+            Try
+                ControlInvokeFast(LIST_PLANS,
+                  Sub()
+                      Dim indx% = If(PlanIndex >= 0, PlanIndex, ListSelectedIndex)
+                      If indx = -1 And PlanIndex = -1 Then indx = _LatestSelected
+                      If indx.ValueBetween(0, Settings.Automation.Count - 1) Then _
+                         LIST_PLANS.Items(indx) = Settings.Automation(indx).LVI
+                      If _LatestSelected.ValueBetween(0, LIST_PLANS.Items.Count - 1) Then ListSelectedIndex = _LatestSelected
+                  End Sub, EDP.None)
+            Catch
+            Finally
+                _LatestSelectedChangeEnabled = True
+            End Try
+        End Sub
+        Private Sub Scheduler_PauseChanged(ByVal Plan As AutoDownloader)
+            LIST_PLANS_Refresh(Plan.Index)
+        End Sub
+        Private SchedulerCheckerThread As Thread
+        Private Sub SchedulerChecker()
+            Try
+                While Not _CloseRequested
+                    Try
+                        With Settings.Automation
+                            If .Count > 0 Then
+                                For i% = 0 To .Count - 1
+                                    If .Item(i).LVIStateChanged Then LIST_PLANS_Refresh(i)
+                                    If _CloseRequested Then Exit Sub
+                                    Thread.Sleep(200)
+                                Next
+                            Else
+                                If _CloseRequested Then Exit Sub
+                                Thread.Sleep(200)
+                            End If
+                        End With
+                    Catch
+                    End Try
+                End While
+            Catch
+            End Try
+        End Sub
 #End Region
 #Region "Settings, Start, Skip, Pause"
         Private Function GetSchedulerFiles() As List(Of SFile)
             Return SFile.GetFiles(SettingsFolderName.CSFileP, $"{Scheduler.FileNameDefault}*.xml",, EDP.ReturnValue)
         End Function
+        Private Class SchedulerList : Inherits SimpleListForm(Of String)
+            Friend Sub New(ByVal Source As IEnumerable(Of String), Optional ByRef DesignXML As EContainer = Nothing)
+                MyBase.New(Source, DesignXML)
+            End Sub
+            Protected Overrides Sub MyForm_Load(sender As Object, e As EventArgs)
+                MyBase.MyForm_Load(sender, e)
+                CMB_DATA.Button(ADB.Add).ToolTipText = "Create a new scheduler"
+                CMB_DATA.Button(ADB.SaveAs).ToolTipText = "Clone an existing scheduler and save it as a new one"
+                CMB_DATA.Button(ADB.Delete).ToolTipText = "Delete the selected scheduler"
+                CMB_DATA.Buttons.UpdateButtonsPositions()
+            End Sub
+            Protected Overrides Sub CMB_DATA_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As ActionButtonEventArgs)
+                If e.DefaultButton = ADB.SaveAs Then
+                    AddNewItem(e, e.Key, e.KeyEventArgs)
+                Else
+                    MyBase.CMB_DATA_ActionOnButtonClick(Sender, e)
+                End If
+            End Sub
+        End Class
         Private Sub BTT_SETTINGS_Click(sender As Object, e As EventArgs) Handles BTT_SETTINGS.Click
             Const msgTitle$ = "Change scheduler"
             Try
@@ -244,7 +349,7 @@ Namespace DownloadObjects
                     If .ListExists Then .ForEach(Sub(ff) l.Add(ff, ff.Name.Replace(Scheduler.FileNameDefault, String.Empty).StringTrimStart("_").IfNullOrEmpty(defName)))
                 End With
                 If l.Count > 0 Then
-                    Using chooser As New SimpleListForm(Of String)(l.Values.Cast(Of String), Settings.Design) With {
+                    Using chooser As New SchedulerList(l.Values.Cast(Of String), Settings.Design) With {
                         .DesignXMLNodeName = "SchedulerChooserForm",
                         .Icon = ImageRenderer.GetIcon(My.Resources.ScriptPic_32, EDP.ReturnValue),
                         .FormText = "Schedulers",
@@ -256,17 +361,56 @@ Namespace DownloadObjects
                             Dim f As SFile
                             Dim selectedName$
                             Dim addedObj$ = String.Empty
+                            Dim addedObjIsClone As Boolean = False
+                            Dim createSchedulerPath As Func(Of String, SFile) = Function(n) $"{SettingsFolderName}\{Scheduler.FileNameDefault}_{n.StringRemoveWinForbiddenSymbols}.xml"
                             .ClearButtons()
-                            .Buttons = {ADB.Add, ADB.Delete}
+                            .Buttons = {ADB.Add, ADB.SaveAs, ADB.Delete}
                             AddHandler .AddClick, Sub(ByVal obj As Object, ByVal args As SimpleListFormEventArgs)
                                                       If addedObj.IsEmptyString Then
                                                           addedObj = InputBoxE("Enter a new scheduler name:", msgTitle)
                                                           args.Result = Not addedObj.IsEmptyString
-                                                          If args.Result Then args.Item = addedObj
+                                                          If args.Result Then
+                                                              If l.Values.Count > 0 AndAlso l.Values.ListIndexOf(Function(n) n.StringToLower = addedObj.StringToLower) >= 0 Then
+                                                                  args.Result = False
+                                                                  MsgBoxE({$"A scheduler named '{addedObj}' already exists", msgTitle}, vbCritical)
+                                                              Else
+                                                                  args.Item = addedObj
+                                                                  addedObjIsClone = Not args.ButtonEventArgs Is Nothing AndAlso
+                                                                                    TypeOf args.ButtonEventArgs Is ActionButtonEventArgs AndAlso
+                                                                                    DirectCast(args.ButtonEventArgs, ActionButtonEventArgs).DefaultButton = ADB.SaveAs
+                                                                  If addedObjIsClone Then
+                                                                      Dim cloneF As SFile = createSchedulerPath.Invoke(addedObj)
+                                                                      If Not cloneF.Exists And Settings.Automation.File.Exists Then
+                                                                          Using x As New XmlFile(Settings.Automation.File, Protector.Modes.All, False) With {.AllowSameNames = True, .XmlReadOnly = True}
+                                                                              x.LoadData()
+                                                                              x.Save(cloneF, EDP.SendToLog)
+                                                                          End Using
+                                                                      End If
+                                                                  End If
+                                                              End If
+                                                          End If
                                                       Else
                                                           MsgBoxE({"You can only create one scheduler at a time", "Create a new scheduler"}, vbCritical)
                                                       End If
                                                   End Sub
+                            AddHandler .DeleteClick, Sub(ByVal obj As Object, ByVal args As SimpleListFormEventArgs)
+                                                         Dim n$ = AConvert(Of String)(args.Item, String.Empty)
+                                                         If Not n.IsEmptyString Then
+                                                             If MsgBoxE({$"Are you sure you want to delete the '{n}' scheduler?", msgTitle}, vbExclamation,,,
+                                                                        {"Process", "Cancel"}) = 0 Then
+                                                                 Dim delF As SFile = createSchedulerPath.Invoke(n)
+                                                                 If delF.Exists AndAlso delF.Delete Then
+                                                                     args.Result = True
+                                                                     If l.ContainsKey(delF) Then
+                                                                         l.Remove(delF)
+                                                                     Else
+                                                                         Dim delIndx% = l.ListIndexOf(Function(dd) dd.Value = n)
+                                                                         If delIndx >= 0 Then l.Remove(l.Keys(delIndx))
+                                                                     End If
+                                                                 End If
+                                                             End If
+                                                         End If
+                                                     End Sub
                             If Settings.Automation.File.Name = Scheduler.FileNameDefault Then
                                 .DataSelectedIndexes.Add(0)
                             Else
@@ -279,7 +423,7 @@ Namespace DownloadObjects
                                     If selectedName = defName Then
                                         f = Settings.Automation.FileDefault
                                     Else
-                                        f = $"{SettingsFolderName}\{Scheduler.FileNameDefault}_{selectedName.StringRemoveWinForbiddenSymbols}.xml"
+                                        f = createSchedulerPath.Invoke(selectedName)
                                     End If
                                     If Not Settings.Automation.File = f AndAlso Settings.Automation.Reset(f, False) Then
                                         Settings.Automation.File = f
@@ -312,13 +456,13 @@ Namespace DownloadObjects
         Private Sub BTT_START_Click(sender As Object, e As EventArgs) Handles BTT_START.Click
             If _LatestSelected.ValueBetween(0, LIST_PLANS.Items.Count - 1) Then
                 With Settings.Automation(_LatestSelected) : .Start(.IsNewPlan) : End With
-                Refill()
+                LIST_PLANS_Refresh(_LatestSelected)
             End If
         End Sub
         Private Sub BTT_START_FORCE_Click(sender As Object, e As EventArgs) Handles BTT_START_FORCE.Click
             If _LatestSelected.ValueBetween(0, LIST_PLANS.Items.Count - 1) Then
                 With Settings.Automation(_LatestSelected)
-                    If .Working Or .IsManual Then .ForceStart() : Refill()
+                    If .Working Or .IsManual Then .ForceStart() : LIST_PLANS_Refresh(_LatestSelected)
                 End With
             End If
         End Sub
@@ -328,10 +472,10 @@ Namespace DownloadObjects
                 Select Case mode
                     Case String.Empty
                         Settings.Automation(_LatestSelected).Skip()
-                        Refill()
+                        LIST_PLANS_Refresh(_LatestSelected)
                     Case "m"
                         Dim mins% = AConvert(Of Integer)(InputBoxE("Enter a number of minutes you want to delay:", Sender.Text, 60), -1)
-                        If mins > 0 Then Settings.Automation(_LatestSelected).Skip(mins) : Refill()
+                        If mins > 0 Then Settings.Automation(_LatestSelected).Skip(mins) : LIST_PLANS_Refresh(_LatestSelected)
                     Case "d"
                         Dim d As Date? = Nothing
                         Using f As New DateTimeSelectionForm(DateTimeSelectionForm.Modes.Date +
@@ -342,16 +486,13 @@ Namespace DownloadObjects
                             f.ShowDialog()
                             If f.DialogResult = DialogResult.OK Then d = f.MyDateStart
                         End Using
-                        If d.HasValue Then Settings.Automation(_LatestSelected).Skip(d.Value) : Refill()
+                        If d.HasValue Then Settings.Automation(_LatestSelected).Skip(d.Value) : LIST_PLANS_Refresh(_LatestSelected)
                     Case "r"
                         Settings.Automation(_LatestSelected).SkipReset()
-                        Refill()
+                        LIST_PLANS_Refresh(_LatestSelected)
                 End Select
             End If
         End Sub
-        Private Sub PauseArr_Updating() Handles PauseArr.Updating
-            Refill()
-        End Sub
 #End Region
 #Region "Move"
         Private Sub BTT_MOVE_UP_DOWN_Click(sender As Object, e As EventArgs) Handles BTT_MOVE_UP.Click, BTT_MOVE_DOWN.Click
@@ -381,7 +522,7 @@ Namespace DownloadObjects
                                 users.ListAddList(Groups.DownloadGroup.GetUsers(.Self))
                             End If
                         End If
-                        Groups.GroupUsersViewer.Show(users)
+                        Groups.GroupUsersViewer.Show(users, $"S { .Name}")
                         users.Clear()
                     End With
                 End If

SCrawler/Download/Feed/DownloadFeedForm.Designer.vb

@@ -35,6 +35,7 @@ Namespace DownloadObjects
             Dim MENU_LOAD_SEP_7 As System.Windows.Forms.ToolStripSeparator
             Dim MENU_LOAD_SEP_0 As System.Windows.Forms.ToolStripSeparator
             Dim MENU_LOAD_SEP_8 As System.Windows.Forms.ToolStripSeparator
+            Dim MENU_LOAD_SEP_9 As System.Windows.Forms.ToolStripSeparator
             Me.OPT_DEFAULT = New System.Windows.Forms.ToolStripMenuItem()
             Me.OPT_SUBSCRIPTIONS = New System.Windows.Forms.ToolStripMenuItem()
             Me.ToolbarTOP = New System.Windows.Forms.ToolStrip()
@@ -44,6 +45,8 @@ Namespace DownloadObjects
             Me.BTT_LOAD_SESSION_CHOOSE = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_COPY_TO = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_MOVE_TO = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_COPY_SPEC_TO = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_MOVE_SPEC_TO = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_LOAD_FAV = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_LOAD_SPEC = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_FEED_ADD_FAV = New System.Windows.Forms.ToolStripMenuItem()
@@ -57,19 +60,22 @@ Namespace DownloadObjects
             Me.BTT_FEED_DELETE_SPEC = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_FEED_DELETE_DAILY_LIST = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_FEED_DELETE_DAILY_DATE = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_CURR_SESSION_SET = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_MERGE_SESSIONS = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_CLEAR_DAILY = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_RESET_DAILY = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_MERGE_FEEDS = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_CHECK_ALL = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_CHECK_NONE = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_VIEW_SAVE = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_VIEW_LOAD = New System.Windows.Forms.ToolStripMenuItem()
             Me.SEP_0 = New System.Windows.Forms.ToolStripSeparator()
             Me.MENU_DOWN = New System.Windows.Forms.ToolStripDropDownButton()
             Me.BTT_DOWN_ALL = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_DOWN_SELECTED = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_REFRESH = New System.Windows.Forms.ToolStripButton()
             Me.TP_DATA = New System.Windows.Forms.TableLayoutPanel()
-            Me.BTT_VIEW_SAVE = New System.Windows.Forms.ToolStripMenuItem()
-            Me.BTT_VIEW_LOAD = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_CURR_SESSION_SET_LAST = New System.Windows.Forms.ToolStripMenuItem()
             SEP_1 = New System.Windows.Forms.ToolStripSeparator()
             SEP_2 = New System.Windows.Forms.ToolStripSeparator()
             MENU_VIEW = New System.Windows.Forms.ToolStripDropDownButton()
@@ -82,6 +88,7 @@ Namespace DownloadObjects
             MENU_LOAD_SEP_7 = New System.Windows.Forms.ToolStripSeparator()
             MENU_LOAD_SEP_0 = New System.Windows.Forms.ToolStripSeparator()
             MENU_LOAD_SEP_8 = New System.Windows.Forms.ToolStripSeparator()
+            MENU_LOAD_SEP_9 = New System.Windows.Forms.ToolStripSeparator()
             Me.ToolbarTOP.SuspendLayout()
             Me.SuspendLayout()
             '
@@ -157,6 +164,16 @@ Namespace DownloadObjects
             MENU_LOAD_SEP_0.Name = "MENU_LOAD_SEP_0"
             MENU_LOAD_SEP_0.Size = New System.Drawing.Size(349, 6)
             '
+            'MENU_LOAD_SEP_8
+            '
+            MENU_LOAD_SEP_8.Name = "MENU_LOAD_SEP_8"
+            MENU_LOAD_SEP_8.Size = New System.Drawing.Size(349, 6)
+            '
+            'MENU_LOAD_SEP_9
+            '
+            MENU_LOAD_SEP_9.Name = "MENU_LOAD_SEP_9"
+            MENU_LOAD_SEP_9.Size = New System.Drawing.Size(349, 6)
+            '
             'ToolbarTOP
             '
             Me.ToolbarTOP.GripStyle = System.Windows.Forms.ToolStripGripStyle.Hidden
@@ -170,7 +187,7 @@ Namespace DownloadObjects
             '
             Me.MENU_LOAD_SESSION.AutoToolTip = False
             Me.MENU_LOAD_SESSION.DisplayStyle = System.Windows.Forms.ToolStripItemDisplayStyle.Image
-            Me.MENU_LOAD_SESSION.DropDownItems.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_LOAD_SESSION_CURRENT, Me.BTT_LOAD_SESSION_LAST, Me.BTT_LOAD_SESSION_CHOOSE, MENU_LOAD_SEP_0, Me.BTT_COPY_TO, Me.BTT_MOVE_TO, MENU_LOAD_SEP_1, Me.BTT_LOAD_FAV, Me.BTT_LOAD_SPEC, MENU_LOAD_SEP_2, Me.BTT_FEED_ADD_FAV, Me.BTT_FEED_ADD_FAV_REMOVE, Me.BTT_FEED_REMOVE_FAV, MENU_LOAD_SEP_3, Me.BTT_FEED_ADD_SPEC, Me.BTT_FEED_ADD_SPEC_REMOVE, Me.BTT_FEED_REMOVE_SPEC, MENU_LOAD_SEP_4, Me.BTT_FEED_CLEAR_FAV, Me.BTT_FEED_CLEAR_SPEC, Me.BTT_FEED_DELETE_SPEC, Me.BTT_FEED_DELETE_DAILY_LIST, Me.BTT_FEED_DELETE_DAILY_DATE, MENU_LOAD_SEP_5, Me.BTT_MERGE_SESSIONS, Me.BTT_CLEAR_DAILY, MENU_LOAD_SEP_6, Me.BTT_MERGE_FEEDS, MENU_LOAD_SEP_7, Me.BTT_CHECK_ALL, Me.BTT_CHECK_NONE, MENU_LOAD_SEP_8, Me.BTT_VIEW_SAVE, Me.BTT_VIEW_LOAD})
+            Me.MENU_LOAD_SESSION.DropDownItems.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_LOAD_SESSION_CURRENT, Me.BTT_LOAD_SESSION_LAST, Me.BTT_LOAD_SESSION_CHOOSE, MENU_LOAD_SEP_0, Me.BTT_COPY_TO, Me.BTT_MOVE_TO, MENU_LOAD_SEP_1, Me.BTT_COPY_SPEC_TO, Me.BTT_MOVE_SPEC_TO, MENU_LOAD_SEP_2, Me.BTT_LOAD_FAV, Me.BTT_LOAD_SPEC, MENU_LOAD_SEP_3, Me.BTT_FEED_ADD_FAV, Me.BTT_FEED_ADD_FAV_REMOVE, Me.BTT_FEED_REMOVE_FAV, MENU_LOAD_SEP_4, Me.BTT_FEED_ADD_SPEC, Me.BTT_FEED_ADD_SPEC_REMOVE, Me.BTT_FEED_REMOVE_SPEC, MENU_LOAD_SEP_5, Me.BTT_FEED_CLEAR_FAV, Me.BTT_FEED_CLEAR_SPEC, Me.BTT_FEED_DELETE_SPEC, Me.BTT_FEED_DELETE_DAILY_LIST, Me.BTT_FEED_DELETE_DAILY_DATE, MENU_LOAD_SEP_6, Me.BTT_CURR_SESSION_SET, Me.BTT_CURR_SESSION_SET_LAST, Me.BTT_MERGE_SESSIONS, Me.BTT_CLEAR_DAILY, Me.BTT_RESET_DAILY, MENU_LOAD_SEP_7, Me.BTT_MERGE_FEEDS, MENU_LOAD_SEP_8, Me.BTT_CHECK_ALL, Me.BTT_CHECK_NONE, MENU_LOAD_SEP_9, Me.BTT_VIEW_SAVE, Me.BTT_VIEW_LOAD})
             Me.MENU_LOAD_SESSION.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
             Me.MENU_LOAD_SESSION.ImageTransparentColor = System.Drawing.Color.Magenta
             Me.MENU_LOAD_SESSION.Name = "MENU_LOAD_SESSION"
@@ -212,6 +229,24 @@ Namespace DownloadObjects
             Me.BTT_MOVE_TO.Size = New System.Drawing.Size(352, 22)
             Me.BTT_MOVE_TO.Text = "Move checked to..."
             '
+            'BTT_COPY_SPEC_TO
+            '
+            Me.BTT_COPY_SPEC_TO.AutoToolTip = True
+            Me.BTT_COPY_SPEC_TO.Image = Global.SCrawler.My.Resources.Resources.PastePic_32
+            Me.BTT_COPY_SPEC_TO.Name = "BTT_COPY_SPEC_TO"
+            Me.BTT_COPY_SPEC_TO.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_COPY_SPEC_TO.Text = "Copy feed/session files to..."
+            Me.BTT_COPY_SPEC_TO.ToolTipText = "Copy all the files of the loaded feed/session to..."
+            '
+            'BTT_MOVE_SPEC_TO
+            '
+            Me.BTT_MOVE_SPEC_TO.AutoToolTip = True
+            Me.BTT_MOVE_SPEC_TO.Image = Global.SCrawler.My.Resources.Resources.CutPic_48
+            Me.BTT_MOVE_SPEC_TO.Name = "BTT_MOVE_SPEC_TO"
+            Me.BTT_MOVE_SPEC_TO.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_MOVE_SPEC_TO.Text = "Move feed/session files to..."
+            Me.BTT_MOVE_SPEC_TO.ToolTipText = "Move all the files of the loaded feed/session to..."
+            '
             'BTT_LOAD_FAV
             '
             Me.BTT_LOAD_FAV.Image = Global.SCrawler.My.Resources.Resources.HeartPic_32
@@ -303,6 +338,15 @@ Namespace DownloadObjects
             Me.BTT_FEED_DELETE_DAILY_DATE.Size = New System.Drawing.Size(352, 22)
             Me.BTT_FEED_DELETE_DAILY_DATE.Text = "Delete daily feed (by date)"
             '
+            'BTT_CURR_SESSION_SET
+            '
+            Me.BTT_CURR_SESSION_SET.AutoToolTip = True
+            Me.BTT_CURR_SESSION_SET.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
+            Me.BTT_CURR_SESSION_SET.Name = "BTT_CURR_SESSION_SET"
+            Me.BTT_CURR_SESSION_SET.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_CURR_SESSION_SET.Text = "Set current session..."
+            Me.BTT_CURR_SESSION_SET.ToolTipText = "Select one of the download sessions and set it as the current session"
+            '
             'BTT_MERGE_SESSIONS
             '
             Me.BTT_MERGE_SESSIONS.AutoToolTip = True
@@ -322,6 +366,15 @@ Namespace DownloadObjects
             Me.BTT_CLEAR_DAILY.Text = "Clear session"
             Me.BTT_CLEAR_DAILY.ToolTipText = "Clear current session"
             '
+            'BTT_RESET_DAILY
+            '
+            Me.BTT_RESET_DAILY.AutoToolTip = True
+            Me.BTT_RESET_DAILY.Image = Global.SCrawler.My.Resources.Resources.RefreshPic_24
+            Me.BTT_RESET_DAILY.Name = "BTT_RESET_DAILY"
+            Me.BTT_RESET_DAILY.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_RESET_DAILY.Text = "Reset current session"
+            Me.BTT_RESET_DAILY.ToolTipText = "A new file will be created for the current session"
+            '
             'BTT_MERGE_FEEDS
             '
             Me.BTT_MERGE_FEEDS.AutoToolTip = True
@@ -343,6 +396,20 @@ Namespace DownloadObjects
             Me.BTT_CHECK_NONE.Size = New System.Drawing.Size(352, 22)
             Me.BTT_CHECK_NONE.Text = "Select none"
             '
+            'BTT_VIEW_SAVE
+            '
+            Me.BTT_VIEW_SAVE.Name = "BTT_VIEW_SAVE"
+            Me.BTT_VIEW_SAVE.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_VIEW_SAVE.Text = "Save current view"
+            '
+            'BTT_VIEW_LOAD
+            '
+            Me.BTT_VIEW_LOAD.AutoToolTip = True
+            Me.BTT_VIEW_LOAD.Name = "BTT_VIEW_LOAD"
+            Me.BTT_VIEW_LOAD.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_VIEW_LOAD.Text = "Load view (from saved)"
+            Me.BTT_VIEW_LOAD.ToolTipText = "Load one of your previously saved views"
+            '
             'SEP_0
             '
             Me.SEP_0.Name = "SEP_0"
@@ -409,24 +476,12 @@ Namespace DownloadObjects
             Me.TP_DATA.Size = New System.Drawing.Size(484, 436)
             Me.TP_DATA.TabIndex = 1
             '
-            'MENU_LOAD_SEP_8
+            'BTT_CURR_SESSION_SET_LAST
             '
-            MENU_LOAD_SEP_8.Name = "MENU_LOAD_SEP_8"
-            MENU_LOAD_SEP_8.Size = New System.Drawing.Size(349, 6)
-            '
-            'BTT_VIEW_SAVE
-            '
-            Me.BTT_VIEW_SAVE.Name = "BTT_VIEW_SAVE"
-            Me.BTT_VIEW_SAVE.Size = New System.Drawing.Size(352, 22)
-            Me.BTT_VIEW_SAVE.Text = "Save current view"
-            '
-            'BTT_VIEW_LOAD
-            '
-            Me.BTT_VIEW_LOAD.AutoToolTip = True
-            Me.BTT_VIEW_LOAD.Name = "BTT_VIEW_LOAD"
-            Me.BTT_VIEW_LOAD.Size = New System.Drawing.Size(352, 22)
-            Me.BTT_VIEW_LOAD.Text = "Load view (from saved)"
-            Me.BTT_VIEW_LOAD.ToolTipText = "Load one of your previously saved views"
+            Me.BTT_CURR_SESSION_SET_LAST.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
+            Me.BTT_CURR_SESSION_SET_LAST.Name = "BTT_CURR_SESSION_SET_LAST"
+            Me.BTT_CURR_SESSION_SET_LAST.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_CURR_SESSION_SET_LAST.Text = "Set last download session as current session"
             '
             'DownloadFeedForm
             '
@@ -483,5 +538,10 @@ Namespace DownloadObjects
         Private WithEvents BTT_MOVE_TO As ToolStripMenuItem
         Private WithEvents BTT_VIEW_SAVE As ToolStripMenuItem
         Private WithEvents BTT_VIEW_LOAD As ToolStripMenuItem
+        Private WithEvents BTT_CURR_SESSION_SET As ToolStripMenuItem
+        Private WithEvents BTT_COPY_SPEC_TO As ToolStripMenuItem
+        Private WithEvents BTT_MOVE_SPEC_TO As ToolStripMenuItem
+        Private WithEvents BTT_RESET_DAILY As ToolStripMenuItem
+        Private WithEvents BTT_CURR_SESSION_SET_LAST As ToolStripMenuItem
     End Class
 End Namespace

SCrawler/Download/Feed/DownloadFeedForm.resx

@@ -159,10 +159,13 @@
   <metadata name="MENU_LOAD_SEP_0.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <metadata name="ToolbarTOP.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
-    <value>17, 17</value>
-  </metadata>
   <metadata name="MENU_LOAD_SEP_8.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
+  <metadata name="MENU_LOAD_SEP_9.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
+  <metadata name="ToolbarTOP.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
+    <value>17, 17</value>
+  </metadata>
 </root>

SCrawler/Download/Feed/DownloadFeedForm.vb

@@ -173,6 +173,7 @@ Namespace DownloadObjects
                 MENU_DOWN.Visible = OPT_SUBSCRIPTIONS.Checked
                 UpdateSettings()
                 FeedChangeMode(FeedModes.Current)
+                Downloader.FilesLoadLastSession()
                 RefillList(True, False)
                 .EndLoaderOperations(False)
             End With
@@ -189,7 +190,6 @@ Namespace DownloadObjects
             DataList.Clear()
         End Sub
         Private Sub DownloadFeedForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown
-            'If e.KeyCode = Keys.F5 Then RefillList() : e.Handled = True
             If Not e.Handled Then
                 Dim b As Boolean = False
                 If e = GoToButton Then
@@ -206,6 +206,8 @@ Namespace DownloadObjects
                         Case Keys.Down, Keys.Right, Keys.PageDown : changePage = 1
                         Case Keys.Home : gotoHome = True
                         Case Keys.End : gotoHome = False
+                        Case Keys.Escape : If Settings.FeedEscToClose Then Close()
+                        Case Else : If e.Control And e.KeyCode = Keys.W Then Close()
                     End Select
                     If changePage.HasValue Then
                         b = True
@@ -275,7 +277,6 @@ Namespace DownloadObjects
                 End With
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "Feed removed")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
         Private Sub Feed_SPEC_LOAD(ByVal Source As ToolStripMenuItem, ByVal e As EventArgs)
@@ -439,7 +440,9 @@ Namespace DownloadObjects
         End Sub
         Private Sub SessionChooser(ByVal GetLast As Boolean, Optional ByVal GetFilesOnly As Boolean = False,
                                    Optional ByRef ResultFilesList As List(Of SFile) = Nothing,
-                                   Optional ByVal SelectedMode As FeedModes = -1)
+                                   Optional ByVal SelectedMode As FeedModes = -1,
+                                   Optional ByVal GetSessionFile As Boolean = False,
+                                   Optional ByRef SessionFile As SFile = Nothing)
             Try
                 LoadedSessionName = String.Empty
                 Downloader.ClearSessions()
@@ -474,6 +477,12 @@ Namespace DownloadObjects
                             If fList.ListExists Then
                                 If GetFilesOnly Then
                                     ResultFilesList.AddRange(fList)
+                                ElseIf GetSessionFile Then
+                                    If fList.Count > 1 Then
+                                        MsgBoxE({"You must select one session file", "Get session file"}, vbExclamation)
+                                    Else
+                                        SessionFile = fList(0)
+                                    End If
                                 Else
                                     DataList.Clear()
                                     If SelectedMode >= 0 Then
@@ -507,17 +516,21 @@ Namespace DownloadObjects
                         f = Downloader.FilesSessionActual(False)
                     End If
                     If f.Exists Then
-                        If SelectedMode >= 0 Then
-                            If SelectedMode = FeedModes.Saved Then LoadedSessionName = f.Name
-                            FeedChangeMode(SelectedMode)
+                        If GetSessionFile Then
+                            If Not Downloader.FilesSessionActual(False) = f Then SessionFile = f
+                        Else
+                            If SelectedMode >= 0 Then
+                                If SelectedMode = FeedModes.Saved Then LoadedSessionName = f.Name
+                                FeedChangeMode(SelectedMode)
+                            End If
+                            DataList.Clear()
+                            x = New XmlFile(f,, False) With {.AllowSameNames = True, .XmlReadOnly = True}
+                            x.LoadData()
+                            If x.Count > 0 Then DataList.ListAddList(x, lcr)
+                            x.Dispose()
+                            CleanDataList()
+                            RefillList(False, False)
                         End If
-                        DataList.Clear()
-                        x = New XmlFile(f,, False) With {.AllowSameNames = True, .XmlReadOnly = True}
-                        x.LoadData()
-                        If x.Count > 0 Then DataList.ListAddList(x, lcr)
-                        x.Dispose()
-                        CleanDataList()
-                        RefillList(False, False)
                     End If
                 Else
                     m.Text = "Saved sessions not found"
@@ -532,14 +545,20 @@ Namespace DownloadObjects
         Private Sub BTT_COPY_MOVE_TO_Click(sender As Object, e As EventArgs) Handles BTT_COPY_TO.Click, BTT_MOVE_TO.Click
             MoveCopyFiles(True, sender, Nothing, Nothing)
         End Sub
-        Private Function MoveCopyFiles(ByVal IsInternal As Boolean, ByVal Sender As Object, ByVal MCTOptions As FeedMoveCopyTo, ByVal FeedMediaData As FeedMedia) As Boolean
+        Private Sub BTT_COPY_MOVE_SPEC_TO_Click(sender As Object, e As EventArgs) Handles BTT_COPY_SPEC_TO.Click, BTT_MOVE_SPEC_TO.Click
+            MoveCopyFiles(True, sender, Nothing, Nothing, False)
+        End Sub
+        Private Function MoveCopyFiles(ByVal IsInternal As Boolean, ByVal Sender As Object, ByVal MCTOptions As FeedMoveCopyTo,
+                                       ByVal FeedMediaData As FeedMedia, Optional ByVal GetChecked As Boolean = True) As Boolean
             Const MsgTitle$ = "Copy/Move checked files"
             Try
-                Dim isCopy As Boolean = Not Sender Is Nothing AndAlso Sender Is BTT_COPY_TO
+                Dim isCopy As Boolean = Not Sender Is Nothing AndAlso (Sender Is BTT_COPY_TO OrElse Sender Is BTT_COPY_SPEC_TO)
                 Dim moveOptions As FeedMoveCopyTo = Nothing
                 Dim ff As SFile = Nothing, df As SFile
                 Dim data As IEnumerable(Of UserMediaD) = Nothing
                 Dim dd As UserMediaD
+                Dim __user As UserInfo
+                Dim __isSavedPosts As Boolean
                 Dim data_files As IEnumerable(Of SFile) = Nothing
                 Dim new_files As New List(Of SFile)
                 Dim mm As UserMediaD
@@ -558,7 +577,12 @@ Namespace DownloadObjects
                 Dim result As Boolean = False
 
                 If FeedMediaData Is Nothing Then
-                    data = GetCheckedMedia()
+                    If GetChecked Then
+                        data = GetCheckedMedia()
+                    ElseIf DataList.Count > 0 Then
+                        data = DataList.Where(Function(__dd) Not If(__dd.User?.IsSubscription, __dd.UserInfo.IsSubscription) AndAlso __dd.Data.File.Exists)
+                    End If
+
                     With data
                         If .ListExists Then data_files = .Select(Function(m) m.Data.File)
                     End With
@@ -579,7 +603,6 @@ Namespace DownloadObjects
                     With moveOptions
                         If Not .Destination.IsEmptyString And .ReplaceUserProfile And .ReplaceUserProfile_CreateIfNull And .ReplaceUserProfile_Profile Is Nothing Then
                             Dim existingPathInstances As IEnumerable(Of String) = Nothing
-                            Dim __user As UserInfo
                             Dim __host As Plugin.Hosts.SettingsHost = Settings(API.PathPlugin.PluginKey).Default
                             Dim __userName$ = .Destination.Segments.LastOrDefault
                             If Settings.UsersList.Count > 0 Then _
@@ -651,9 +674,12 @@ Namespace DownloadObjects
                                             indx = Downloader.Files.FindIndex(finder)
                                             If indx >= 0 Then
                                                 mm = Downloader.Files(indx)
+                                                __user = mm.UserInfo
                                                 mm_data = mm.Data
                                                 mm_data.File = df
-                                                mm = New UserMediaD(mm_data, If(moveOptions.ReplaceUserProfile_Profile, mm.User), mm.Session, mm.Date)
+                                                __isSavedPosts = mm.IsSavedPosts And moveOptions.ReplaceUserProfile_Profile Is Nothing
+                                                mm = New UserMediaD(mm_data, If(moveOptions.ReplaceUserProfile_Profile, mm.User), mm.Session, mm.Date) With {.IsSavedPosts = __isSavedPosts}
+                                                If __isSavedPosts Then mm.UserInfo = __user
                                                 Downloader.Files(indx) = mm
                                                 downloaderFilesUpdated = True
                                             End If
@@ -679,9 +705,12 @@ Namespace DownloadObjects
                                             indx = sessionData.FindIndex(finder)
                                             If indx >= 0 Then
                                                 mm = sessionData(indx)
+                                                __user = mm.UserInfo
                                                 mm_data = mm.Data
                                                 mm_data.File = df
-                                                mm = New UserMediaD(mm_data, If(moveOptions.ReplaceUserProfile_Profile, mm.User), mm.Session, mm.Date)
+                                                __isSavedPosts = mm.IsSavedPosts And moveOptions.ReplaceUserProfile_Profile Is Nothing
+                                                mm = New UserMediaD(mm_data, If(moveOptions.ReplaceUserProfile_Profile, mm.User), mm.Session, mm.Date) With {.IsSavedPosts = __isSavedPosts}
+                                                If __isSavedPosts Then mm.UserInfo = __user
                                                 sessionData(indx) = mm
                                                 sesFilesReplaced = True
                                                 If DataList.Count > 0 Then
@@ -860,7 +889,6 @@ Namespace DownloadObjects
                 End If
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "Delete daily sessions (by list)")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
         Private Sub BTT_FEED_DELETE_DAILY_DATE_Click(sender As Object, e As EventArgs) Handles BTT_FEED_DELETE_DAILY_DATE.Click
@@ -898,20 +926,23 @@ Namespace DownloadObjects
                 End Using
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "Delete daily sessions (by date)")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
 #End Region
-#Region "Clear session"
-        Private Sub BTT_CLEAR_DAILY_Click(sender As Object, e As EventArgs) Handles BTT_CLEAR_DAILY.Click
-            If MsgBoxE({"Are you sure you want to clear this session data?", "Clear session"}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then
-                Downloader.Files.Clear()
-                ClearTable()
-                RefillList()
-            End If
+#Region "Sessions set, merge, clear"
+        Private Sub BTT_CURR_SESSION_SET_Click(sender As Object, e As EventArgs) Handles BTT_CURR_SESSION_SET.Click, BTT_CURR_SESSION_SET_LAST.Click
+            Try
+                Dim f As SFile = Nothing
+                SessionChooser(sender Is BTT_CURR_SESSION_SET_LAST,,,, True, f)
+                If Not f.IsEmptyString AndAlso f.Exists Then
+                    Downloader.FilesLoadLastSession(f)
+                    FeedChangeMode(FeedModes.Current)
+                    RefillList(True, False)
+                End If
+            Catch ex As Exception
+                ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "Replace current session")
+            End Try
         End Sub
-#End Region
-#Region "Merge feeds"
         Private Sub BTT_MERGE_SESSIONS_Click(sender As Object, e As EventArgs) Handles BTT_MERGE_SESSIONS.Click
             Try
                 Const msgTitle$ = "Merge feeds"
@@ -972,6 +1003,23 @@ Namespace DownloadObjects
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[DownloadFeedForm.MergeSessions]")
             End Try
         End Sub
+        Private Sub BTT_CLEAR_DAILY_Click(sender As Object, e As EventArgs) Handles BTT_CLEAR_DAILY.Click
+            If MsgBoxE({"Are you sure you want to clear this session data?", "Clear session"}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then
+                Downloader.Files.Clear()
+                ClearTable()
+                RefillList()
+            End If
+        End Sub
+        Private Sub BTT_RESET_DAILY_Click(sender As Object, e As EventArgs) Handles BTT_RESET_DAILY.Click
+            If MsgBoxE({"Are you sure you want to reset the current session?" & vbCr &
+                        "A new file will be created for the current session", "Reset current session"}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then
+                Downloader.ResetSession()
+                FeedChangeMode(FeedModes.Current)
+                RefillList(True, False)
+            End If
+        End Sub
+#End Region
+#Region "Merge feeds"
         Private Sub BTT_MERGE_FEEDS_Click(sender As Object, e As EventArgs) Handles BTT_MERGE_FEEDS.Click
             Try
                 Const msgTitle$ = "Merge feeds"

SCrawler/Download/Feed/FeedMedia.vb

@@ -221,8 +221,8 @@ Namespace DownloadObjects
                     Information = $"Type: {Media.Data.Type}"
                     Information.StringAppendLine($"File: {File.File}")
                     Information.StringAppendLine($"Address: {File}")
-                    Information.StringAppendLine($"Downloaded: {Media.Date.ToStringDate(ADateTime.Formats.BaseDateTime)}")
-                    If Media.Data.Post.Date.HasValue Then Information.StringAppendLine($"Posted: {Media.Data.Post.Date.Value.ToStringDate(ADateTime.Formats.BaseDateTime)}")
+                    Information.StringAppendLine($"Downloaded: {Media.Date.ToStringDateDef}")
+                    If Media.Data.Post.Date.HasValue Then Information.StringAppendLine($"Post date: {Media.Data.Post.Date.Value.ToStringDateDef}")
                     Dim infoType As UserMedia.Types = If(IsSubscription, UserMedia.Types.Picture, Media.Data.Type)
                     Dim h%
                     Dim s As Size
@@ -267,23 +267,27 @@ Namespace DownloadObjects
                             BTT_CONTEXT_OPEN_MEDIA.Text &= " video"
                             BTT_CONTEXT_DELETE.Text &= " video"
                             h = VideoHeight
+                            AddHandler MyVideo.DoubleClick, AddressOf MyPicture_DoubleClick
                         Case Else : Throw New ArgumentNullException With {.HelpLink = 1}
                     End Select
 
-                    Dim info$ = $"[{infoType}] - "
+                    Dim info$ = If(Settings.FeedAddTypeToCaption.Value And Not IsSubscription, $"[{infoType}] - ", String.Empty)
 
                     If Not Media.User Is Nothing Then
                         With Media.User
+                            Dim otherName$ = If(Media.IsSavedPosts, "Saved", Media.UserInfo.Name)
+                            Dim site$ = If(Settings.FeedAddSiteToCaption.Value, $"{ .Site} - ", String.Empty)
+
                             UserKey = .Key
                             Information &= vbNewLine.StringDup(2)
                             If .IncludedInCollection Then Information.StringAppendLine($"User collection: { .CollectionName}")
                             Information.StringAppendLine($"User site: { .Site}")
-                            Information.StringAppendLine($"User name: {IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)}")
+                            Information.StringAppendLine($"User name: {CStr(IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)).IfNullOrEmpty(otherName)}")
                             If .IncludedInCollection Then info &= $"[{ .CollectionName}]: "
                             If Settings.FeedShowFriendlyNames Or Not DirectCast(.Self, UserDataBase).FeedIsUser Then
-                                info &= $"{ .Site} - { .FriendlyName.IfNullOrEmpty(.Name)}"
+                                info &= $"{site}{ .FriendlyName.IfNullOrEmpty(.Name).IfNullOrEmpty(otherName)}"
                             Else
-                                info &= $"{ .Site} - {IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)}"
+                                info &= $"{site}{CStr(IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)).IfNullOrEmpty(otherName)}"
                             End If
                         End With
                     End If

SCrawler/Download/Feed/FeedSpecial.vb

@@ -139,7 +139,6 @@ Namespace DownloadObjects
                 End If
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[FeedSpecial.UpdateUsers]")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
         Friend Overloads Shared Function UpdateUsers(ByVal Item As UserMediaD, ByVal InitialUser As UserInfo, ByVal NewUser As UserInfo,
@@ -148,6 +147,7 @@ Namespace DownloadObjects
             Dim user As IUserData
             Dim path$ = InitialUser.File.CutPath.PathWithSeparator
             Dim pathNew$ = NewUser.File.CutPath.PathWithSeparator
+            Dim isSaved As Boolean = Item.IsSavedPosts
             If Item.UserInfo.Equals(InitialUser) Or Item.UserInfo.Equals(NewUser) Then
                 If Item.Data.File.PathWithSeparator.Contains(path) Then
                     data = Item.Data
@@ -157,8 +157,9 @@ Namespace DownloadObjects
                     Else
                         user = Item.User
                     End If
+                    If isSaved Then isSaved = Item.UserInfo.Equals(DirectCast(user, UserDataBase).User)
                     If Not If(user?.IsSubscription, False) Then
-                        Item = New UserMediaD(data, user, Item.Session, Item.Date)
+                        Item = New UserMediaD(data, user, Item.Session, Item.Date) With {.IsSavedPosts = isSaved}
                         Result = True
                         Return Item
                     End If
@@ -175,8 +176,11 @@ Namespace DownloadObjects
                 If indx >= 0 Then
                     Dim m As UserMediaD = Items(indx)
                     Dim mm As UserMedia = m.Data
+                    Dim user As UserInfo = m.UserInfo
+                    Dim __isSavedPosts As Boolean = m.IsSavedPosts And MCTOptions.ReplaceUserProfile_Profile Is Nothing
                     mm.File = NewFile
-                    m = New UserMediaD(mm, If(MCTOptions.ReplaceUserProfile_Profile, m.User), m.Session, m.Date)
+                    m = New UserMediaD(mm, If(MCTOptions.ReplaceUserProfile_Profile, m.User), m.Session, m.Date) With {.IsSavedPosts = __isSavedPosts}
+                    If __isSavedPosts Then m.UserInfo = user
                     Items(indx) = m
                     _FilesUpdated = True
                 End If
@@ -231,7 +235,62 @@ Namespace DownloadObjects
         Friend Function RemoveNotExist(ByVal p As Predicate(Of UserMediaD)) As Integer
             If Count > 0 And Not _NotExistRemoved Then
                 _NotExistRemoved = True
-                Dim ri% = Items.RemoveAll(p)
+                Dim ri% = 0
+                If Settings.FeedSpecialSearchForMissing Then
+                    Dim i% = -1
+                    Dim d As UserMediaD = Nothing
+                    Dim m As UserMedia
+                    Dim f As SFile = Nothing
+                    Dim ff As SFile
+                    Dim user As IUserData
+                    Dim processRemove As Boolean
+                    Dim userArr As New List(Of IUserData)
+
+                    Dim updateUser As Func(Of IUserData, Boolean) =
+                        Function(ByVal replaceUser As IUserData) As Boolean
+                            If f.Exists(SFO.Path, False) Then
+                                ff = SFile.GetFiles(f, d.Data.File.File, IO.SearchOption.AllDirectories, EDP.ReturnValue).FirstOrDefault
+                                If Not ff.IsEmptyString Then
+                                    m = d.Data
+                                    m.File = ff
+                                    d = New UserMediaD(m, If(replaceUser, d.User), d.Session, d.Date) With {
+                                        .IsSavedPosts = If(replaceUser Is Nothing, d.IsSavedPosts, DirectCast(replaceUser, UserDataBase).IsSavedPosts)
+                                    }
+                                    Items(i) = d
+                                    ri += 1
+                                    processRemove = False
+                                    Return True
+                                End If
+                            End If
+                            Return False
+                        End Function
+
+                    For i = Count - 1 To 0 Step -1
+                        If p.Invoke(Items(i)) Then
+                            d = Items(i)
+                            f = Nothing
+                            ff = Nothing
+                            processRemove = True
+
+                            f = If(d.User?.File, New SFile).IfNullOrEmpty(d.UserInfo.File).CutPath
+                            If updateUser(Nothing) Then Continue For
+
+                            If Settings.FeedSpecialSearchForMissing_Deep Then
+                                If userArr.Count = 0 Then userArr.ListAddList(Settings.GetUsers(Function(u) True))
+                                If userArr.Count > 0 Then
+                                    For Each user In userArr
+                                        f = user.File.CutPath
+                                        If updateUser(user) Then Exit For
+                                    Next
+                                End If
+                            End If
+
+                            If processRemove Then Items.RemoveAt(i) : ri += 1
+                        End If
+                    Next
+                Else
+                    ri = Items.RemoveAll(p)
+                End If
                 If ri > 0 Then Save()
                 Return ri
             Else

SCrawler/Download/Feed/FeedSpecialCollection.vb

@@ -111,7 +111,6 @@ Namespace DownloadObjects
                 End If
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[FeedSpecialCollection.Load]")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
         Private Sub Feeds_FeedDeleted(ByVal Source As FeedSpecialCollection, ByVal Feed As FeedSpecial)
@@ -236,7 +235,7 @@ Namespace DownloadObjects
         Friend Sub UpdateUsers(ByVal InitialUser As UserInfo, ByVal NewUser As UserInfo)
             Try
                 Load()
-                If Count > 0 Then
+                If Count > 0 AndAlso Not UserInfo.ExactEquals(InitialUser, NewUser) Then
                     Feeds.ForEach(Sub(f) f.UpdateUsers(InitialUser, NewUser))
                     If Downloader.Files.Count > 0 Then
                         PendingUsersToUpdate.Add(New KeyValuePair(Of UserInfo, UserInfo)(InitialUser, NewUser))
@@ -245,7 +244,6 @@ Namespace DownloadObjects
                 End If
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[FeedSpecialCollection.UpdateUsers]")
-                MainFrameObj.UpdateLogButton()
             End Try
         End Sub
 #End Region

SCrawler/Download/Feed/FeedVideo.vb

@@ -176,5 +176,9 @@ Namespace DownloadObjects
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, $"An error occurred while performing action [{ActionName}] on file [{MediaFile}]")
             End Try
         End Function
+        Private Sub MyVideo_DoubleClick(sender As Object, e As EventArgs) Handles MyVideo.DoubleClick
+            [Stop]()
+            OnDoubleClick(e)
+        End Sub
     End Class
 End Namespace

SCrawler/Download/Groups/DownloadGroup.vb

@@ -20,6 +20,7 @@ Namespace DownloadObjects.Groups
         Private Const Name_FilterViewMode As String = "FilterViewMode"
         Private Const Name_FilterGroupUsers As String = "FilterGroupUsers"
         Private Const Name_FilterShowGroupsInsteadLabels As String = "FilterShowGroupsInsteadLabels"
+        Private Const Name_FilterShowAllUsers As String = "FilterShowAllUsers"
 #End Region
 #Region "Declarations"
 #Region "Controls"
@@ -36,6 +37,7 @@ Namespace DownloadObjects.Groups
         Friend Property FilterViewMode As ViewModes = ViewModes.IconLarge
         Friend Property FilterGroupUsers As Boolean = True
         Friend Property FilterShowGroupsInsteadLabels As Boolean = True
+        Friend Property FilterShowAllUsers As Boolean = False
 #End Region
         Private File As SFile = Nothing
         Friend Overrides Property Name As String
@@ -144,13 +146,15 @@ Namespace DownloadObjects.Groups
                 FilterViewMode = e.Value(Name_FilterViewMode).FromXML(Of Integer)(ViewModes.IconLarge)
                 FilterGroupUsers = e.Value(Name_FilterGroupUsers).FromXML(Of Boolean)(True)
                 FilterShowGroupsInsteadLabels = e.Value(Name_FilterShowGroupsInsteadLabels).FromXML(Of Boolean)(True)
+                FilterShowAllUsers = e.Value(Name_FilterShowAllUsers).FromXML(Of Boolean)(False)
             End If
         End Sub
         Protected Overrides Function Export(ByVal e As EContainer) As EContainer
             MyBase.Export(e)
             e.AddRange({New EContainer(Name_FilterViewMode, CInt(FilterViewMode)),
                         New EContainer(Name_FilterGroupUsers, FilterGroupUsers.BoolToInteger),
-                        New EContainer(Name_FilterShowGroupsInsteadLabels, FilterShowGroupsInsteadLabels.BoolToInteger)})
+                        New EContainer(Name_FilterShowGroupsInsteadLabels, FilterShowGroupsInsteadLabels.BoolToInteger),
+                        New EContainer(Name_FilterShowAllUsers, FilterShowAllUsers.BoolToInteger)})
             Return e
         End Function
 #End Region
@@ -166,6 +170,7 @@ Namespace DownloadObjects.Groups
                         FilterViewMode = .FilterViewMode
                         FilterGroupUsers = .FilterGroupUsers
                         FilterShowGroupsInsteadLabels = .FilterShowGroupsInsteadLabels
+                        FilterShowAllUsers = .FilterShowAllUsers
                     End If
                 End With
             End If