2023.12.7.0

Update updater
YT: add new version check at start

CONTRIBUTING.md

@@ -39,5 +39,4 @@ I welcome requests! Follow these steps to contribute:
 If I'm interested in a site you want to add, it may be added in future releases.
 
 # Sites I will never develop
-- Facebook
 - Tumblr

Changelog.md

@@ -1,3 +1,80 @@
+# 2023.12.7.0
+
+*2023-12-07*
+
+- Added
+  - Saved posts: add downloaded saved posts to the feed
+  - **YouTube (SCrawler): the ability to download YouTube user community feeds**
+  - Main window: add `Alt+A` hotkey to show scheduler
+  - Main window: add `Alt+P` hotkey to show progress form
+  - YouTube: check of adding a URL if it has already been downloaded
+  - YouTube: ability to check for a new version at start
+  - **Updater**
+- Fixed
+  - Standalone downloader: URL files are not deleted along with the file
+  - Minor bugs
+
+# 2023.11.25.0
+
+*2023-11-25*
+
+- Fixed
+  - Reddit: missing refresh token button in the settings form
+
+# 2023.11.24.0
+
+*2023-11-24*
+
+For those of you who use TikTok, I recommend updating [TikTok plugin](https://github.com/bashonly/yt-dlp-TTUser) to the latest version using [these instructions](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-install-yt-dlp-ttuser-plugin).
+
+- Added
+  - Automation: manual task option
+  - Scheduler: add scheduler name to form title
+  - Feeds: update when users' location and/or basic information changes
+  - Reddit: special notification for error 429
+  - TikTok: ID, username and friendly name extraction from data
+  - TikTok: new option `Use video date as file date`
+  - YouTube: absolute path for a single playlist
+- Updated
+  - yt-dlp up to version 2023.11.16
+- Fixed
+  - Scheduler: scheduler change error
+  - Twitter: JSON deserialization error
+  - xHamster, XVideos, PornHub, ThisVid: incorrect parsing of search queries
+  - YouTube: the file name is not changed manually
+  - YouTube: path not set when adding array to download
+  - Minor bugs
+
+# 2023.11.17.0
+
+*2023-11-17*
+
+- Added
+  - **Facebook**
+  - **Multi-account**
+  - **Special feeds**
+  - Site settings: option `Download saved posts`
+  - Standalone downloader: support for multiple account
+  - PornHub: add playlists downloading
+  - YouTube: ability to download subtitles **and** `CC` if they both exists
+  - Other improvements
+- PluginProvider
+  - `IDownloadableMedia`: added `AccountName` property
+  - `IPluginContentProvider`: added `AccountName` property
+  - `ISiteSettings`: added properties: `AccountName`, `Temporary`, `AvailableText`, `DefaultInstance`; added functions: `Clone`, `Update`, `Delete`; removed `Load` function; implement `IDisposable` interface
+  - `PropertyValue`: added functions: `BeginInit`, `EndInit`, `Clone`
+  - `Attributes.DoNotUse` - add `Value` field
+- Fixed
+  - Instagram: handling 401 error
+  - OnlyFans: handling 401 error
+  - xHamster: handling 503 error
+  - xHamster: incorrect parsing of search queries
+  - XVideos: incorrect parsing of search queries
+  - ThisVid: incorrect parsing of search queries
+  - PornHub: incorrect parsing of search queries
+  - Automation: handle automation start error (in some cases) when changing scheduler
+  - Minor bugs
+
 # 2023.10.10.0
 
 *2023-10-10*

FAQ.md

@@ -113,4 +113,8 @@ A: I can only [suggest](#q-you-lost-me-your-program-is-too-complicated) you find
 
 #### Q: **Can you add a step-by-step guide or video on how to use the program?**
 
-A: **NO! NEVER!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.
+A: **NO! NEVER!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.
+
+**The following video shows how to add credentials:**
+
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)

Plugins.md

@@ -1,3 +1,7 @@
+You can create a plugin for any site you want. **To create a plugin, read [this guide](https://github.com/AAndyProgram/SCrawler/wiki/Plugins).**
+
+If you've created a plugin, you can create a [new issue](https://github.com/AAndyProgram/SCrawler/issues/new?assignees=&labels=New+Plugin&projects=&template=plugin_add.md&title=%5BNEW+PLUGIN%5D) and I'll add your plugin to the list below.
+
 List of available plugins:
 
 Tools:

README.md

@@ -12,7 +12,7 @@
 :eu:
 :greece:
 
-A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
+A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, Facebook, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
 
 **If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
 <!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
@@ -30,7 +30,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 
 # What can program do:
 - Download pictures and videos from users' profiles and subreddits:
-  - YouTube videos, shorts, users, artists, playlists, music, tracks;
+  - YouTube videos, shorts, community feeds, users, artists, playlists, music, tracks;
   - Reddit images, galleries of images, videos, saved posts;
   - Redgifs videos (https://www.redgifs.com/);
   - Twitter images and videos, saved (bookmarked) posts;
@@ -39,6 +39,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
   - Mastodon images and videos, saved (bookmarked) posts;
   - Instagram images and videos, tagged posts, stories, saved posts;
   - Threads images and videos;
+  - Facebook images and videos, saved posts;
   - TikTok videos;
   - Pinterest boards, users, saved posts;
   - Imgur images, galleries and videos;
@@ -54,6 +55,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **Advanced user management**
 - **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every ```X``` minutes)
 - **Feed** ([feed](https://github.com/AAndyProgram/SCrawler/wiki#feed) of downloaded media files and subscriptions posts)
+- Multiple accounts support
 - Labeling users
 - Create [download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
 - Adding users to favorites and temporary
@@ -76,6 +78,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **Mastodon**
 - **Instagram**
 - **Threads**
+- **Facebook**
 - JustForFans
 - TikTok
 - RedGifs
@@ -128,6 +131,7 @@ First, the program downloads the full profile. After the program downloads only
   - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
   - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
   - [Threads](https://github.com/AAndyProgram/SCrawler/wiki/Settings#threads)
+  - [Facebook](https://github.com/AAndyProgram/SCrawler/wiki/Settings#facebook)
   - [JustForFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#justforfans)
   - [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
   - [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)
@@ -141,6 +145,10 @@ First, the program downloads the full profile. After the program downloads only
 
 **Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
 
+**Video on how to configure**
+
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
+
 # Installation
 
 **Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder and enjoy.** :blush:
@@ -163,6 +171,8 @@ The program has an intuitive interface.
 
 **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
+
 Just add a user profile and **click the ```Download``` button**.
 
 ```mermaid
@@ -195,7 +205,6 @@ F5-->[*]
 
 Discord server: https://discord.gg/uFNUXvFFmg
 
-[e-mail](mailto:andyprogram@proton.me): andyprogram@proton.me
 <!--
 [e-mail](mailto:andyprogram@proton.me): andyprogram@proton.me
 

SCrawler.PluginProvider/.editorconfig

@@ -1,3 +1,3 @@
 [*.vb]
 # Modifier preferences
-file_header_template = Copyright (C) 2023  Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.PluginProvider/Attributes/Attributes.vb

@@ -65,6 +65,12 @@ Namespace Plugin.Attributes
     End Class
     ''' <summary>Attribute to disable some properties for host use</summary>
     <AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class DoNotUse : Inherits Attribute
+        Public ReadOnly Value As Boolean = True
+        Public Sub New()
+        End Sub
+        Public Sub New(ByVal Value As Boolean)
+            Me.Value = Value
+        End Sub
     End Class
     ''' <summary>Special property updater</summary>
     <AttributeUsage(AttributeTargets.Method, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class PropertyUpdater : Inherits Attribute

SCrawler.PluginProvider/Interfaces/IDownloadableMedia.vb

@@ -14,6 +14,7 @@ Namespace Plugin
         ReadOnly Property SiteIcon As Drawing.Image
         ReadOnly Property Site As String
         ReadOnly Property SiteKey As String
+        Property AccountName As String
         Property ThumbnailUrl As String
         Property ThumbnailFile As String
         Property Title As String

SCrawler.PluginProvider/Interfaces/IPluginContentProvider.vb

@@ -15,6 +15,7 @@ Namespace Plugin
         Property Thrower As IThrower
         Property LogProvider As ILogProvider
         Property Settings As ISiteSettings
+        Property AccountName As String
         Property Name As String
         Property ID As String
         Property Options As String

SCrawler.PluginProvider/Interfaces/ISiteSettings.vb

@@ -8,7 +8,7 @@
 ' but WITHOUT ANY WARRANTY
 Imports System.Drawing
 Namespace Plugin
-    Public Interface ISiteSettings
+    Public Interface ISiteSettings : Inherits IDisposable
         Enum Download As Integer
             Main = 0
             SavedPosts = 1
@@ -17,6 +17,9 @@ Namespace Plugin
         ReadOnly Property Icon As Icon
         ReadOnly Property Image As Image
         ReadOnly Property Site As String
+        Property AccountName As String
+        Property Temporary As Boolean
+        Property DefaultInstance As ISiteSettings
         ReadOnly Property SubscriptionsAllowed As Boolean
         Property Logger As ILogProvider
         Function GetUserUrl(ByVal User As IPluginContentProvider) As String
@@ -25,9 +28,6 @@ Namespace Plugin
         Function GetInstance(ByVal What As Download) As IPluginContentProvider
         Function GetSingleMediaInstance(ByVal URL As String, ByVal OutputFile As String) As IDownloadableMedia
         Function GetUserPostUrl(ByVal User As IPluginContentProvider, ByVal Media As IUserMedia) As String
-#Region "XML Support"
-        Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String)))
-#End Region
 #Region "Initialization"
         Sub BeginInit()
         Sub EndInit()
@@ -37,6 +37,7 @@ Namespace Plugin
         Sub EndEdit()
 #End Region
 #Region "Site availability"
+        Property AvailableText As String
         Function Available(ByVal What As Download, ByVal Silent As Boolean) As Boolean
         Function ReadyToDownload(ByVal What As Download) As Boolean
 #End Region
@@ -46,7 +47,10 @@ Namespace Plugin
         Sub AfterDownload(ByVal User As Object, ByVal What As Download)
         Sub DownloadDone(ByVal What As Download)
 #End Region
-        Sub Update()
+        Function Clone(ByVal Full As Boolean) As ISiteSettings
+        Sub Delete()
+        Overloads Sub Update()
+        Overloads Sub Update(ByVal Source As ISiteSettings)
         Sub Reset()
         Sub OpenSettingsForm()
         Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)

SCrawler.PluginProvider/My Project/AssemblyInfo.vb

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
 <Assembly: AssemblyDescription("Plugin provider for SCrawler")>
 <Assembly: AssemblyCompany("AndyProgram")>
 <Assembly: AssemblyProduct("SCrawler.PluginProvider")>
-<Assembly: AssemblyCopyright("Copyright © 2023")>
+<Assembly: AssemblyCopyright("Copyright © 2024")>
 <Assembly: AssemblyTrademark("AndyProgram")>
 
 <Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.10.1.0")>
-<Assembly: AssemblyFileVersion("2023.10.1.0")>
+<Assembly: AssemblyVersion("2023.11.24.0")>
+<Assembly: AssemblyFileVersion("2023.11.24.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.PluginProvider/Objects/PropertyValue.vb

@@ -11,6 +11,7 @@ Namespace Plugin
         Public Event ValueChanged As IPropertyValue.ValueChangedEventHandler Implements IPropertyValue.ValueChanged
         Public Property [Type] As Type Implements IPropertyValue.Type
         Public Property OnChangeFunction As IPropertyValue.ValueChangedEventHandler
+        Private _Initialization As Boolean = False
         ''' <inheritdoc cref="PropertyValue.New(Object, Type, ByRef IPropertyValue.ValueChangedEventHandler)"/>
         ''' <exception cref="ArgumentNullException"></exception>
         Public Sub New(ByVal InitValue As Object)
@@ -41,10 +42,25 @@ Namespace Plugin
             End Get
             Set(ByVal NewValue As Object)
                 _Value = NewValue
-                If Not OnChangeFunction Is Nothing Then OnChangeFunction.Invoke(Value)
-                RaiseEvent ValueChanged(_Value)
+                If Not _Initialization Then
+                    If Not OnChangeFunction Is Nothing Then OnChangeFunction.Invoke(Value)
+                    RaiseEvent ValueChanged(_Value)
+                End If
             End Set
         End Property
+        Public Sub BeginInit()
+            _Initialization = True
+        End Sub
+        Public Sub EndInit()
+            _Initialization = False
+        End Sub
+        Public Sub Clone(ByVal Source As PropertyValue)
+            _Initialization = True
+            Type = Source.Type
+            OnChangeFunction = Source.OnChangeFunction
+            _Value = Source._Value
+            _Initialization = False
+        End Sub
     End Class
     Public Interface IPropertyValue
         ''' <summary>Event for internal exchange</summary>

SCrawler.Shared/.editorconfig

@@ -0,0 +1,3 @@
+[*.vb]
+# Modifier preferences
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.Shared/Functions.vb

@@ -0,0 +1,33 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace [Shared]
+    Public Module Functions
+        Public Const NewReleaseFolderName As String = "__NewRelease"
+        Public Function GetCurrentMaxVer(Optional ByVal Path As SFile = Nothing) As Version
+            Try
+                If Path.IsEmptyString Then Path = Application.StartupPath.CSFileP
+                If Path.Exists(SFO.Path, False) Then
+                    Dim versions As New List(Of Version)
+                    Dim v As FileVersionInfo
+                    With SFile.GetFiles(Path, "*.exe",, EDP.ReturnValue).ListIfNothing.Where(Function(f) f.Name = "SCrawler" Or f.Name = "YouTubeDownloader")
+                        If .ListExists Then
+                            For Each f As SFile In .Self
+                                v = FileVersionInfo.GetVersionInfo(f)
+                                versions.Add(New Version(v.ProductVersion))
+                            Next
+                        End If
+                    End With
+                    If versions.Count > 0 Then Return versions.LastOrDefault
+                End If
+            Catch
+            End Try
+            Return Nothing
+        End Function
+    End Module
+End Namespace

SCrawler.Shared/My Project/Application.Designer.vb

@@ -0,0 +1,13 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+

SCrawler.Shared/My Project/Application.myapp

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="utf-8"?>
+<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <MySubMain>false</MySubMain>
+  <SingleInstance>false</SingleInstance>
+  <ShutdownMode>0</ShutdownMode>
+  <EnableVisualStyles>true</EnableVisualStyles>
+  <AuthenticationMode>0</AuthenticationMode>
+  <ApplicationType>1</ApplicationType>
+  <SaveMySettingsOnExit>true</SaveMySettingsOnExit>
+</MyApplicationData>

SCrawler.Shared/My Project/AssemblyInfo.vb

@@ -0,0 +1,37 @@
+Imports System.Resources
+Imports System
+Imports System.Reflection
+Imports System.Runtime.InteropServices
+
+' General Information about an assembly is controlled through the following
+' set of attributes. Change these attribute values to modify the information
+' associated with an assembly.
+
+' Review the values of the assembly attributes
+
+<Assembly: AssemblyTitle("SCrawler.Shared")>
+<Assembly: AssemblyDescription("SCrawler shared functions")>
+<Assembly: AssemblyCompany("AndyProgram")>
+<Assembly: AssemblyProduct("SCrawler.Shared")>
+<Assembly: AssemblyCopyright("Copyright © 2024")>
+<Assembly: AssemblyTrademark("AndyProgram")>
+
+<Assembly: ComVisible(False)>
+
+'The following GUID is for the ID of the typelib if this project is exposed to COM
+<Assembly: Guid("75a22c7c-6f90-49cb-852b-078ea0b8f2b6")>
+
+' Version information for an assembly consists of the following four values:
+'
+'      Major Version
+'      Minor Version
+'      Build Number
+'      Revision
+'
+' You can specify all the values or you can default the Build and Revision Numbers
+' by using the '*' as shown below:
+' <Assembly: AssemblyVersion("1.0.*")>
+
+<Assembly: AssemblyVersion("2023.12.7.0")>
+<Assembly: AssemblyFileVersion("2023.12.7.0")>
+<Assembly: NeutralResourcesLanguage("en")>

SCrawler.Shared/My Project/Resources.Designer.vb

@@ -0,0 +1,63 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+Imports System
+
+Namespace My.Resources
+    
+    'This class was auto-generated by the StronglyTypedResourceBuilder
+    'class via a tool like ResGen or Visual Studio.
+    'To add or remove a member, edit your .ResX file then rerun ResGen
+    'with the /str option, or rebuild your VS project.
+    '''<summary>
+    '''  A strongly-typed resource class, for looking up localized strings, etc.
+    '''</summary>
+    <Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.Microsoft.VisualBasic.HideModuleNameAttribute()>  _
+    Friend Module Resources
+        
+        Private resourceMan As Global.System.Resources.ResourceManager
+        
+        Private resourceCulture As Global.System.Globalization.CultureInfo
+        
+        '''<summary>
+        '''  Returns the cached ResourceManager instance used by this class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
+            Get
+                If Object.ReferenceEquals(resourceMan, Nothing) Then
+                    Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("SCrawler.Resources", GetType(Resources).Assembly)
+                    resourceMan = temp
+                End If
+                Return resourceMan
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Overrides the current thread's CurrentUICulture property for all
+        '''  resource lookups using this strongly typed resource class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend Property Culture() As Global.System.Globalization.CultureInfo
+            Get
+                Return resourceCulture
+            End Get
+            Set
+                resourceCulture = value
+            End Set
+        End Property
+    End Module
+End Namespace

SCrawler.Shared/My Project/Resources.resx

@@ -0,0 +1,117 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+</root>

SCrawler.Shared/My Project/Settings.Designer.vb

@@ -0,0 +1,73 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+
+Namespace My
+    
+    <Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"),  _
+     Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+    Partial Friend NotInheritable Class MySettings
+        Inherits Global.System.Configuration.ApplicationSettingsBase
+        
+        Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
+        
+#Region "My.Settings Auto-Save Functionality"
+#If _MyType = "WindowsForms" Then
+    Private Shared addedHandler As Boolean
+
+    Private Shared addedHandlerLockObject As New Object
+
+    <Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
+    Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
+        If My.Application.SaveMySettingsOnExit Then
+            My.Settings.Save()
+        End If
+    End Sub
+#End If
+#End Region
+        
+        Public Shared ReadOnly Property [Default]() As MySettings
+            Get
+                
+#If _MyType = "WindowsForms" Then
+               If Not addedHandler Then
+                    SyncLock addedHandlerLockObject
+                        If Not addedHandler Then
+                            AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
+                            addedHandler = True
+                        End If
+                    End SyncLock
+                End If
+#End If
+                Return defaultInstance
+            End Get
+        End Property
+    End Class
+End Namespace
+
+Namespace My
+    
+    <Global.Microsoft.VisualBasic.HideModuleNameAttribute(),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()>  _
+    Friend Module MySettingsProperty
+        
+        <Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")>  _
+        Friend ReadOnly Property Settings() As Global.SCrawler.My.MySettings
+            Get
+                Return Global.SCrawler.My.MySettings.Default
+            End Get
+        End Property
+    End Module
+End Namespace

SCrawler.Shared/My Project/Settings.settings

@@ -0,0 +1,7 @@
+<?xml version='1.0' encoding='utf-8'?>
+<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
+  <Profiles>
+    <Profile Name="(Default)" />
+  </Profiles>
+  <Settings />
+</SettingsFile>

SCrawler.Shared/SCrawler.Shared.vbproj

@@ -0,0 +1,151 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
+  <PropertyGroup>
+    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
+    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
+    <ProjectGuid>{DC634700-24C7-42DD-BF8F-87E6CC54E625}</ProjectGuid>
+    <OutputType>Library</OutputType>
+    <RootNamespace>SCrawler</RootNamespace>
+    <AssemblyName>SCrawler.Shared</AssemblyName>
+    <FileAlignment>512</FileAlignment>
+    <MyType>Windows</MyType>
+    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
+    <Deterministic>true</Deterministic>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
+    <DebugSymbols>true</DebugSymbols>
+    <DebugType>full</DebugType>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\Debug\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
+    <DebugType>pdbonly</DebugType>
+    <DefineDebug>false</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <Optimize>true</Optimize>
+    <OutputPath>bin\Release\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionExplicit>On</OptionExplicit>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionCompare>Binary</OptionCompare>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionStrict>Off</OptionStrict>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionInfer>On</OptionInfer>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Debug\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Release\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Debug\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Release\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <ItemGroup>
+    <Reference Include="System" />
+    <Reference Include="System.Data" />
+    <Reference Include="System.Windows.Forms" />
+    <Reference Include="System.Xml" />
+    <Reference Include="System.Core" />
+    <Reference Include="System.Xml.Linq" />
+    <Reference Include="System.Data.DataSetExtensions" />
+    <Reference Include="System.Net.Http" />
+  </ItemGroup>
+  <ItemGroup>
+    <Import Include="Microsoft.VisualBasic" />
+    <Import Include="PersonalUtilities.Functions" />
+    <Import Include="System" />
+    <Import Include="System.Collections" />
+    <Import Include="System.Collections.Generic" />
+    <Import Include="System.Data" />
+    <Import Include="System.Diagnostics" />
+    <Import Include="System.Linq" />
+    <Import Include="System.Windows.Forms" />
+    <Import Include="System.Xml.Linq" />
+    <Import Include="System.Threading.Tasks" />
+  </ItemGroup>
+  <ItemGroup>
+    <Compile Include="Functions.vb" />
+    <Compile Include="My Project\AssemblyInfo.vb" />
+    <Compile Include="My Project\Application.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Application.myapp</DependentUpon>
+      <DesignTime>True</DesignTime>
+    </Compile>
+    <Compile Include="My Project\Resources.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DesignTime>True</DesignTime>
+      <DependentUpon>Resources.resx</DependentUpon>
+    </Compile>
+    <Compile Include="My Project\Settings.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Settings.settings</DependentUpon>
+      <DesignTimeSharedInput>True</DesignTimeSharedInput>
+    </Compile>
+  </ItemGroup>
+  <ItemGroup>
+    <EmbeddedResource Include="My Project\Resources.resx">
+      <Generator>VbMyResourcesResXFileCodeGenerator</Generator>
+      <LastGenOutput>Resources.Designer.vb</LastGenOutput>
+      <CustomToolNamespace>My.Resources</CustomToolNamespace>
+      <SubType>Designer</SubType>
+    </EmbeddedResource>
+  </ItemGroup>
+  <ItemGroup>
+    <None Include=".editorconfig" />
+    <None Include="My Project\Application.myapp">
+      <Generator>MyApplicationCodeGenerator</Generator>
+      <LastGenOutput>Application.Designer.vb</LastGenOutput>
+    </None>
+    <None Include="My Project\Settings.settings">
+      <Generator>SettingsSingleFileGenerator</Generator>
+      <CustomToolNamespace>My</CustomToolNamespace>
+      <LastGenOutput>Settings.Designer.vb</LastGenOutput>
+    </None>
+  </ItemGroup>
+  <ItemGroup>
+    <ProjectReference Include="..\..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj">
+      <Project>{8405896b-2685-4916-bc93-1fb514c323a9}</Project>
+      <Name>PersonalUtilities</Name>
+    </ProjectReference>
+  </ItemGroup>
+  <Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
+</Project>

SCrawler.Updater/.editorconfig

@@ -0,0 +1,3 @@
+[*.vb]
+# Modifier preferences
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.Updater/App.config

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<configuration>
+    <startup> 
+        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1" />
+    </startup>
+</configuration>

SCrawler.Updater/MainMod.vb

@@ -0,0 +1,229 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Net
+Imports System.IO.Compression
+Imports PersonalUtilities.Functions
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports SCrawler.Shared
+Public Module MainMod
+    Private MyProcessID As Integer = -1
+    Private MyWorkingPath As SFile = Nothing
+    Private ReadOnly ProcessNames As String() = {"SCrawler", "YouTubeDownloader", "Updater"}
+    Private Silent As Boolean = False
+    Private Function GetConsoleResponse(ByVal Request As String) As String
+        Console.Write(Request)
+        Return Console.ReadLine
+    End Function
+    Private Function DownloadFile(ByVal URL As String, ByVal Destination As SFile) As Boolean
+        Try
+            Dim lastPerc% = -1
+            Dim currentCursor% = Console.CursorTop
+            Using w As New RWebClient With {.AsyncMode = True}
+                AddHandler w.DownloadProgressChanged,
+                           New DownloadProgressChangedEventHandler(Sub(ByVal Sender As Object, ByVal e As DownloadProgressChangedEventArgs)
+                                                                       If lastPerc < e.ProgressPercentage Then
+                                                                           lastPerc = e.ProgressPercentage
+                                                                           Console.SetCursorPosition(0, currentCursor)
+                                                                           Console.Write("{0}% completed", e.ProgressPercentage)
+                                                                       End If
+                                                                   End Sub)
+                Return w.DownloadFile(URL, Destination, EDP.ReturnValue)
+            End Using
+        Catch ex As Exception
+            Return False
+        End Try
+    End Function
+    Public Sub Main()
+        Try
+            MyWorkingPath = AppDomain.CurrentDomain.BaseDirectory.CSFileP
+            MyProcessID = Process.GetCurrentProcess.Id
+            Console.Title = "SCrawler updater"
+            With Environment.GetCommandLineArgs
+                If .ListExists(2) Then Silent = .Self()(1).FromXML(Of Boolean)(False)
+            End With
+
+            Dim currentDir As SFile = MyWorkingPath.CutPath
+            Dim extractionDir As SFile = $"{currentDir.CSFilePS}{NewReleaseFolderName}\"
+            If extractionDir.Exists(SFO.Path, False) Then extractionDir.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.None)
+
+            Dim currVer As Version = GetCurrentMaxVer(currentDir)
+            If currVer Is Nothing Then
+                Console.WriteLine("The current version of the program cannot be determined")
+            Else
+                Console.WriteLine($"The current version is {currVer} (x{IIf(Environment.Is64BitProcess, 64, 86)})")
+                Dim release As GitRelease = GetGitRelease()
+                If Not release.URL.IsEmptyString And Not release.Version Is Nothing Then
+                    If release.Version > currVer Then
+                        Console.WriteLine($"The new version is {release.Version} ({release.Name})")
+
+                        If Not Silent AndAlso GetConsoleResponse("Do you want to update the program? (y/n): ").IfNullOrEmpty("n") = "n" Then Exit Sub
+
+                        If ActiveProcessesExist() Then
+                            Console.WriteLine("One of the SCrawler programs is still running. Waiting for all SCrawler programs to close.")
+                            While ActiveProcessesExist() : Threading.Thread.Sleep(100) : End While
+                            Console.WriteLine("All SCrawler programs are closed.")
+                        End If
+
+                        If extractionDir.Exists(SFO.Path, True) Then
+                            Dim destFile As SFile = $"{extractionDir.CSFilePS}{New SFile(release.URL).File}"
+                            Console.WriteLine("Downloading new version...")
+                            If DownloadFile(release.URL, destFile) Then
+                                Console.WriteLine("")
+                                Console.WriteLine("New version downloaded!")
+                                Console.WriteLine("Extracting files...")
+                                ZipFile.ExtractToDirectory(destFile, extractionDir)
+                                Console.WriteLine("Files extracted!")
+                                destFile.Delete(SFO.File, SFODelete.DeletePermanently, EDP.None)
+                                If Not MoveFiles(extractionDir, currentDir) Then GetConsoleResponse("Unable to update the program. Press Enter to exit") : Exit Sub
+                            Else
+                                extractionDir.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.None)
+                                Console.WriteLine("Unable to download new version")
+                            End If
+                        Else
+                            Console.WriteLine("Unable to create temp directory")
+                        End If
+                    Else
+                        Console.WriteLine("The program is up to date")
+                    End If
+                Else
+                    Console.WriteLine("Unable to get information about new version")
+                End If
+            End If
+        Catch ex As Exception
+            Console.WriteLine("An error occurred during update")
+            Console.WriteLine(ex.Message)
+        Finally
+            GetConsoleResponse("Press Enter to exit")
+        End Try
+    End Sub
+    Private Function MoveFiles(ByVal Source As SFile, ByVal Destination As SFile) As Boolean
+        Console.WriteLine("Updating files")
+        Try
+
+            Dim oldFiles As List(Of SFile) = SFile.GetFiles(Destination,,, EDP.ReturnValue)
+            Dim oldFolders As List(Of SFile) = SFile.GetDirectories(Destination,,, EDP.ReturnValue)
+
+            Dim newFiles As List(Of SFile) = SFile.GetFiles(Source,,, EDP.ReturnValue)
+            Dim newFolders As List(Of SFile) = SFile.GetDirectories(Source,,, EDP.ReturnValue)
+
+            Dim obj As SFile = Nothing
+            Dim wSegment As String = MyWorkingPath.Segments.Last
+            Dim filesPredicate As Predicate(Of SFile) = Function(ByVal f As SFile) As Boolean
+                                                            If obj = f Or obj.Name = f.Name Then
+                                                                f.Delete(SFO.File, SFODelete.DeleteToRecycleBin, EDP.None)
+                                                                Return True
+                                                            Else
+                                                                Return False
+                                                            End If
+                                                        End Function
+            Dim foldersPredicate As Predicate(Of SFile) = Function(ByVal f As SFile) As Boolean
+                                                              Dim ls$ = f.Segments.Last
+                                                              If ls = obj.Segments.Last And Not ls = NewReleaseFolderName And Not ls = wSegment Then
+                                                                  f.Delete(SFO.Path, SFODelete.DeleteToRecycleBin, EDP.None)
+                                                                  Return True
+                                                              Else
+                                                                  Return False
+                                                              End If
+                                                          End Function
+
+            Dim getDestFile As Func(Of SFile, Boolean, SFile) = Function(ByVal f As SFile, ByVal isFolder As Boolean) As SFile
+                                                                    Dim ff As SFile = f
+                                                                    If isFolder Then
+                                                                        ff = $"{Destination.PathWithSeparator}{f.Segments.Last}\"
+                                                                    Else
+                                                                        ff.Path = Destination.Path
+                                                                    End If
+                                                                    Console.WriteLine(ff)
+                                                                    Return ff
+                                                                End Function
+            If newFiles.ListExists Then
+                If oldFiles.ListExists Then
+                    For Each obj In newFiles : oldFiles.RemoveAll(filesPredicate) : Next
+                End If
+                newFiles.ForEach(Sub(ff) SFile.Move(ff, getDestFile(ff, False), SFO.File, True, SFODelete.DeleteToRecycleBin, EDP.None))
+            End If
+
+            If newFolders.ListExists Then
+                If oldFolders.ListExists Then
+                    For Each obj In newFolders : oldFolders.RemoveAll(foldersPredicate) : Next
+                End If
+                newFolders.ForEach(Sub(ff) If Not ff.Segments.Last = wSegment Then _
+                                              SFile.Move(ff, getDestFile(ff, True), SFO.Path, True, SFODelete.DeleteToRecycleBin, EDP.None))
+            End If
+
+            Console.WriteLine("Files updated")
+            Return True
+        Catch
+            Return False
+        End Try
+    End Function
+    Private Function ActiveProcessesExist() As Boolean
+        Try
+            Return Process.GetProcesses.Any(Function(p) ProcessNames.Contains(p.ProcessName) And Not p.Id = MyProcessID)
+        Catch
+            Return True
+        End Try
+    End Function
+    Private Structure GitRelease
+        Friend URL As String
+        Friend Name As String
+        Friend Version As Version
+    End Structure
+    Private Function GetGitRelease() As GitRelease
+        Try
+            Dim nameEnd$ = $"_x{IIf(Environment.Is64BitProcess, 64, 86)}.zip"
+            Dim name$, relName$, relTag$
+
+            Using resp As New Responser With {.Accept = "application/vnd.github.v3+json"}
+                Dim r$ = resp.GetResponse("https://api.github.com/repos/AAndyProgram/SCrawler/releases",, EDP.ReturnValue)
+                If Not r.IsEmptyString Then
+                    Dim getver As Func(Of String, Version) = Function(ByVal input As String) As Version
+                                                                 Try
+                                                                     If Not input.IsEmptyString Then
+                                                                         If input.ToLower.StartsWith("scrawler") Then
+                                                                             Return New Version(input.Split("_")(1))
+                                                                         Else
+                                                                             Return New Version(input)
+                                                                         End If
+                                                                     End If
+                                                                 Catch
+                                                                 End Try
+                                                                 Return Nothing
+                                                             End Function
+                    Using j As EContainer = JsonDocument.Parse(r, EDP.ReturnValue)
+                        If j.ListExists Then
+                            With j.FirstOrDefault(Function(e) Not e.Value("draft").FromXML(Of Boolean) And Not e.Value("prerelease").FromXML(Of Boolean))
+                                If .ListExists Then
+                                    relName = .Value("name")
+                                    relTag = .Value("tag_name")
+                                    With .Item("assets")
+                                        If .ListExists Then
+                                            For Each asset As EContainer In .Self
+                                                name = asset.Value("name")
+                                                If Not name.IsEmptyString AndAlso name.EndsWith(nameEnd) Then _
+                                                   Return New GitRelease With {
+                                                    .Name = name,
+                                                    .URL = asset.Value("browser_download_url"),
+                                                    .Version = getver(name).IfNullOrEmpty(getver(relName).IfNullOrEmpty(getver(relTag)))}
+                                            Next
+                                        End If
+                                    End With
+                                End If
+                            End With
+                        End If
+                    End Using
+                End If
+            End Using
+        Catch
+        End Try
+        Return Nothing
+    End Function
+End Module

SCrawler.Updater/My Project/Application.Designer.vb

@@ -0,0 +1,13 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+

SCrawler.Updater/My Project/Application.myapp

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="utf-8"?>
+<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <MySubMain>false</MySubMain>
+  <SingleInstance>false</SingleInstance>
+  <ShutdownMode>0</ShutdownMode>
+  <EnableVisualStyles>true</EnableVisualStyles>
+  <AuthenticationMode>0</AuthenticationMode>
+  <ApplicationType>2</ApplicationType>
+  <SaveMySettingsOnExit>true</SaveMySettingsOnExit>
+</MyApplicationData>

SCrawler.Updater/My Project/AssemblyInfo.vb

@@ -0,0 +1,37 @@
+Imports System.Resources
+Imports System
+Imports System.Reflection
+Imports System.Runtime.InteropServices
+
+' General Information about an assembly is controlled through the following
+' set of attributes. Change these attribute values to modify the information
+' associated with an assembly.
+
+' Review the values of the assembly attributes
+
+<Assembly: AssemblyTitle("SCrawler updater")>
+<Assembly: AssemblyDescription("SCrawler updater")>
+<Assembly: AssemblyCompany("AndyProgram")>
+<Assembly: AssemblyProduct("SCrawler.Updater")>
+<Assembly: AssemblyCopyright("Copyright © 2024")>
+<Assembly: AssemblyTrademark("AndyProgram")>
+
+<Assembly: ComVisible(False)>
+
+'The following GUID is for the ID of the typelib if this project is exposed to COM
+<Assembly: Guid("df008b29-ad5e-4271-acfe-650396d15c40")>
+
+' Version information for an assembly consists of the following four values:
+'
+'      Major Version
+'      Minor Version
+'      Build Number
+'      Revision
+'
+' You can specify all the values or you can default the Build and Revision Numbers
+' by using the '*' as shown below:
+' <Assembly: AssemblyVersion("1.0.*")>
+
+<Assembly: AssemblyVersion("2023.12.7.0")>
+<Assembly: AssemblyFileVersion("2023.12.7.0")>
+<Assembly: NeutralResourcesLanguage("en")>

SCrawler.Updater/My Project/Resources.Designer.vb

@@ -0,0 +1,73 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+Imports System
+
+Namespace My.Resources
+    
+    'This class was auto-generated by the StronglyTypedResourceBuilder
+    'class via a tool like ResGen or Visual Studio.
+    'To add or remove a member, edit your .ResX file then rerun ResGen
+    'with the /str option, or rebuild your VS project.
+    '''<summary>
+    '''  A strongly-typed resource class, for looking up localized strings, etc.
+    '''</summary>
+    <Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.Microsoft.VisualBasic.HideModuleNameAttribute()>  _
+    Friend Module Resources
+        
+        Private resourceMan As Global.System.Resources.ResourceManager
+        
+        Private resourceCulture As Global.System.Globalization.CultureInfo
+        
+        '''<summary>
+        '''  Returns the cached ResourceManager instance used by this class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
+            Get
+                If Object.ReferenceEquals(resourceMan, Nothing) Then
+                    Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("SCrawler.Resources", GetType(Resources).Assembly)
+                    resourceMan = temp
+                End If
+                Return resourceMan
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Overrides the current thread's CurrentUICulture property for all
+        '''  resource lookups using this strongly typed resource class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend Property Culture() As Global.System.Globalization.CultureInfo
+            Get
+                Return resourceCulture
+            End Get
+            Set
+                resourceCulture = value
+            End Set
+        End Property
+        
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
+        '''</summary>
+        Friend ReadOnly Property RainbowIcon_48() As System.Drawing.Icon
+            Get
+                Dim obj As Object = ResourceManager.GetObject("RainbowIcon_48", resourceCulture)
+                Return CType(obj,System.Drawing.Icon)
+            End Get
+        End Property
+    End Module
+End Namespace

SCrawler.Updater/My Project/Resources.resx

@@ -0,0 +1,124 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" use="required" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
+  <data name="RainbowIcon_48" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>..\Content\Icons\RainbowIcon_48.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
+</root>

SCrawler.Updater/My Project/Settings.Designer.vb

@@ -0,0 +1,73 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+
+Namespace My
+    
+    <Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"),  _
+     Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+    Partial Friend NotInheritable Class MySettings
+        Inherits Global.System.Configuration.ApplicationSettingsBase
+        
+        Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
+        
+#Region "My.Settings Auto-Save Functionality"
+#If _MyType = "WindowsForms" Then
+    Private Shared addedHandler As Boolean
+
+    Private Shared addedHandlerLockObject As New Object
+
+    <Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
+    Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
+        If My.Application.SaveMySettingsOnExit Then
+            My.Settings.Save()
+        End If
+    End Sub
+#End If
+#End Region
+        
+        Public Shared ReadOnly Property [Default]() As MySettings
+            Get
+                
+#If _MyType = "WindowsForms" Then
+               If Not addedHandler Then
+                    SyncLock addedHandlerLockObject
+                        If Not addedHandler Then
+                            AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
+                            addedHandler = True
+                        End If
+                    End SyncLock
+                End If
+#End If
+                Return defaultInstance
+            End Get
+        End Property
+    End Class
+End Namespace
+
+Namespace My
+    
+    <Global.Microsoft.VisualBasic.HideModuleNameAttribute(),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()>  _
+    Friend Module MySettingsProperty
+        
+        <Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")>  _
+        Friend ReadOnly Property Settings() As Global.SCrawler.My.MySettings
+            Get
+                Return Global.SCrawler.My.MySettings.Default
+            End Get
+        End Property
+    End Module
+End Namespace

SCrawler.Updater/My Project/Settings.settings

@@ -0,0 +1,7 @@
+<?xml version='1.0' encoding='utf-8'?>
+<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
+  <Profiles>
+    <Profile Name="(Default)" />
+  </Profiles>
+  <Settings />
+</SettingsFile>

SCrawler.Updater/My Project/app.manifest

@@ -0,0 +1,79 @@
+<?xml version="1.0" encoding="utf-8"?>
+<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
+  <assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
+  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
+    <security>
+      <requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
+        <!-- UAC Manifest Options
+             If you want to change the Windows User Account Control level replace the 
+             requestedExecutionLevel node with one of the following.
+
+        <requestedExecutionLevel  level="asInvoker" uiAccess="false" />
+        <requestedExecutionLevel  level="requireAdministrator" uiAccess="false" />
+        <requestedExecutionLevel  level="highestAvailable" uiAccess="false" />
+
+            Specifying requestedExecutionLevel element will disable file and registry virtualization. 
+            Remove this element if your application requires this virtualization for backwards
+            compatibility.
+        -->
+        <requestedExecutionLevel level="asInvoker" uiAccess="false" />
+      </requestedPrivileges>
+    </security>
+  </trustInfo>
+
+  <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
+    <application>
+      <!-- A list of the Windows versions that this application has been tested on
+           and is designed to work with. Uncomment the appropriate elements
+           and Windows will automatically select the most compatible environment. -->
+
+      <!-- Windows Vista -->
+      <!--<supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}" />-->
+
+      <!-- Windows 7 -->
+      <!--<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}" />-->
+
+      <!-- Windows 8 -->
+      <!--<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}" />-->
+
+      <!-- Windows 8.1 -->
+      <!--<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}" />-->
+
+      <!-- Windows 10 -->
+      <!--<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />-->
+
+    </application>
+  </compatibility>
+
+  <!-- Indicates that the application is DPI-aware and will not be automatically scaled by Windows at higher
+       DPIs. Windows Presentation Foundation (WPF) applications are automatically DPI-aware and do not need 
+       to opt in. Windows Forms applications targeting .NET Framework 4.6 that opt into this setting, should 
+       also set the 'EnableWindowsFormsHighDpiAutoResizing' setting to 'true' in their app.config. 
+       
+       Makes the application long-path aware. See https://docs.microsoft.com/windows/win32/fileio/maximum-file-path-limitation -->
+  <!--
+  <application xmlns="urn:schemas-microsoft-com:asm.v3">
+    <windowsSettings>
+      <dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware>
+      <longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
+    </windowsSettings>
+  </application>
+  -->
+
+  <!-- Enable themes for Windows common controls and dialogs (Windows XP and later) -->
+  <!--
+  <dependency>
+    <dependentAssembly>
+      <assemblyIdentity
+          type="win32"
+          name="Microsoft.Windows.Common-Controls"
+          version="6.0.0.0"
+          processorArchitecture="*"
+          publicKeyToken="6595b64144ccf1df"
+          language="*"
+        />
+    </dependentAssembly>
+  </dependency>
+  -->
+
+</assembly>

SCrawler.Updater/SCrawler.Updater.vbproj

@@ -0,0 +1,173 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
+  <PropertyGroup>
+    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
+    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
+    <ProjectGuid>{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}</ProjectGuid>
+    <OutputType>Exe</OutputType>
+    <StartupObject>Sub Main</StartupObject>
+    <RootNamespace>SCrawler</RootNamespace>
+    <AssemblyName>Updater</AssemblyName>
+    <FileAlignment>512</FileAlignment>
+    <MyType>Console</MyType>
+    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
+    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
+    <Deterministic>true</Deterministic>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
+    <PlatformTarget>AnyCPU</PlatformTarget>
+    <DebugSymbols>true</DebugSymbols>
+    <DebugType>full</DebugType>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\Debug\Updater\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <Prefer32Bit>false</Prefer32Bit>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
+    <PlatformTarget>AnyCPU</PlatformTarget>
+    <DebugType>pdbonly</DebugType>
+    <DefineDebug>false</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <Optimize>true</Optimize>
+    <OutputPath>bin\Release\Updater\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <Prefer32Bit>false</Prefer32Bit>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionExplicit>On</OptionExplicit>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionCompare>Binary</OptionCompare>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionStrict>Off</OptionStrict>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionInfer>On</OptionInfer>
+  </PropertyGroup>
+  <PropertyGroup>
+    <ApplicationIcon>Content\Icons\RainbowIcon_48.ico</ApplicationIcon>
+  </PropertyGroup>
+  <PropertyGroup>
+    <ApplicationManifest>My Project\app.manifest</ApplicationManifest>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Debug\Updater\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Release\Updater\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Debug\Updater\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Release\Updater\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <ItemGroup>
+    <Reference Include="System" />
+    <Reference Include="System.Data" />
+    <Reference Include="System.Deployment" />
+    <Reference Include="System.Drawing" />
+    <Reference Include="System.IO.Compression" />
+    <Reference Include="System.IO.Compression.FileSystem" />
+    <Reference Include="System.Xml" />
+    <Reference Include="System.Core" />
+    <Reference Include="System.Xml.Linq" />
+    <Reference Include="System.Data.DataSetExtensions" />
+    <Reference Include="System.Net.Http" />
+  </ItemGroup>
+  <ItemGroup>
+    <Import Include="Microsoft.VisualBasic" />
+    <Import Include="System" />
+    <Import Include="System.Collections" />
+    <Import Include="System.Collections.Generic" />
+    <Import Include="System.Data" />
+    <Import Include="System.Diagnostics" />
+    <Import Include="System.Linq" />
+    <Import Include="System.Xml.Linq" />
+    <Import Include="System.Threading.Tasks" />
+  </ItemGroup>
+  <ItemGroup>
+    <Compile Include="MainMod.vb" />
+    <Compile Include="My Project\AssemblyInfo.vb" />
+    <Compile Include="My Project\Application.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Application.myapp</DependentUpon>
+      <DesignTime>True</DesignTime>
+    </Compile>
+    <Compile Include="My Project\Resources.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DesignTime>True</DesignTime>
+      <DependentUpon>Resources.resx</DependentUpon>
+    </Compile>
+    <Compile Include="My Project\Settings.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Settings.settings</DependentUpon>
+      <DesignTimeSharedInput>True</DesignTimeSharedInput>
+    </Compile>
+  </ItemGroup>
+  <ItemGroup>
+    <EmbeddedResource Include="My Project\Resources.resx">
+      <Generator>VbMyResourcesResXFileCodeGenerator</Generator>
+      <LastGenOutput>Resources.Designer.vb</LastGenOutput>
+      <CustomToolNamespace>My.Resources</CustomToolNamespace>
+      <SubType>Designer</SubType>
+    </EmbeddedResource>
+  </ItemGroup>
+  <ItemGroup>
+    <None Include=".editorconfig" />
+    <None Include="My Project\app.manifest" />
+    <None Include="My Project\Application.myapp">
+      <Generator>MyApplicationCodeGenerator</Generator>
+      <LastGenOutput>Application.Designer.vb</LastGenOutput>
+    </None>
+    <None Include="My Project\Settings.settings">
+      <Generator>SettingsSingleFileGenerator</Generator>
+      <CustomToolNamespace>My</CustomToolNamespace>
+      <LastGenOutput>Settings.Designer.vb</LastGenOutput>
+    </None>
+    <None Include="App.config" />
+  </ItemGroup>
+  <ItemGroup>
+    <Content Include="Content\Icons\RainbowIcon_48.ico" />
+  </ItemGroup>
+  <ItemGroup>
+    <ProjectReference Include="..\..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj">
+      <Project>{8405896b-2685-4916-bc93-1fb514c323a9}</Project>
+      <Name>PersonalUtilities</Name>
+    </ProjectReference>
+    <ProjectReference Include="..\SCrawler.Shared\SCrawler.Shared.vbproj">
+      <Project>{dc634700-24c7-42dd-bf8f-87e6cc54e625}</Project>
+      <Name>SCrawler.Shared</Name>
+    </ProjectReference>
+  </ItemGroup>
+  <Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
+</Project>

SCrawler.YouTube/.editorconfig

@@ -1,3 +1,3 @@
 [*.vb]
 # Modifier preferences
-file_header_template = Copyright (C) 2023  Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.YouTube/Base/Structures.vb

@@ -28,8 +28,19 @@ Namespace API.YouTube.Base
     End Structure
     Public Structure Subtitles : Implements IIndexable, IComparable(Of Subtitles)
         Public ID As String
-        Public Name As String
+        Private _Name As String
+        Public Property Name As String
+            Get
+                Dim n$ = _Name.IfNullOrEmpty(ID)
+                If CC Then n &= " (CC)"
+                Return n
+            End Get
+            Set(ByVal NewName As String)
+                _Name = NewName
+            End Set
+        End Property
         Public Formats As String
+        Public CC As Boolean
         Public ReadOnly Property FullID As String
             Get
                 Return IIf(ID = "en", "en.*", ID)

SCrawler.YouTube/Base/YouTubeSettings.vb

@@ -35,6 +35,7 @@ Namespace API.YouTube.Base
         <Browsable(False)> Private Property Mode As GridUpdateModes = GridUpdateModes.OnConfirm Implements IGridValuesContainer.Mode
         <Browsable(False), XMLVV(-1)> Friend ReadOnly Property PlaylistFormSplitterDistance As XMLValue(Of Integer)
         <Browsable(False)> Friend ReadOnly Property DownloadLocations As DownloadLocationsCollection
+        <Browsable(False)> Public Overridable Property AccountName As String
 #Region "Environment"
 #Region "Programs"
         <Browsable(True), GridVisible(False), XMLVN({"Environment"}), Category("Environment programs"), DisplayName("Path to yt-dlp.exe"),
@@ -131,6 +132,8 @@ Namespace API.YouTube.Base
                 OpenFolderInOtherProgram.Value = command
             End Set
         End Property
+        <Browsable(True), GridVisible(False), XMLVN({"Environment"}, True), Category("Environment"), DisplayName("Check new version at start")>
+        Friend ReadOnly Property CheckUpdatesAtStart As XMLValue(Of Boolean)
 #End Region
 #Region "Info"
         <Browsable(True), GridVisible, XMLVN({"Info"}), Category("Info"), DisplayName("Create URL files"),
@@ -300,9 +303,17 @@ Namespace API.YouTube.Base
 #End Region
 #Region "Initializer"
         Public Sub New()
+            Me.New(String.Empty)
+        End Sub
+        Public Sub New(ByVal AccountName As String)
+            Me.AccountName = AccountName
             DownloadLocations = New DownloadLocationsCollection
             DownloadLocations.Load(False, True)
-            XML = New XmlFile(YouTubeSettingsFile,, False) With {.AutoUpdateFile = True}
+            Dim acc$ = String.Empty
+            If Not AccountName.IsEmptyString Then acc = $"_{AccountName}"
+            Dim f As SFile = YouTubeSettingsFile
+            f.Name &= acc
+            XML = New XmlFile(f,, False) With {.AutoUpdateFile = True}
             XML.LoadData(EDP.None)
             DesignXml = New XmlFile("Settings\DesignDownloader.xml", Protector.Modes.All, False)
             DesignXml.LoadData(EDP.None)
@@ -310,7 +321,9 @@ Namespace API.YouTube.Base
             AddHandler ShowNotificationsEveryDownload.TempValueChanged, AddressOf ShowNotificationsEveryDownload_TempValueChanged
             Cookies = New CookieKeeper
             Grid.Abstract.DesignerXmlSource.Add(New Grid.Abstract.DesignerXmlData(GetType(CookieListForm2), DesignXml, "CookiesListForm"))
-            If YouTubeCookieNetscapeFile.Exists Then Cookies.AddRange(CookieKeeper.ParseNetscapeText(YouTubeCookieNetscapeFile.GetText(EDP.ReturnValue), EDP.None),, EDP.None)
+            f = YouTubeCookieNetscapeFile
+            f.Name &= acc
+            If f.Exists Then Cookies.AddRange(CookieKeeper.ParseNetscapeText(f.GetText(EDP.ReturnValue), EDP.None),, EDP.None)
             If Not YTDLP.Value.Exists Then YTDLP.Value = ProgramPath("yt-dlp.exe")
             If Not FFMPEG.Value.Exists Then FFMPEG.Value = ProgramPath("ffmpeg.exe")
             If Not OutputPath.Value.Exists(SFO.Path, False) Then OutputPath.Value = YouTubeDownloadPathDefault

SCrawler.YouTube/Controls/MusicPlaylistsForm.Designer.vb

@@ -427,10 +427,14 @@ Namespace API.YouTube.Controls
             Me.TXT_OUTPUT_PATH.Buttons.Add(ActionButton8)
             Me.TXT_OUTPUT_PATH.Buttons.Add(ActionButton9)
             Me.TXT_OUTPUT_PATH.Buttons.Add(ActionButton10)
-            Me.TXT_OUTPUT_PATH.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.Label
+            Me.TXT_OUTPUT_PATH.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.CheckBox
             Me.TXT_OUTPUT_PATH.CaptionText = "Output path"
+            Me.TXT_OUTPUT_PATH.CaptionToolTipEnabled = True
+            Me.TXT_OUTPUT_PATH.CaptionToolTipText = "If this checkbox is selected, this path is absolute and artist folder will not be" &
+    " created in it"
             Me.TXT_OUTPUT_PATH.CaptionVisible = True
             Me.TXT_OUTPUT_PATH.CaptionWidth = 112.0R
+            Me.TXT_OUTPUT_PATH.ChangeControlsEnableOnCheckedChange = False
             ListColumn1.Name = "COL_NAME"
             ListColumn1.Text = "Name"
             ListColumn1.Width = -1

SCrawler.YouTube/Controls/MusicPlaylistsForm.vb

@@ -80,6 +80,14 @@ Namespace API.YouTube.Controls
                 End If
                 LIST_PLAYLISTS.SelectedIndex = 0
 
+                If .ObjectType = Base.YouTubeMediaType.Channel Then
+                    With TXT_OUTPUT_PATH
+                        .CaptionMode = ICaptionControl.Modes.Label
+                        .CaptionToolTipText = String.Empty
+                        .CaptionToolTipEnabled = False
+                    End With
+                End If
+
                 TXT_OUTPUT_PATH.Text = MyYouTubeSettings.OutputPath.Value
 
                 If Not .UserTitle.IsEmptyString Then
@@ -266,6 +274,7 @@ Namespace API.YouTube.Controls
                     If Not TXT_SUBS.Checked Then .PostProcessing_OutputSubtitlesFormats.Clear()
                     .OutputAudioCodec = CMB_FORMATS.Text
                     If Not TXT_FORMATS_ADDIT.Checked Then .PostProcessing_OutputAudioFormats.Clear()
+                    .AbsolutePath = TXT_OUTPUT_PATH.Checked
                     .File = TXT_OUTPUT_PATH.Text.CSFileP
                     If MyYouTubeSettings.OutputPathAutoChange Then MyYouTubeSettings.OutputPath.Value = .File
                     If MyDownloaderSettings.OutputPathAutoAddPaths Then MyYouTubeSettings.DownloadLocations.Add(.File, False)

SCrawler.YouTube/Controls/PlaylistArrayForm.Designer.vb

@@ -26,7 +26,7 @@ Namespace API.YouTube.Controls
             Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
             Dim FRM_PLS As System.Windows.Forms.GroupBox
             Me.CH_PLS_ONE = New System.Windows.Forms.CheckBox()
-            Me.TXT_URLS = New System.Windows.Forms.RichTextBox()
+            Me.TXT_URLS = New System.Windows.Forms.TextBox()
             CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
             TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
             FRM_PLS = New System.Windows.Forms.GroupBox()
@@ -94,13 +94,14 @@ Namespace API.YouTube.Controls
             '
             'TXT_URLS
             '
-            Me.TXT_URLS.DetectUrls = False
+            Me.TXT_URLS.AcceptsReturn = True
             Me.TXT_URLS.Dock = System.Windows.Forms.DockStyle.Fill
             Me.TXT_URLS.Location = New System.Drawing.Point(3, 16)
+            Me.TXT_URLS.MaxLength = 2147483647
+            Me.TXT_URLS.Multiline = True
             Me.TXT_URLS.Name = "TXT_URLS"
             Me.TXT_URLS.Size = New System.Drawing.Size(372, 261)
             Me.TXT_URLS.TabIndex = 0
-            Me.TXT_URLS.Text = ""
             '
             'PlaylistArrayForm
             '
@@ -119,10 +120,11 @@ Namespace API.YouTube.Controls
             TP_MAIN.ResumeLayout(False)
             TP_MAIN.PerformLayout()
             FRM_PLS.ResumeLayout(False)
+            FRM_PLS.PerformLayout()
             Me.ResumeLayout(False)
 
         End Sub
         Private WithEvents CH_PLS_ONE As CheckBox
-        Private WithEvents TXT_URLS As RichTextBox
+        Private WithEvents TXT_URLS As TextBox
     End Class
 End Namespace

SCrawler.YouTube/Controls/VideoOptionsForm.Designer.vb

@@ -275,6 +275,7 @@ Namespace API.YouTube.Controls
             ActionButton1.Name = "ArrowDown"
             ActionButton1.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.ArrowDown
             Me.TXT_FILE.Buttons.Add(ActionButton1)
+            Me.TXT_FILE.ChangeControlsEnableOnCheckedChange = False
             ListColumn1.Name = "COL_NAME"
             ListColumn1.Text = "Name"
             ListColumn1.Width = -1

SCrawler.YouTube/Controls/VideoOptionsForm.vb

@@ -76,6 +76,12 @@ Namespace API.YouTube.Controls
                             If Not def.ValueBetween(-1, 10000) Then def = 1080
                         End If
                         NUM_RES.Value = def
+                        With TXT_FILE
+                            .CaptionMode = ICaptionControl.Modes.CheckBox
+                            .CaptionWidth = 18
+                            .CaptionToolTipText = "If this checkbox is selected, this path is absolute and artist folder will not be created in it"
+                            .CaptionToolTipEnabled = True
+                        End With
                     Else
                         TP_OPTIONS.Controls.Remove(NUM_RES)
                         TP_OPTIONS.ColumnStyles(3).Width = 0
@@ -297,8 +303,8 @@ Namespace API.YouTube.Controls
                             .SelectedVideoIndex = -1
                             .SelectedAudioIndex = cntIndex
                         End If
-                        .File = f
                         .FileSetManually = True
+                        .File = f
                         .UpdateInfoFields()
                         '#If DEBUG Then
                         'Debug.WriteLine(.Command(False))
@@ -309,6 +315,7 @@ Namespace API.YouTube.Controls
                         Else
                             .SetMaxResolution(NUM_RES.Value)
                         End If
+                        .AbsolutePath = TXT_FILE.Checked
                         .File = f
                     End If
                 End With

SCrawler.YouTube/Declarations.vb

@@ -108,5 +108,16 @@ Namespace API.YouTube
                 Throw New NotImplementedException("'GetFormat' is not available in the 'DurationXmlConverter' context")
             End Function
         End Class
+        Friend Sub CheckVersion(ByVal Force As Boolean)
+            If Not MyYouTubeSettings Is Nothing Then
+                With MyYouTubeSettings
+                    If .CheckUpdatesAtStart Or Force Then
+                        ShowProgramInfo(.ProgramText.Value.IfNullOrEmpty("YouTube Downloader"),
+                                        SCrawler.Shared.GetCurrentMaxVer(Application.StartupPath.CSFileP).IfNullOrEmpty(My.Application.Info.Version),
+                                        True, Force, .Self, True,, False, .ProgramDescription)
+                    End If
+                End With
+            End If
+        End Sub
     End Module
 End Namespace

SCrawler.YouTube/Downloader/VideoListForm.vb

@@ -59,6 +59,8 @@ Namespace DownloadObjects.STDownloader
                 MyDownloaderSettings = MyYouTubeSettings
             End If
 
+            CheckVersion(False)
+
             With MyView : .Import() : .SetFormSize() : End With
             BTT_DELETE.Enabled = False
             If Not AppMode Then
@@ -278,6 +280,17 @@ Namespace DownloadObjects.STDownloader
                                             .IsMusic = containers.Any(Function(cc) cc.IsMusic)
                                         }
                                         c.Elements.AddRange(containers)
+                                        Dim path$ = c.Elements(0).File.PathWithSeparator
+                                        For Each list As List(Of String) In {
+                                            c.Elements.Select(Function(cc) cc.UserTitle).ListWithRemove(Function(cc) cc.IsEmptyString).ListIfNothing,
+                                            c.Elements.Select(Function(cc) cc.PlaylistTitle).ListWithRemove(Function(cc) cc.IsEmptyString).ListIfNothing
+                                        }
+                                            If list.Count > 0 AndAlso
+                                               (list.Count = 1 OrElse
+                                               ListAddList(Nothing, list, LAP.NotContainsOnly, EDP.ReturnValue).ListIfNothing.Count = 1) Then _
+                                               path &= $"{list(0)}\"
+                                        Next
+                                        c.File = path
                                     End If
                                 End If
                             End With
@@ -315,7 +328,7 @@ Namespace DownloadObjects.STDownloader
                         If Not f Is Nothing Then
                             If TypeOf f Is IDesignXMLContainer Then DirectCast(f, IDesignXMLContainer).DesignXML = DesignXML
                             f.ShowDialog()
-                            If f.DialogResult = DialogResult.OK Then ControlCreateAndAdd(c, disableDown)
+                            If f.DialogResult = DialogResult.OK AndAlso ValidateContainerURL(c) Then ControlCreateAndAdd(c, disableDown)
                             f.Dispose()
                         End If
                     End If
@@ -329,6 +342,61 @@ Namespace DownloadObjects.STDownloader
                 If Not pForm Is Nothing Then pForm.Dispose()
             End Try
         End Sub
+        Protected Function ValidateContainerURL(ByVal c As IYouTubeMediaContainer) As Boolean
+            Try
+                If Not c Is Nothing AndAlso Not c.IsMusic Then
+                    Dim urls As List(Of String) = Nothing
+                    Dim files As List(Of SFile) = Nothing
+                    Dim msg As New MMessage("The media file to be added is already downloaded. Do you want to download it again?", "Download media file", {"Process", "Cancel"}, vbExclamation)
+                    If TP_CONTROLS.Controls.Count > 0 Then
+                        With TP_CONTROLS.Controls.Cast(Of MediaItem)
+                            urls.ListAddList(.SelectMany(Function(ByVal m As MediaItem) As IEnumerable(Of String)
+                                                             If Not m.MyContainer Is Nothing Then
+                                                                 Return DirectCast(m.MyContainer, YouTubeMediaContainerBase).GetUrls()
+                                                             Else
+                                                                 Return New String() {}
+                                                             End If
+                                                         End Function), LAP.NotContainsOnly, EDP.ReturnValue)
+                            files.ListAddList(.SelectMany(Function(ByVal m As MediaItem) As IEnumerable(Of SFile)
+                                                              If Not m.MyContainer Is Nothing Then
+                                                                  Return DirectCast(m.MyContainer, YouTubeMediaContainerBase).GetFiles()
+                                                              Else
+                                                                  Return New SFile() {}
+                                                              End If
+                                                          End Function), LAP.NotContainsOnly, EDP.ReturnValue)
+                        End With
+                    End If
+                    If urls.ListExists Then
+                        Dim cUrls As New List(Of String)
+                        cUrls.ListAddList({c.URL, c.URL_BASE}, LAP.NotContainsOnly)
+                        If urls.ListContains(cUrls) Then Return msg.Show = 0
+                    End If
+                    If files.ListExists And Not c.File.IsEmptyString Then Return Not files.Contains(c.File) OrElse msg.Show = 0
+                    If c.ObjectType = YouTubeMediaType.Single AndAlso c.File.Exists Then
+                        Dim callBack As MsgBoxButtonCallBack = Sub(r, m, b)
+                                                                   Dim __sfo As SFO = IIf(r.Button.CallBackObject = 0, SFO.File, SFO.Path)
+                                                                   If __sfo = SFO.File Then
+                                                                       c.File.Open(__sfo)
+                                                                   Else
+                                                                       GlobalOpenPath(c.File)
+                                                                   End If
+                                                               End Sub
+                        Return MsgBoxE(New MMessage("The following file already exists at the destination." & vbCr &
+                                        "Do you want to download it again?" & vbCr & vbCr &
+                                        $"File: {c.File}", "Download media file",
+                                        {"Process",
+                                         New MsgBoxButton("Open file") With {.IsDialogResultButton = False, .CallBackObject = 0, .CallBack = callBack},
+                                         New MsgBoxButton("Open folder") With {.IsDialogResultButton = False, .CallBackObject = 1, .CallBack = callBack},
+                                         "Cancel"}, vbExclamation) With {.ButtonsPerRow = 4}) = 0
+                    End If
+                    urls.ListClearDispose
+                    files.ListClearDispose
+                End If
+                Return True
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, "[VideoListForm.ValidateContainerURL]", True)
+            End Try
+        End Function
         Private Sub BTT_DOWN_Click(sender As Object, e As EventArgs) Handles BTT_DOWN.Click
             With TP_CONTROLS
                 If .Controls.Count > 0 Then
@@ -376,8 +444,7 @@ Namespace DownloadObjects.STDownloader
             Try : Process.Start("https://github.com/AAndyProgram/SCrawler/blob/main/HowToSupport.md") : Catch : End Try
         End Sub
         Private Sub BTT_INFO_Click(sender As Object, e As EventArgs) Handles BTT_INFO.Click
-            ShowProgramInfo(MyYouTubeSettings.ProgramText.Value.IfNullOrEmpty("YouTube Downloader"),
-                            My.Application.Info.Version, False, True, MyYouTubeSettings, True,, False, MyYouTubeSettings.ProgramDescription)
+            CheckVersion(True)
         End Sub
         Protected Overloads Sub RemoveControls(Optional ByVal Predicate As Predicate(Of MediaItem) = Nothing, Optional ByVal RemoveFiles As Boolean = False)
             ControlInvokeFast(TP_CONTROLS, Sub()
@@ -450,12 +517,16 @@ Namespace DownloadObjects.STDownloader
             UpdateLogButton()
         End Sub
         Protected Sub AddToDownload(ByRef Item As MediaItem, ByVal RunThread As Boolean)
-            Dim hc% = Item.MyContainer.GetHashCode
-            If MyJob.Count = 0 OrElse Not MyJob.Items.Exists(Function(i) i.MyContainer.GetHashCode = hc) Then
-                MyJob.Add(Item)
-                Item.AddToQueue()
-                If RunThread Then StartDownloading()
-            End If
+            Try
+                Dim hc% = Item.MyContainer.GetHashCode
+                If MyJob.Count = 0 OrElse Not MyJob.Items.Exists(Function(i) i.MyContainer.GetHashCode = hc) Then
+                    MyJob.Add(Item)
+                    Item.AddToQueue()
+                    If RunThread Then StartDownloading()
+                End If
+            Catch ex As Exception
+                ErrorsDescriber.Execute(EDP.SendToLog, ex, "[VideoListForm.AddToDownload]")
+            End Try
         End Sub
         Private Sub StartDownloading()
             If Not MyJob.Working And MyJob.Count > 0 Then

SCrawler.YouTube/MainModShared.vb

@@ -6,6 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Threading
 Imports PersonalUtilities.Tools
 Imports PersonalUtilities.Tools.Web
 Imports PersonalUtilities.Functions.Messaging
@@ -38,19 +39,54 @@ Public Module MainModShared
             End If
         End Try
     End Sub
+    Public Sub CheckNewReleaseFolder()
+        Try
+            Const updaterFolderName$ = "Updater\"
+            Dim f As SFile = SCrawler.Shared.NewReleaseFolderName.CSFileP
+            If f.Exists(SFO.Path, False) Then
+                Dim updater As SFile = updaterFolderName
+                Dim updaterNR As SFile = f.PathWithSeparator & updaterFolderName
+                If updaterNR.Exists(SFO.Path, False) Then
+                    If updater.Exists(SFO.Path, False) Then updater.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.ReturnValue)
+                    SFile.Move(updaterNR, updater, SFO.Path, True, SFODelete.DeletePermanently, EDP.ReturnValue)
+                End If
+                f.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.None)
+            End If
+        Catch ex As Exception
+        End Try
+    End Sub
     Public Sub ShowProgramInfo(ByVal ProgramText As String, ByVal CurrentVersion As Version, ByVal CheckForUpdate As Boolean, ByVal Force As Boolean,
                                ByVal EnvirData As IDownloaderSettings, ByVal IsYouTube As Boolean,
-                               Optional ByRef NewVersionDestination As String = Nothing, Optional ByVal ShowNewVersionNotification As Boolean = True,
+                               Optional ByRef NewVersionDestination As String = Nothing, Optional ByRef ShowNewVersionNotification As Boolean = True,
                                Optional ByVal AdditText As String = Nothing)
         Try
             Dim GoToSite As New MsgBoxButton("Go to site") With {.CallBack = Sub(r, n, b) Process.Start("https://github.com/AAndyProgram/SCrawler/releases/latest")}
             If CheckForUpdate AndAlso GitHub.NewVersionExists(CurrentVersion, "AAndyProgram", "SCrawler", NewVersionDestination) Then
                 If ShowNewVersionNotification Or Force Then
-                    If MsgBoxE(New MMessage($"{ProgramText}: new version detected" & vbCr &
-                                            $"Current version: {CurrentVersion}" & vbCr &
-                                            $"New version: {NewVersionDestination}",
-                                            "New version",
-                                            {"OK", GoToSite, "Disable notifications"})) = 2 Then ShowNewVersionNotification = False
+                    Dim b As New List(Of MsgBoxButton)
+                    Dim updaterFile As SFile = Nothing
+                    Dim updateBtt As New MsgBoxButton("Update", "Update the program using the updater") With {
+                        .CallBack = Sub(r, n, btt)
+                                        Dim th As New Thread(Sub() Process.Start(New ProcessStartInfo(updaterFile, 1))) With {.IsBackground = True}
+                                        th.SetApartmentState(ApartmentState.MTA)
+                                        th.Start()
+                                    End Sub}
+                    With SFile.GetFiles("Updater\", "*.exe",, EDP.ReturnValue).ListIfNothing
+                        If .ListExists Then
+                            With .FirstOrDefault(Function(f) f.Name = "Updater")
+                                If Not .IsEmptyString Then updaterFile = .Self
+                            End With
+                        End If
+                    End With
+                    b.AddRange({"OK", GoToSite})
+                    If Not updaterFile.IsEmptyString Then b.Add(updateBtt)
+                    b.Add(New MsgBoxButton("Disable notifications") With {.CallBackObject = 10})
+                    If AConvert(Of Integer)(
+                                MsgBoxE(New MMessage($"{ProgramText}: new version detected" & vbCr &
+                                                     $"Current version: {CurrentVersion}" & vbCr &
+                                                     $"New version: {NewVersionDestination}",
+                                                     "New version", b) With {.ButtonsPerRow = 4}).Button.CallBackObject, -1) = 10 Then _
+                       ShowNewVersionNotification = False
                 End If
             Else
                 If Force Then

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
 <Assembly: AssemblyDescription("YouTube plugin environment")>
 <Assembly: AssemblyCompany("AndyProgram")>
 <Assembly: AssemblyProduct("SCrawler.YouTube")>
-<Assembly: AssemblyCopyright("Copyright © 2023")>
+<Assembly: AssemblyCopyright("Copyright © 2024")>
 <Assembly: AssemblyTrademark("AndyProgram")>
 
 <Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.10.10.0")>
-<Assembly: AssemblyFileVersion("2023.10.10.0")>
+<Assembly: AssemblyVersion("2023.12.7.0")>
+<Assembly: AssemblyFileVersion("2023.12.7.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Objects/YouTubeMediaContainerBase.vb

@@ -111,6 +111,7 @@ Namespace API.YouTube.Objects
                 _SiteKey = Key
             End Set
         End Property
+        <XMLEC> Public Property AccountName As String = String.Empty Implements IDownloadableMedia.AccountName
         <XMLEC(Name_IsMusic)> Public Property IsMusic As Boolean = False Implements IYouTubeMediaContainer.IsMusic
         <XMLEC> Public Property IsShorts As Boolean = False Implements IYouTubeMediaContainer.IsShorts
         <XMLEC> Public Property ID As String Implements IYouTubeMediaContainer.ID, IUserMedia.PostID
@@ -495,6 +496,12 @@ Namespace API.YouTube.Objects
                 _IUserMedia_URL_BASE = u
             End Set
         End Property
+        Friend Function GetUrls() As IEnumerable(Of String)
+            Dim urls As New List(Of String)
+            urls.ListAddList({URL, IUserMedia_URL_BASE}, LAP.NotContainsOnly)
+            If HasElements And Not IsMusic Then urls.ListAddList(Elements.SelectMany(Function(elem As YouTubeMediaContainerBase) elem.GetUrls()), LAP.NotContainsOnly)
+            Return urls
+        End Function
         Protected Overridable Sub GenerateFileName()
         End Sub
         Protected Function GetPlayListTitle() As String
@@ -530,7 +537,7 @@ Namespace API.YouTube.Objects
             If ObjectType = YouTubeMediaType.Single AndAlso Not GetPlayListTitle.IsEmptyString Then _SpecialPath.StringAppend(GetPlayListTitle(), "\")
             If Elements.Count > 0 Then Elements.ForEach(Sub(e) e.SpecialFolder = Path)
         End Sub
-        <XMLEC> Friend ReadOnly Property Files As List(Of SFile) Implements IYouTubeMediaContainer.Files
+        <XMLEC> Protected Friend ReadOnly Property Files As List(Of SFile) Implements IYouTubeMediaContainer.Files
         <XMLEC> Protected _File As SFile
         <XMLEC> Protected Friend Property FileSetManually As Boolean = False
         Public Property FileIgnorePlaylist As Boolean = False
@@ -542,6 +549,16 @@ Namespace API.YouTube.Objects
                 Return _FileIsPlaylistObject
             End Get
         End Property
+        Private _AbsolutePath As Boolean = False
+        Public Property AbsolutePath As Boolean
+            Get
+                Return _AbsolutePath
+            End Get
+            Set(ByVal ap As Boolean)
+                _AbsolutePath = ap
+                If Elements.Count > 0 Then Elements.ForEach(Sub(e As YouTubeMediaContainerBase) e.AbsolutePath = ap)
+            End Set
+        End Property
         Public Overridable Property File As SFile Implements IYouTubeMediaContainer.File
             Get
                 Return _File
@@ -549,11 +566,16 @@ Namespace API.YouTube.Objects
             Set(ByVal f As SFile)
                 Select Case ObjectType
                     Case YouTubeMediaType.Channel : _File = f.Path
-                    Case YouTubeMediaType.PlayList : _File.Path = $"{f.PathWithSeparator}{GetPlayListTitle()}"
+                    Case YouTubeMediaType.PlayList
+                        If AbsolutePath Then
+                            _File.Path = f.Path
+                        Else
+                            _File.Path = $"{f.PathWithSeparator}{GetPlayListTitle()}"
+                        End If
                     Case YouTubeMediaType.Single
                         If PlaylistCount > 0 And Not FileIgnorePlaylist Then
                             _File.Path = f.Path
-                            Dim pls$ = GetPlayListTitle()
+                            Dim pls$ = If(AbsolutePath, String.Empty, GetPlayListTitle())
                             If Not _File.Path.Contains(pls) Then _File.Path = $"{_File.PathWithSeparator(Not pls.IsEmptyString)}{pls}"
                         ElseIf Not f.Name.IsEmptyString Then
                             _File = f
@@ -575,6 +597,11 @@ Namespace API.YouTube.Objects
                 File = f
             End Set
         End Property
+        Friend Function GetFiles() As IEnumerable(Of SFile)
+            Dim urls As New List(Of String)({File})
+            If HasElements And Not IsMusic Then urls.ListAddList(Elements.SelectMany(Function(elem As YouTubeMediaContainerBase) elem.GetFiles()), LAP.NotContainsOnly)
+            Return urls
+        End Function
 #End Region
 #Region "Command"
         <XMLEC> Public Property UseCookies As Boolean = MyYouTubeSettings.DefaultUseCookies Implements IYouTubeMediaContainer.UseCookies
@@ -709,7 +736,7 @@ Namespace API.YouTube.Objects
         End Sub
 #End Region
 #Region "Download"
-        Protected Shared Sub CreateUrlFile(ByVal URL As String, ByVal File As SFile)
+        Protected Shared Function CreateUrlFile(ByVal URL As String, ByVal File As SFile) As SFile
             Try
                 File.Extension = "url"
                 Using t As New TextSaver(File)
@@ -719,9 +746,11 @@ Namespace API.YouTube.Objects
                     t.AppendLine()
                     t.Save(EDP.None)
                 End Using
+                Return File
             Catch ex As Exception
+                Return Nothing
             End Try
-        End Sub
+        End Function
         Private ReadOnly DownloadProgressPattern As RParams = RParams.DMS("\[download\]\s*([\d\.,]+)", 1, EDP.ReturnValue)
         Public Property Progress As MyProgress Implements IYouTubeMediaContainer.Progress
         Private Property IDownloadableMedia_Progress As Object Implements IDownloadableMedia.Progress
@@ -1421,28 +1450,36 @@ Namespace API.YouTube.Objects
         Protected Sub ParseSubtitles(ByVal e As EContainer)
             Dim subt As Subtitles
             Dim ee As EContainer
-            Dim se As EContainer = e({"subtitles"})
-            If If(se?.Count, 0) = 0 OrElse (se.Count = 1 And se(0).Name = "live_chat") Then se = e({"automatic_captions"})
-            If If(se?.Count, 0) > 0 Then
-                If se.Count > 1 OrElse Not se(0).Name = "live_chat" Then
+            Dim eSUB As EContainer = e({"subtitles"})
+            Dim eCC As EContainer = e({"automatic_captions"})
+            If If(eSUB?.Count, 0) = 0 OrElse (eSUB.Count = 1 And eSUB(0).Name = "live_chat") Then eSUB = Nothing
+            If If(eCC?.Count, 0) = 0 OrElse (eCC.Count = 1 And eCC(0).Name = "live_chat") Then eCC = Nothing
+            If If(eSUB?.Count, 0) > 0 Or If(eCC?.Count, 0) > 0 Then
+                Dim sl As New List(Of EContainer)
+                Dim ccExists As Boolean = False
+                Dim ccIndx% = -1, rIndx% = -1
+                If If(eSUB?.Count, 0) > 0 Then sl.Add(eSUB) : ccIndx += 1
+                If If(eCC?.Count, 0) > 0 Then sl.Add(eCC) : ccIndx += 1 : ccExists = True
+                For Each se As EContainer In sl
+                    rIndx += 1
                     For Each ee In se
-                        subt = New Subtitles With {.ID = ee.Name}
+                        subt = New Subtitles With {.ID = ee.Name, .CC = rIndx = ccIndx And ccExists}
                         If ee.Count > 0 Then
                             subt.Name = ee(0).Value("name")
                             subt.Formats = ee.Select(Function(f) f.Value("ext")).ListToString(",")
                         End If
                         If Not subt.ID.IsEmptyString Then _Subtitles.Add(subt)
                     Next
-                    With MyYouTubeSettings
-                        If Not .DefaultSubtitlesFormat.IsEmptyString Then OutputSubtitlesFormat = .DefaultSubtitlesFormat
-                        If _Subtitles.Count > 0 And .DefaultSubtitles.Count > 0 Then
-                            _Subtitles.Sort()
-                            _Subtitles.ListReindex
-                            SubtitlesSelectedIndexesReset()
-                            PostProcessing_OutputSubtitlesFormats_Reset()
-                        End If
-                    End With
-                End If
+                Next
+                With MyYouTubeSettings
+                    If Not .DefaultSubtitlesFormat.IsEmptyString Then OutputSubtitlesFormat = .DefaultSubtitlesFormat
+                    If _Subtitles.Count > 0 And .DefaultSubtitles.Count > 0 Then
+                        _Subtitles.Sort()
+                        _Subtitles.ListReindex
+                        SubtitlesSelectedIndexesReset()
+                        PostProcessing_OutputSubtitlesFormats_Reset()
+                    End If
+                End With
             End If
         End Sub
 #End Region

SCrawler.YouTube/SCrawler.YouTube.vbproj

@@ -273,6 +273,10 @@
       <Project>{d4650f6b-5a54-44b6-999b-6c675b7116b1}</Project>
       <Name>SCrawler.PluginProvider</Name>
     </ProjectReference>
+    <ProjectReference Include="..\SCrawler.Shared\SCrawler.Shared.vbproj">
+      <Project>{dc634700-24c7-42dd-bf8f-87e6cc54e625}</Project>
+      <Name>SCrawler.Shared</Name>
+    </ProjectReference>
   </ItemGroup>
   <ItemGroup>
     <None Include="Content\Pictures\YouTubeMusicPic_96.png" />

SCrawler.YouTubeDownloader/.editorconfig

@@ -1,3 +1,3 @@
 [*.vb]
 # Modifier preferences
-file_header_template = Copyright (C) 2023  Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
 <Assembly: AssemblyDescription("SCrawler YouTube downloader")>
 <Assembly: AssemblyCompany("AndyProgram")>
 <Assembly: AssemblyProduct("SCrawler.YouTubeDownloader")>
-<Assembly: AssemblyCopyright("Copyright © 2023")>
+<Assembly: AssemblyCopyright("Copyright © 2024")>
 <Assembly: AssemblyTrademark("AndyProgram")>
 
 <Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.10.10.0")>
-<Assembly: AssemblyFileVersion("2023.10.10.0")>
+<Assembly: AssemblyVersion("2023.12.7.0")>
+<Assembly: AssemblyFileVersion("2023.12.7.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.sln

@@ -4,6 +4,9 @@ Microsoft Visual Studio Solution File, Format Version 12.00
 VisualStudioVersion = 16.0.31515.178
 MinimumVisualStudioVersion = 10.0.40219.1
 Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler", "SCrawler\SCrawler.vbproj", "{4A016FAD-9F07-4957-8BB2-AE86C88BA342}"
+	ProjectSection(ProjectDependencies) = postProject
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA} = {71263EEE-E25F-44DD-B0A9-F09047C0BEEA}
+	EndProjectSection
 EndProject
 Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "PersonalUtilities", "..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj", "{8405896B-2685-4916-BC93-1FB514C323A9}"
 EndProject
@@ -22,6 +25,10 @@ Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.YouTube", "SCrawle
 EndProject
 Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.YouTubeDownloader", "SCrawler.YouTubeDownloader\SCrawler.YouTubeDownloader.vbproj", "{3F2F2C29-4ADB-48B5-A66E-EE0F51D0DCEF}"
 EndProject
+Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.Updater", "SCrawler.Updater\SCrawler.Updater.vbproj", "{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}"
+EndProject
+Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.Shared", "SCrawler.Shared\SCrawler.Shared.vbproj", "{DC634700-24C7-42DD-BF8F-87E6CC54E625}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -104,6 +111,30 @@ Global
 		{3F2F2C29-4ADB-48B5-A66E-EE0F51D0DCEF}.Release|x64.Build.0 = Release|x64
 		{3F2F2C29-4ADB-48B5-A66E-EE0F51D0DCEF}.Release|x86.ActiveCfg = Release|x86
 		{3F2F2C29-4ADB-48B5-A66E-EE0F51D0DCEF}.Release|x86.Build.0 = Release|x86
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|x64.ActiveCfg = Debug|x64
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|x64.Build.0 = Debug|x64
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|x86.ActiveCfg = Debug|x86
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Debug|x86.Build.0 = Debug|x86
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|Any CPU.Build.0 = Release|Any CPU
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|x64.ActiveCfg = Release|x64
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|x64.Build.0 = Release|x64
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|x86.ActiveCfg = Release|x86
+		{71263EEE-E25F-44DD-B0A9-F09047C0BEEA}.Release|x86.Build.0 = Release|x86
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|x64.ActiveCfg = Debug|x64
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|x64.Build.0 = Debug|x64
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|x86.ActiveCfg = Debug|x86
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Debug|x86.Build.0 = Debug|x86
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|Any CPU.Build.0 = Release|Any CPU
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|x64.ActiveCfg = Release|x64
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|x64.Build.0 = Release|x64
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|x86.ActiveCfg = Release|x86
+		{DC634700-24C7-42DD-BF8F-87E6CC54E625}.Release|x86.Build.0 = Release|x86
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE

SCrawler/.editorconfig

@@ -123,4 +123,4 @@ insert_final_newline=false
 [*.vb]
 # Modifier preferences
 visual_basic_preferred_modifier_order = Partial,Default,Private,Protected,Public,Friend,NotOverridable,Overridable,MustOverride,Overloads,Overrides,MustInherit,NotInheritable,Static,Shared,Shadows,ReadOnly,WriteOnly,Dim,Const,WithEvents,Widening,Narrowing,Custom,Async:suggestion
-file_header_template = Copyright (C) 2023  Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>
+file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler/API/Base/DeclaredNames.vb

@@ -11,6 +11,8 @@ Namespace API.Base
         Friend Const Header_Authorization As String = "authorization"
         Friend Const Header_CSRFToken As String = "x-csrf-token"
 
+        Friend Const Header_FB_FRIENDLY_NAME As String = "x-fb-friendly-name"
+
         Friend Const ConcurrentDownloadsCaption As String = "Concurrent downloads"
         Friend Const ConcurrentDownloadsToolTip As String = "The number of concurrent downloads."
         Friend Const SavedPostsUserNameCaption As String = "Saved posts user"

SCrawler/API/Base/IUserData.vb

@@ -50,6 +50,8 @@ Namespace API.Base
         Property Suspended As Boolean
         Property ReadyForDownload As Boolean
         Property HOST As SettingsHost
+        Property HostStatic As Boolean
+        Property AccountName As String
         Property [File] As SFile
         Property FileExists As Boolean
         Property DownloadedPictures(ByVal Total As Boolean) As Integer

SCrawler/API/Base/ProfileSaved.vb

@@ -10,45 +10,93 @@ Imports System.Threading
 Imports SCrawler.Plugin.Hosts
 Imports PersonalUtilities.Forms.Toolbars
 Imports PDownload = SCrawler.Plugin.ISiteSettings.Download
+Imports UserMediaD = SCrawler.DownloadObjects.TDownloader.UserMediaD
 Namespace API.Base
     Friend NotInheritable Class ProfileSaved
-        Private ReadOnly Property HOST As SettingsHost
+        Private ReadOnly Property HOST As SettingsHostCollection
         Private ReadOnly Property Progress As MyProgress
-        Friend Sub New(ByRef h As SettingsHost, ByRef Bar As MyProgress)
+        Private _Unavailable As Integer, _NotReady As Integer, _ErrorCount As Integer
+        Private _TotalImages As Integer, _TotalVideos As Integer
+        Friend Property Session As Integer
+        Friend Property IncludeInTheFeed As Boolean = False
+        Private _FeedDataExists As Boolean = False
+        Friend ReadOnly Property FeedDataExists As Boolean
+            Get
+                Return _FeedDataExists
+            End Get
+        End Property
+        Friend Sub New(ByRef h As SettingsHostCollection, ByRef Bar As MyProgress)
             HOST = h
             Progress = Bar
         End Sub
-        Friend Sub Download(ByVal Token As CancellationToken, ByVal Multiple As Boolean)
+        Friend Overloads Sub Download(ByVal Token As CancellationToken, ByVal Multiple As Boolean)
+            Dim n% = 0
+            Dim c% = HOST.Sum(Function(h) IIf(h.DownloadSavedPosts, 1, 0))
+            _FeedDataExists = False
+            _Unavailable = 0
+            _NotReady = 0
+            _ErrorCount = 0
+            _TotalImages = 0
+            _TotalVideos = 0
+            If c > 0 Then
+                For i% = 0 To HOST.Count - 1
+                    If Not Token.IsCancellationRequested And HOST(i).DownloadSavedPosts Then n += 1 : Download(HOST(i), n, c, Token, Multiple)
+                Next
+                If c > 1 Then
+                    Dim s% = {_Unavailable, _NotReady, _ErrorCount}.Sum
+                    Progress.InformationTemporary = $"{HOST.Name} ({c - s}/{c}) Images: {_TotalImages}; Videos: {_TotalVideos}"
+                End If
+            End If
+        End Sub
+        Private Overloads Sub Download(ByVal Host As SettingsHost, ByVal Number As Integer, ByVal Count As Integer,
+                                       ByVal Token As CancellationToken, ByVal Multiple As Boolean)
+            Dim aStr$ = String.Empty
+            If Count > 1 Then aStr = $" ({Number}/{Count})"
             Try
-                If HOST.Source.ReadyToDownload(PDownload.SavedPosts) Then
-                    If HOST.Available(PDownload.SavedPosts, Multiple) Then
-                        HOST.DownloadStarted(PDownload.SavedPosts)
-                        Dim u As New UserInfo With {.Plugin = HOST.Key, .Site = HOST.Name, .SpecialPath = HOST.SavedPostsPath}
-                        Using user As IUserData = HOST.GetInstance(PDownload.SavedPosts, Nothing, False, False)
+                If Host.Source.ReadyToDownload(PDownload.SavedPosts) Then
+                    If Host.Available(PDownload.SavedPosts, Multiple Or Count > 1) Then
+                        Host.DownloadStarted(PDownload.SavedPosts)
+                        If Count > 1 Then Progress.Information = $"{Host.Name} - {Host.AccountName.IfNullOrEmpty(SettingsHost.NameAccountNameDefault)}"
+                        Using user As IUserData = Host.GetInstance(PDownload.SavedPosts, Nothing, False, False)
                             If Not user Is Nothing Then
                                 With DirectCast(user, UserDataBase)
+                                    .HostStatic = True
                                     .IsSavedPosts = True
                                     .LoadUserInformation()
                                     .Progress = Progress
                                     If Not .FileExists Then .UpdateUserInformation()
+                                    .IncludeInTheFeed = IncludeInTheFeed
+
+                                    Host.BeforeStartDownload(.Self, PDownload.SavedPosts)
+                                    .DownloadData(Token)
+                                    _TotalImages += .DownloadedPictures(False)
+                                    _TotalVideos += .DownloadedVideos(False)
+                                    If IncludeInTheFeed And .LatestData.Count > 0 Then
+                                        _FeedDataExists = True
+                                        Downloader.Files.AddRange(.LatestData.Select(Function(m) New UserMediaD(m, .Self, Session) With {.IsSavedPosts = True}))
+                                    End If
+                                    Progress.InformationTemporary = $"{Host.Name}{aStr} Images: { .DownloadedPictures(False)}; Videos: { .DownloadedVideos(False)}"
+                                    Host.AfterDownload(.Self, PDownload.SavedPosts)
                                 End With
-                                HOST.BeforeStartDownload(user, PDownload.SavedPosts)
-                                user.DownloadData(Token)
-                                Progress.InformationTemporary = $"{HOST.Name} Images: {user.DownloadedPictures(False)}; Videos: {user.DownloadedVideos(False)}"
-                                HOST.AfterDownload(user, PDownload.SavedPosts)
                             End If
                         End Using
                     Else
-                        Progress.InformationTemporary = $"Host [{HOST.Name}] is unavailable"
+                        _Unavailable += 1
+                        Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is unavailable"
                     End If
                 Else
-                    Progress.InformationTemporary = $"Host [{HOST.Name}] is not ready"
+                    _NotReady += 1
+                    Progress.InformationTemporary = $"Host [{Host.Name}{aStr}] is not ready"
                 End If
+            Catch oex As OperationCanceledException When Token.IsCancellationRequested
+                _ErrorCount += 1
+                Progress.InformationTemporary = $"{Host.Name}{aStr} downloading canceled"
             Catch ex As Exception
-                Progress.InformationTemporary = $"{HOST.Name} downloading error"
-                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"[API.Base.ProfileSaved.Download({HOST.Key})]")
+                _ErrorCount += 1
+                Progress.InformationTemporary = $"{Host.Name}{aStr} downloading error"
+                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"[API.Base.ProfileSaved.Download({Host.Key}{aStr})]")
             Finally
-                HOST.DownloadDone(PDownload.SavedPosts)
+                Host.DownloadDone(PDownload.SavedPosts)
                 MainFrameObj.UpdateLogButton()
             End Try
         End Sub

SCrawler/API/Base/SiteSettingsBase.vb

@@ -6,15 +6,33 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Reflection
 Imports SCrawler.Plugin
+Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Tools
+Imports PersonalUtilities.Tools.Web.Cookies
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports Download = SCrawler.Plugin.ISiteSettings.Download
 Namespace API.Base
     Friend MustInherit Class SiteSettingsBase : Implements ISiteSettings, IResponserContainer
+#Region "Declarations"
         Friend ReadOnly Property Site As String Implements ISiteSettings.Site
+        Protected _Icon As Icon = Nothing
         Friend Overridable ReadOnly Property Icon As Icon Implements ISiteSettings.Icon
+            Get
+                Return _Icon
+            End Get
+        End Property
+        Protected _Image As Image = Nothing
         Friend Overridable ReadOnly Property Image As Image Implements ISiteSettings.Image
+            Get
+                Return _Image
+            End Get
+        End Property
+        Friend Property AccountName As String Implements ISiteSettings.AccountName
+        Friend Property Temporary As Boolean = False Implements ISiteSettings.Temporary
+        Friend Property DefaultInstance As ISiteSettings = Nothing Implements ISiteSettings.DefaultInstance
         Protected _AllowUserAgentUpdate As Boolean = True
         Protected _SubscriptionsAllowed As Boolean = False
         Friend ReadOnly Property SubscriptionsAllowed As Boolean Implements ISiteSettings.SubscriptionsAllowed
@@ -24,7 +42,25 @@ Namespace API.Base
         End Property
         Private Property Logger As ILogProvider = LogConnector Implements ISiteSettings.Logger
         Friend Overridable ReadOnly Property Responser As Responser
+        Private _UserOptionsExists As Boolean = False
+        Private _UserOptionsType As Type = Nothing
+        Protected Property UserOptionsType As Type
+            Get
+                Return _UserOptionsType
+            End Get
+            Set(ByVal t As Type)
+                _UserOptionsType = t
+                _UserOptionsExists = Not t Is Nothing
+            End Set
+        End Property
+#End Region
+#Region "Responser and cookies support"
+        Private _CookiesNetscapeFile As SFile = Nothing
         Friend ReadOnly Property CookiesNetscapeFile As SFile
+            Get
+                Return _CookiesNetscapeFile
+            End Get
+        End Property
         Protected CheckNetscapeCookiesOnEndInit As Boolean = False
         Private _UseNetscapeCookies As Boolean = False
         Protected Property UseNetscapeCookies As Boolean
@@ -38,7 +74,7 @@ Namespace API.Base
                     Responser.Cookies.ChangedAllowInternalDrop = Not _UseNetscapeCookies
                     Responser.Cookies.Changed = False
                 End If
-                If b And _UseNetscapeCookies Then Update_SaveCookiesNetscape()
+                If b AndAlso _UseNetscapeCookies AndAlso Not CookiesNetscapeFile.Exists Then Update_SaveCookiesNetscape()
             End Set
         End Property
         Private Property IResponserContainer_Responser As Responser Implements IResponserContainer.Responser
@@ -47,22 +83,46 @@ Namespace API.Base
             End Get
             Set : End Set
         End Property
+        Protected Sub UpdateResponserFile()
+            Dim acc$ = If(AccountName.IsEmptyString OrElse AccountName = Hosts.SettingsHost.NameAccountNameDefault, String.Empty, $"_{AccountName}")
+            Responser.File = $"{SettingsFolderName}\Responser_{Site}{acc}.xml"
+            _CookiesNetscapeFile = Responser.File
+            _CookiesNetscapeFile.Name &= "_Cookies_Netscape"
+            _CookiesNetscapeFile.Extension = "txt"
+        End Sub
+#End Region
+#Region "GetInstance"
         Friend MustOverride Function GetInstance(ByVal What As Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
-        Friend Sub New(ByVal SiteName As String)
+#End Region
+#Region "Initializers"
+        Friend Sub New(ByVal SiteName As String, Optional ByVal __Icon As Icon = Nothing, Optional ByVal __Image As Image = Nothing)
             Site = SiteName
-            CookiesNetscapeFile = $"{SettingsFolderName}\Responser_{Site}_Cookies_Netscape.txt"
+            _Icon = __Icon
+            _Image = __Image
+            Responser = New Responser With {.DeclaredError = EDP.ThrowException}
+            UpdateResponserFile()
         End Sub
-        Friend Sub New(ByVal SiteName As String, ByVal CookiesDomain As String)
-            Me.New(SiteName)
-            Responser = New Responser($"{SettingsFolderName}\Responser_{Site}.xml") With {.DeclaredError = EDP.ThrowException}
-            With Responser
-                .CookiesDomain = CookiesDomain
-                .CookiesEncryptKey = SettingsCLS.CookieEncryptKey
-                If .File.Exists Then .LoadSettings() Else .SaveSettings()
-            End With
-        End Sub
-#Region "XML"
-        Friend Overridable Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) Implements ISiteSettings.Load
+        Friend Sub New(ByVal SiteName As String, ByVal CookiesDomain As String, ByVal AccName As String, ByVal Temp As Boolean,
+                       Optional ByVal __Icon As Icon = Nothing, Optional ByVal __Image As Image = Nothing)
+            Me.New(SiteName, __Icon, __Image)
+            Temporary = Temp
+            AccountName = AccName
+            If Temporary Then
+                With Responser
+                    .File = Nothing
+                    .Cookies.File = Nothing
+                    .CookiesDomain = CookiesDomain
+                    .CookiesEncryptKey = SettingsCLS.CookieEncryptKey
+                End With
+                _CookiesNetscapeFile = Nothing
+            Else
+                UpdateResponserFile()
+                With Responser
+                    .CookiesDomain = CookiesDomain
+                    .CookiesEncryptKey = SettingsCLS.CookieEncryptKey
+                    If .File.Exists Then .LoadSettings() Else .SaveSettings()
+                End With
+            End If
         End Sub
 #End Region
 #Region "Initialize"
@@ -81,12 +141,18 @@ Namespace API.Base
         Protected _SiteEditorFormOpened As Boolean = False
         Friend Overridable Sub BeginEdit() Implements ISiteSettings.BeginEdit
             _SiteEditorFormOpened = True
+            If UseNetscapeCookies And CookiesNetscapeFile.Exists Then
+                With Responser.Cookies
+                    .Clear()
+                    .AddRange(CookieKeeper.ParseNetscapeText(CookiesNetscapeFile.GetText, EDP.SendToLog + EDP.ReturnValue),, EDP.None)
+                End With
+            End If
         End Sub
         Friend Overridable Sub EndEdit() Implements ISiteSettings.EndEdit
             If _SiteEditorFormOpened Then DomainsReset()
             _SiteEditorFormOpened = False
         End Sub
-        Friend Overridable Sub Update() Implements ISiteSettings.Update
+        Friend Overridable Overloads Sub Update() Implements ISiteSettings.Update
             If _SiteEditorFormOpened Then
                 If UseNetscapeCookies Then Update_SaveCookiesNetscape()
                 If Not Responser Is Nothing Then
@@ -98,6 +164,17 @@ Namespace API.Base
             End If
             If Not Responser Is Nothing Then Responser.SaveSettings()
         End Sub
+        Friend Overridable Overloads Sub Update(ByVal Source As ISiteSettings) Implements ISiteSettings.Update
+            AccountName = Source.AccountName
+            If Not Responser Is Nothing Then Responser.Copy(DirectCast(Source, SiteSettingsBase).Responser) : UpdateResponserFile() : Responser.SaveSettings()
+            Update_CloneProperties(Source)
+            UpdateImpl(Source)
+        End Sub
+        Protected Overridable Sub Update_CloneProperties(ByVal Source As ISiteSettings)
+            CLONE_PROPERTIES(Source, Me, True)
+        End Sub
+        Protected Overridable Sub UpdateImpl(ByVal Source As ISiteSettings)
+        End Sub
         Protected Sub Update_SaveCookiesNetscape(Optional ByVal Force As Boolean = False, Optional ByVal IsInit As Boolean = False)
             If Not Responser Is Nothing Then
                 With Responser
@@ -142,6 +219,7 @@ Namespace API.Base
         End Sub
         ''' <inheritdoc cref="DownloadStarted(Download)"/>
         Friend Overridable Sub DownloadDone(ByVal What As Download) Implements ISiteSettings.DownloadDone
+            AvailableText = String.Empty
         End Sub
 #End Region
 #Region "User info"
@@ -184,6 +262,7 @@ Namespace API.Base
         End Function
 #End Region
 #Region "Ready, Available"
+        Friend Property AvailableText As String Implements ISiteSettings.AvailableText
         ''' <returns>True</returns>
         Friend Overridable Function BaseAuthExists() As Boolean
             Return True
@@ -199,12 +278,119 @@ Namespace API.Base
             Return True
         End Function
 #End Region
+        Protected Sub CLONE_PROPERTIES(ByVal Source As ISiteSettings, ByVal Destination As ISiteSettings, ByVal IsUpdate As Boolean,
+                                       Optional ByVal Full As Boolean = True)
+            Dim comparer As New MembersDistinctComparerExtended
+            '0 = update
+            '1 = clone
+            '2 = any
+            Dim filterUC As Func(Of MemberInfo, Byte, Boolean) = Function(ByVal m As MemberInfo, ByVal __mode As Byte) As Boolean
+                                                                     If If(m.GetCustomAttribute(Of DoNotUse)?.Value, False) Then
+                                                                         Return False
+                                                                     Else
+                                                                         With m.GetCustomAttribute(Of PClonableAttribute)
+                                                                             Return Not .Self Is Nothing AndAlso (__mode = 2 OrElse If(__mode = 0, .Update, .Clone))
+                                                                         End With
+                                                                     End If
+                                                                 End Function
+            Dim filterAll As Func(Of MemberInfo, Boolean) = Function(m) filterUC.Invoke(m, 2)
+            Dim filterC As Func(Of MemberInfo, Boolean) = Function(m) If(Full, filterAll.Invoke(m), filterUC.Invoke(m, 1))
+            Dim filterU As Func(Of MemberInfo, Boolean) = Function(m) filterUC.Invoke(m, 0)
+            Dim membersSource As IEnumerable(Of MemberInfo) = GetObjectMembers(Source, filterAll,, True, comparer)
+            If membersSource.ListExists Then
+                Dim membersDest As IEnumerable(Of MemberInfo) = GetObjectMembers(Destination, If(IsUpdate, filterU, filterC),, True, comparer)
+                If membersDest.ListExists Then
+                    Dim mSource As MemberInfo = Nothing, mDest As MemberInfo = Nothing
+                    Dim destIndx%
+                    Dim isPropertyValue As Boolean
+                    Dim sourceValue As Object
+                    For Each mSource In membersSource
+                        destIndx = membersDest.ListIndexOf(mSource, comparer, EDP.ReturnValue)
+                        If destIndx.ValueBetween(0, membersDest.Count - 1) Then mDest = membersDest(destIndx) Else mDest = Nothing
+                        If Not mDest Is Nothing Then
+                            sourceValue = mSource.GetMemberValue(Source)
+                            If mDest.MemberType = MemberTypes.Property Then
+                                isPropertyValue = DirectCast(mDest, PropertyInfo).PropertyType Is GetType(PropertyValue)
+                            Else
+                                isPropertyValue = DirectCast(mDest, FieldInfo).FieldType Is GetType(PropertyValue)
+                            End If
+                            If isPropertyValue Then
+                                DirectCast(mDest.GetMemberValue(Destination), PropertyValue).Clone(sourceValue)
+                            Else
+                                mDest.SetMemberValue(Destination, sourceValue)
+                            End If
+                        End If
+                    Next
+                End If
+            End If
+        End Sub
+        Protected Overridable Function CloneGetEmptySettingsInstance() As ISiteSettings
+            Dim _max% = -1
+            Dim c As ConstructorInfo = Nothing
+            With Me.GetType.GetTypeInfo.DeclaredConstructors
+                If .ListExists Then
+                    With .Where(Function(m) If(m.GetParameters?.Count, 0).ValueBetween(0, 2))
+                        If .ListExists Then
+                            _max = .Max(Function(m) If(m.GetParameters?.Count, 0))
+                            c = .First(Function(m) If(m.GetParameters?.Count, 0) = _max)
+                        End If
+                    End With
+                End If
+                Select Case _max
+                    Case 2 : Return c.Invoke({String.Empty, True})
+                    Case 1 : Return c.Invoke({String.Empty})
+                    Case 0 : Return c.Invoke(Nothing)
+                    Case Else : Return Activator.CreateInstance(Me.GetType)
+                End Select
+            End With
+        End Function
+        Friend Overridable Function Clone(ByVal Full As Boolean) As ISiteSettings Implements ISiteSettings.Clone
+            Dim obj As ISiteSettings = CloneGetEmptySettingsInstance()
+            CLONE_PROPERTIES(Me, obj, False, Full)
+            Return obj
+        End Function
+        Friend Sub Delete() Implements ISiteSettings.Delete
+            If Not Responser Is Nothing Then
+                With Responser
+                    If .File.Exists Then .File.Delete(SFO.File, SFODelete.DeleteToRecycleBin, EDP.None)
+                    If .Cookies.File.Exists Then .Cookies.File.Delete(SFO.File, SFODelete.DeleteToRecycleBin, EDP.None)
+                End With
+                If _CookiesNetscapeFile.Exists Then _CookiesNetscapeFile.Delete(SFO.File, SFODelete.DeleteToRecycleBin, EDP.None)
+            End If
+        End Sub
         Friend Overridable Sub Reset() Implements ISiteSettings.Reset
         End Sub
         Friend Overridable Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
-            Options = Nothing
+            If _UserOptionsExists Then
+                If Options Is Nothing OrElse Not Options.GetType Is _UserOptionsType Then
+                    Options = AConvert(Me, AModes.Var, _UserOptionsType,, True, Nothing)
+                    If Options Is Nothing Then Options = Activator.CreateInstance(_UserOptionsType)
+                End If
+                If OpenForm Then
+                    Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
+                End If
+            Else
+                Options = Nothing
+            End If
         End Sub
         Friend Overridable Sub OpenSettingsForm() Implements ISiteSettings.OpenSettingsForm
         End Sub
+#Region "IDisposable Support"
+        Protected disposedValue As Boolean = False
+        Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
+            If Not disposedValue Then
+                If disposing Then Responser.DisposeIfReady
+                disposedValue = True
+            End If
+        End Sub
+        Protected Overrides Sub Finalize()
+            Dispose(False)
+            MyBase.Finalize()
+        End Sub
+        Friend Overloads Sub Dispose() Implements IDisposable.Dispose
+            Dispose(True)
+            GC.SuppressFinalize(Me)
+        End Sub
+#End Region
     End Class
 End Namespace

SCrawler/API/Base/Structures.vb

@@ -18,6 +18,7 @@ Namespace API.Base
             Tags = 2
             Categories = 3
             Pornstars = 4
+            Playlists = 5
         End Enum
         Friend Structure UserMedia : Implements IUserMedia, IEquatable(Of UserMedia), IEContainerProvider
 #Region "XML Names"

SCrawler/API/Base/UserDataBase.vb

@@ -126,6 +126,7 @@ Namespace API.Base
 #Region "XML Declarations"
         Private Const Name_Site As String = UserInfo.Name_Site
         Private Const Name_Plugin As String = UserInfo.Name_Plugin
+        Private Const Name_AccountName As String = UserInfo.Name_AccountName
         Protected Const Name_IsChannel As String = "IsChannel"
         Friend Const Name_UserName As String = "UserName"
         Private Const Name_Model_User As String = UserInfo.Name_Model_User
@@ -168,10 +169,56 @@ Namespace API.Base
         Protected Const Name_UseMD5Comparison As String = "UseMD5Comparison"
         Protected Const Name_RemoveExistingDuplicates As String = "RemoveExistingDuplicates"
         Protected Const Name_StartMD5Checked As String = "StartMD5Checked"
+#Region "Additional names"
+        Protected Const Name_SiteMode As String = "SiteMode"
+        Protected Const Name_TrueName As String = "TrueName"
+        Protected Const Name_Arguments As String = "Arguments"
+#End Region
 #End Region
 #Region "Declarations"
 #Region "Host, Site, Progress"
+        Friend Property HostCollection As SettingsHostCollection
+        Private Function HostObtainCollection() As Boolean
+            If HostCollection Is Nothing Then
+                Dim k$ = If(_HOST?.Key, _HostKey)
+                If Not k.IsEmptyString Then HostCollection = Settings(k)
+            End If
+            Return Not HostCollection Is Nothing
+        End Function
+        Private _HOST As SettingsHost
+        Private _HostKey As String = String.Empty
+        Private _HostObtained As Boolean = False
         Friend Property HOST As SettingsHost Implements IUserData.HOST
+            Get
+                If _HostObtained Or HostStatic Then
+                    Return _HOST
+                ElseIf HostObtainCollection() Then
+                    _HOST = HostCollection(AccountName)
+                    _HostObtained = Not _HOST Is Nothing
+                    Return _HOST
+                Else
+                    Return Nothing
+                End If
+            End Get
+            Set(ByVal h As SettingsHost)
+                _HOST = h
+                If Not h Is Nothing Then _HostKey = h.Key
+            End Set
+        End Property
+        Private Sub ResetHost()
+            _HostObtained = False
+        End Sub
+        Friend Property HostStatic As Boolean = False Implements IUserData.HostStatic
+        Private _AccountName As String = String.Empty
+        Friend Overridable Property AccountName As String Implements IUserData.AccountName, IPluginContentProvider.AccountName
+            Get
+                Return _AccountName.IfNullOrEmpty(User.AccountName)
+            End Get
+            Set(ByVal name As String)
+                If Not _AccountName = name Then ResetHost()
+                _AccountName = name
+            End Set
+        End Property
         Friend ReadOnly Property Site As String Implements IUserData.Site
             Get
                 Return HOST.Name
@@ -845,6 +892,7 @@ BlockNullPicture:
         Friend Sub SetEnvironment(ByRef h As SettingsHost, ByVal u As UserInfo, ByVal _LoadUserInformation As Boolean,
                                   Optional ByVal AttachUserInfo As Boolean = True) Implements IUserData.SetEnvironment
             HOST = h
+            HostObtainCollection()
             If AttachUserInfo Then
                 User = u
                 If _LoadUserInformation Then LoadUserInformation()
@@ -853,7 +901,7 @@ BlockNullPicture:
         ''' <exception cref="ArgumentOutOfRangeException"></exception>
         Friend Shared Function GetInstance(ByVal u As UserInfo, Optional ByVal _LoadUserInformation As Boolean = True) As IUserData
             If Not u.Plugin.IsEmptyString Then
-                Return Settings(u.Plugin).GetInstance(ISiteSettings.Download.Main, u, _LoadUserInformation)
+                Return Settings(u.Plugin).Default.GetInstance(ISiteSettings.Download.Main, u, _LoadUserInformation)
             Else
                 Throw New ArgumentOutOfRangeException("Plugin", $"Plugin [{u.Plugin}] information does not recognized by loader")
             End If
@@ -865,7 +913,7 @@ BlockNullPicture:
                     With DirectCast(u, UserDataBase)
                         If Not .User.Plugin.IsEmptyString Then
                             uName = .User.Name
-                            Return Settings(.User.Plugin).GetUserPostUrl(.Self, PostData)
+                            Return Settings(.User.Plugin).Default.GetUserPostUrl(.Self, PostData)
                         End If
                     End With
                 End If
@@ -937,6 +985,7 @@ BlockNullPicture:
                 Using x As New XmlFile With {.Name = "User"}
                     x.Add(Name_Site, Site)
                     x.Add(Name_Plugin, HOST.Key)
+                    x.Add(Name_AccountName, AccountName)
                     x.Add(Name_UserName, User.Name)
                     x.Add(Name_Model_User, CInt(UserModel))
                     x.Add(Name_Model_Collection, CInt(CollectionModel))
@@ -1030,6 +1079,7 @@ BlockNullPicture:
             End Try
         End Sub
         Friend Overridable Sub OpenFolder() Implements IUserData.OpenFolder
+            If MyFile.IsEmptyString And IsSavedPosts Then UpdateDataFiles()
             GlobalOpenPath(MyFile.CutPath)
         End Sub
 #End Region
@@ -1130,11 +1180,13 @@ BlockNullPicture:
             End If
         End Sub
         Friend Overridable Sub DownloadData(ByVal Token As CancellationToken) Implements IUserData.DownloadData
+            ResetHost()
             __DOWNLOAD_IN_PROGRESS = True
             OnUserDownloadStateChanged(True)
             Dim Canceled As Boolean = False
             TokenQueue = Token
             Try
+                If HOST Is Nothing Then Throw New ExitException($"Host '{AccountName}' not found")
                 EnvirDownloadSet()
                 If Not Responser Is Nothing Then Responser.Dispose()
                 Responser = New Responser
@@ -1291,7 +1343,12 @@ BlockNullPicture:
 #Region "DownloadSingleObject"
         Protected IsSingleObjectDownload As Boolean = False
         Friend Overridable Sub DownloadSingleObject(ByVal Data As YouTube.Objects.IYouTubeMediaContainer, ByVal Token As CancellationToken) Implements IUserData.DownloadSingleObject
+            Dim URL$ = String.Empty
             Try
+                ResetHost()
+                URL = Data.URL
+                AccountName = Data.AccountName
+                If HOST Is Nothing Then Throw New ExitException($"Host '{AccountName}' not found")
                 Data.DownloadState = UserMediaStates.Tried
                 Progress = Data.Progress
                 If Not Progress Is Nothing Then Progress.ResetProgressOnMaximumChanges = False
@@ -1307,11 +1364,19 @@ BlockNullPicture:
                 DownloadSingleObject_PostProcessing(Data)
             Catch oex As OperationCanceledException When Token.IsCancellationRequested
                 Data.DownloadState = UserMediaStates.Missing
-                ErrorsDescriber.Execute(EDP.SendToLog, oex, $"{Site} download canceled: {Data.URL}")
+                ErrorsDescriber.Execute(EDP.SendToLog, oex, $"{Site} download canceled: {URL}")
             Catch dex As ObjectDisposedException When Disposed
+            Catch exit_ex As ExitException
+                If Not exit_ex.Silent Then
+                    If exit_ex.SimpleLogLine Then
+                        MyMainLOG = $"{URL}: downloading canceled (exit) ({exit_ex.Message})"
+                    Else
+                        ErrorsDescriber.Execute(EDP.SendToLog, exit_ex, $"{URL}: downloading canceled (exit)")
+                    End If
+                End If
             Catch ex As Exception
                 Data.DownloadState = UserMediaStates.Missing
-                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{Site} single data downloader error: {Data.URL}")
+                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{Site} single data downloader error: {URL}")
             End Try
         End Sub
         Protected Overridable Sub DownloadSingleObject_CreateMedia(ByVal Data As YouTube.Objects.IYouTubeMediaContainer, ByVal Token As CancellationToken)
@@ -1665,8 +1730,6 @@ BlockNullPicture:
                                         DownloadContentDefault_PostProcessing(v, f, Token)
                                         dCount += 1
                                     Catch woex As OperationCanceledException When Token.IsCancellationRequested
-                                        'TODELETE: UserDataBase.DownloadContentDefault: remove file when 'OperationCanceledException'
-                                        'If f.Exists Then f.Delete(,, EDP.SendToLog)
                                         __deleteFile.Invoke(f, v.URL_BASE)
                                         v.State = UStates.Missing
                                         v.Attempts += 1
@@ -1878,6 +1941,7 @@ BlockNullPicture:
             Try
                 Dim f As SFile
                 Dim v As Boolean = IsVirtual
+                Settings.Feeds.Load()
 
                 If IncludedInCollection And __CollectionName.IsEmptyString And __SpecialCollectionPath.IsEmptyString Then
                     Settings.Users.Add(Me)
@@ -1920,6 +1984,7 @@ BlockNullPicture:
 
                 Settings.UsersList.Remove(UserBefore)
                 Settings.UpdateUsersList(User)
+                Settings.Feeds.UpdateUsers(UserBefore, User)
                 UpdateUserInformation()
                 Return True
             Catch ex As Exception
@@ -1973,6 +2038,7 @@ BlockNullPicture:
                     End If
                     If Not ScriptData.IsEmptyString AndAlso ScriptData.Contains(UserBefore.File.PathNoSeparator) Then _
                        ScriptData = ScriptData.Replace(UserBefore.File.PathNoSeparator, MyFile.PathNoSeparator)
+                    Settings.Feeds.UpdateUsers(UserBefore, User)
                     UpdateUserInformation()
                 End If
             Catch ioex As InvalidOperationException When ioex.HelpLink = 1

SCrawler/API/BaseObjects/DomainsContainer.vb

@@ -11,7 +11,7 @@ Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Tools
 Imports ADB = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons
 Namespace API.Base
-    Friend Class DomainsContainer : Implements IEnumerable(Of String), IMyEnumerator(Of String)
+    Friend Class DomainsContainer : Implements IEnumerable(Of String), IMyEnumerator(Of String), IDisposable
         Friend Event DomainsUpdated(ByVal Sender As DomainsContainer)
         Friend ReadOnly Property Domains As List(Of String)
         Friend ReadOnly Property DomainsTemp As List(Of String)
@@ -98,11 +98,33 @@ Namespace API.Base
                 End If
             End Using
         End Sub
+#Region "IEnumerable Support"
         Private Function GetEnumerator() As IEnumerator(Of String) Implements IEnumerable(Of String).GetEnumerator
             Return New MyEnumerator(Of String)(Me)
         End Function
         Private Function IEnumerable_GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator
             Return GetEnumerator()
         End Function
+#End Region
+#Region "IDisposable Support"
+        Private disposedValue As Boolean = False
+        Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
+            If Not disposedValue Then
+                If disposing Then
+                    Domains.Clear()
+                    DomainsTemp.Clear()
+                End If
+                disposedValue = True
+            End If
+        End Sub
+        Protected Overrides Sub Finalize()
+            Dispose(False)
+            MyBase.Finalize()
+        End Sub
+        Friend Overloads Sub Dispose() Implements IDisposable.Dispose
+            Dispose(True)
+            GC.SuppressFinalize(Me)
+        End Sub
+#End Region
     End Class
 End Namespace

SCrawler/API/BaseObjects/InternalSettingsForm.vb

@@ -20,7 +20,7 @@ Namespace API.Base
         Private ReadOnly Property MyMembers As List(Of MemberOption)
         ''' <summary>Default: 200</summary>
         Friend Property MinimumWidth As Integer = 200
-        Private Class MemberOption : Inherits Hosts.PropertyValueHost : Implements IDisposable
+        Private Class MemberOption : Inherits Hosts.PropertyValueHost
             Friend ToolTip As String
             Friend Caption As String
             Friend ThreeState As Boolean = False
@@ -102,24 +102,6 @@ Namespace API.Base
                 CreateControl(TT)
                 If Not Provider Is Nothing Then f.AddControl(Control, Caption, Type, AllowNull, Activator.CreateInstance(Provider))
             End Sub
-#Region "IDisposable Support"
-            Private disposedValue As Boolean = False
-            Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
-                If Not disposedValue Then
-                    If disposing Then Control.Dispose()
-                    Control = Nothing
-                    disposedValue = True
-                End If
-            End Sub
-            Protected Overrides Sub Finalize()
-                Dispose(False)
-                MyBase.Finalize()
-            End Sub
-            Friend Overloads Sub Dispose() Implements IDisposable.Dispose
-                Dispose(True)
-                GC.SuppressFinalize(Me)
-            End Sub
-#End Region
         End Class
         Friend Sub New(ByVal Obj As Object, ByVal s As ISiteSettings, ByVal _IsSettingsForm As Boolean)
             InitializeComponent()

SCrawler/API/Facebook/Declarations.vb

@@ -0,0 +1,37 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Text.RegularExpressions
+Imports PersonalUtilities.Functions.XML.Base
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.Facebook
+    Friend Module Declarations
+        Friend ReadOnly Regex_UserToken_dtsg As RParams = RParams.DMS("DTSGInitialData.:.?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_UserToken_lsd As RParams = RParams.DMS("LSD.:.?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_UserID As RParams = RParams.DMS("userid.:.(\d+)", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+
+        Friend ReadOnly Regex_Photos_by As RParams = RParams.DMS("photos_by"",""id"":""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_FileName As RParams = RParams.DM("([^/\?]+\..{3,4})(?=(\?|\Z))", 0, EDP.ReturnValue)
+        Friend ReadOnly Regex_ProfileUrlID As RParams = RParams.DMS("profile.php\?id=(\d+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_VideoPageID As RParams = RParams.DMS("pageid.:.(\d+)", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly Regex_StoryBucket As RParams = RParams.DMS("story_bucket[^\>]*?(\d+)", 1, EDP.ReturnValue)
+
+        Friend ReadOnly Regex_VideoIDFromURL As RParams = RParams.DMS("facebook.com/([^/]+/videos/|watch/\D*[\?&]{1}v=)(\d+)", 2, EDP.ReturnValue)
+        Friend ReadOnly Regex_PostHtmlFullPicture As RParams = RParams.DM("^((?!_[ps]{1}\d+x\d+).)*$", 0, EDP.ReturnValue)
+
+        Friend ReadOnly SpecialNode() As NodeParams = {New NodeParams("attachment", True, True, True, True, 30),
+                                                       New NodeParams("media", True, True, True, True, 0),
+                                                       New NodeParams("photo_image", True, True, True, True, 0),
+                                                       New NodeParams("uri", True, True, True, True, 0)}
+        Friend ReadOnly SpecialNode2() As NodeParams = {New NodeParams("result", True, True, True, True, 30),
+                                                        New NodeParams("data", True, True, True, True, 0),
+                                                        New NodeParams("currmedia", True, True, True, True, 0),
+                                                        New NodeParams("image", True, True, True, True, 0),
+                                                        New NodeParams("uri", True, True, True, True, 0)}
+    End Module
+End Namespace

SCrawler/API/Facebook/SiteSettings.vb

@@ -0,0 +1,110 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
+Imports SCrawler.Plugin
+Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.Facebook
+    <Manifest("AndyProgram_Facebook"), SavedPosts, SeparatedTasks(1), SpecialForm(False)>
+    Friend Class SiteSettings : Inherits ThreadsNet.SiteSettings
+#Region "Declarations"
+#Region "Auth"
+        <PropertyOption(AllowNull:=False, ControlText:="Accept", ControlToolTip:="Header 'Accept'", IsAuth:=True), ControlNumber(21), PXML, PClonable>
+        Friend ReadOnly Property Header_Accept As PropertyValue
+        <PropertyOption(ControlText:="x-ig-app-id", AllowNull:=True, IsAuth:=True)>
+        Friend Overrides ReadOnly Property HH_IG_APP_ID As PropertyValue
+            Get
+                Return __HH_IG_APP_ID
+            End Get
+        End Property
+        <DoNotUse> Friend Overrides ReadOnly Property HH_CSRF_TOKEN As PropertyValue
+            Get
+                Return __HH_CSRF_TOKEN
+            End Get
+        End Property
+        <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, LeftOffset:=120), ControlNumber(51), PXML, PClonable>
+        Friend ReadOnly Property HH_PLATFORM_VER As PropertyValue
+#End Region
+#Region "Defaults"
+        <PropertyOption(ControlText:="Download photos", IsAuth:=False), PXML, PClonable>
+        Friend ReadOnly Property ParsePhotoBlock As PropertyValue
+        <PropertyOption(ControlText:="Download videos", IsAuth:=False), PXML, PClonable>
+        Friend ReadOnly Property ParseVideoBlock As PropertyValue
+        <PropertyOption(ControlText:="Download stories", IsAuth:=False), PXML, PClonable>
+        Friend ReadOnly Property ParseStoriesBlock As PropertyValue
+#End Region
+#End Region
+#Region "Initializer"
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("Facebook", "facebook.com", AccName, Temp, My.Resources.SiteResources.FacebookIcon_32, My.Resources.SiteResources.FacebookPic_37)
+
+            With Responser.Headers
+                .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Authority, "www.facebook.com"))
+                .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Origin, "https://www.facebook.com"))
+                .Remove(DeclaredNames.Header_FB_FRIENDLY_NAME)
+            End With
+            Header_Accept = New PropertyValue(String.Empty, GetType(String))
+            HH_PLATFORM_VER = New PropertyValue(String.Empty, GetType(String))
+            ParsePhotoBlock = New PropertyValue(True)
+            ParseVideoBlock = New PropertyValue(True)
+            ParseStoriesBlock = New PropertyValue(True)
+
+            UrlPatternUser = "https://www.facebook.com/{0}"
+            UserRegex = RParams.DMS("facebook.com/(profile.php\?id=\d+|[^\?&/]+)", 1)
+            ImageVideoContains = "facebook.com"
+            UserOptionsType = GetType(UserExchangeOptions)
+        End Sub
+#End Region
+#Region "GetInstance"
+        Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
+            Return New UserData
+        End Function
+#End Region
+#Region "UpdateResponserData"
+        Friend Overrides Sub UpdateResponserData(ByVal Resp As Responser)
+            With Responser.Cookies
+                .Update(Resp.Cookies)
+                If .Changed Then Responser.SaveCookies() : .Changed = False
+            End With
+        End Sub
+#End Region
+#Region "BaseAuthExists, GetUserUrl, GetUserPostUrl, IsMyUser, IsMyImageVideo"
+        Friend Overrides Function BaseAuthExists() As Boolean
+            Return Responser.CookiesExists And ACheck(HH_IG_APP_ID.Value)
+        End Function
+        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
+            Return DirectCast(User, UserData).GetProfileUrl
+        End Function
+        Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
+            Return Media.URL_BASE
+        End Function
+        Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
+            Dim e As ExchangeOptions = MyBase.IsMyUser(UserURL)
+            If e.Exists Then
+                e.Options = e.UserName
+                Dim v$ = RegexReplace(e.UserName, Regex_ProfileUrlID)
+                If Not v.IsEmptyString Then
+                    e.UserName = v
+                Else
+                    e.UserName = e.UserName.StringRemoveWinForbiddenSymbols
+                End If
+            End If
+            Return e
+        End Function
+        Friend Overrides Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions
+            If Not URL.IsEmptyString AndAlso Not CStr(AConvert(Of String)(URL, Regex_VideoIDFromURL, String.Empty)).IsEmptyString Then
+                Return New ExchangeOptions(Site, String.Empty) With {.Exists = True}
+            Else
+                Return Nothing
+            End If
+        End Function
+#End Region
+    End Class
+End Namespace

SCrawler/API/Facebook/UserData.vb

@@ -0,0 +1,713 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Threading
+Imports System.Text.RegularExpressions
+Imports SCrawler.API.Base
+Imports SCrawler.API.YouTube.Objects
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports IG = SCrawler.API.Instagram.SiteSettings
+Imports UTypes = SCrawler.API.Base.UserMedia.Types
+Imports UStates = SCrawler.API.Base.UserMedia.States
+Namespace API.Facebook
+    Friend Class UserData : Inherits Instagram.UserData
+#Region "XML names"
+        Private Const Name_IsNoNameProfile As String = "IsNoNameProfile"
+        Private Const Name_OptionsParsed As String = "OptionsParsed"
+        Private Const Name_VideoPageID As String = "VideoPageID"
+        Private Const Name_StoryBucket As String = "StoryBucket"
+        Private Const Name_ParsePhotoBlock As String = "ParsePhotoBlock"
+        Private Const Name_ParseVideoBlock As String = "ParseVideoBlock"
+        Private Const Name_ParseStoriesBlock As String = "ParseStoriesBlock"
+#End Region
+#Region "Declarations"
+        Friend ReadOnly Property MySettings As SiteSettings
+            Get
+                Return HOST.Source
+            End Get
+        End Property
+        Private IsNoNameProfile As Boolean = False
+        Private OptionsParsed As Boolean = False
+        Private Property VideoPageID As String = String.Empty
+        Private Property StoryBucket As String = String.Empty
+        Friend Property ParsePhotoBlock As Boolean = True
+        Friend Property ParseVideoBlock As Boolean = True
+        Friend Property ParseStoriesBlock As Boolean = True
+        Private Enum PageBlock As Integer
+            Timeline = Sections.Timeline
+            Stories = Sections.Stories
+            Photos = 100
+            Videos = 101
+            Undefined = -1
+        End Enum
+#End Region
+#Region "GetProfileUrl"
+        Friend Function GetProfileUrl() As String
+            If IsNoNameProfile Then
+                Return $"https://www.facebook.com/profile.php?id={ID}"
+            Else
+                Return $"https://www.facebook.com/{NameTrue}"
+            End If
+        End Function
+#End Region
+#Region "Exchange"
+        Friend Overrides Function ExchangeOptionsGet() As Object
+            Return New UserExchangeOptions(Me)
+        End Function
+        Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
+                With DirectCast(Obj, UserExchangeOptions)
+                    ParsePhotoBlock = .ParsePhotoBlock
+                    ParseVideoBlock = .ParseVideoBlock
+                    ParseStoriesBlock = .ParseStoriesBlock
+                End With
+            End If
+        End Sub
+#End Region
+#Region "Loader"
+        Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+            Dim updateNames As Action = Sub()
+                                            If Not OptionsParsed AndAlso Not Options.IsEmptyString Then
+                                                OptionsParsed = True
+                                                Dim v$ = RegexReplace(Options, Regex_ProfileUrlID)
+                                                If Not v.IsEmptyString Then ID = v : IsNoNameProfile = True
+                                            End If
+                                        End Sub
+            With Container
+                If Loading Then
+                    If .Contains(Name_IsNoNameProfile) Then
+                        IsNoNameProfile = .Value(Name_IsNoNameProfile).FromXML(Of Boolean)(False)
+                    Else
+                        updateNames.Invoke
+                    End If
+                    OptionsParsed = .Value(Name_OptionsParsed).FromXML(Of Boolean)(False)
+                    VideoPageID = .Value(Name_VideoPageID)
+                    StoryBucket = .Value(Name_StoryBucket)
+                    ParsePhotoBlock = .Value(Name_ParsePhotoBlock).FromXML(Of Boolean)(True)
+                    ParseVideoBlock = .Value(Name_ParseVideoBlock).FromXML(Of Boolean)(True)
+                    ParseStoriesBlock = .Value(Name_ParseStoriesBlock).FromXML(Of Boolean)(True)
+                Else
+                    updateNames.Invoke
+                    .Add(Name_IsNoNameProfile, IsNoNameProfile.BoolToInteger)
+                    .Add(Name_OptionsParsed, OptionsParsed.BoolToInteger)
+                    .Add(Name_VideoPageID, VideoPageID)
+                    .Add(Name_StoryBucket, StoryBucket)
+                    .Add(Name_ParsePhotoBlock, ParsePhotoBlock.BoolToInteger)
+                    .Add(Name_ParseVideoBlock, ParseVideoBlock.BoolToInteger)
+                    .Add(Name_ParseStoriesBlock, ParseStoriesBlock.BoolToInteger)
+                End If
+            End With
+        End Sub
+#End Region
+#Region "Download functions"
+        Private Token_dtsg As String = String.Empty
+        Private Token_lsd As String = String.Empty
+        Private Token_Photosby As String = String.Empty
+        Private Limit As Integer = -1
+        Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
+            Try
+                GetUserTokens(Token)
+                LoadSavePostsKV(True)
+                Limit = If(DownloadTopCount, -1)
+                If IsSavedPosts Then
+                    DownloadData_SavedPosts(String.Empty, Token)
+                Else
+                    If DownloadImages And ParsePhotoBlock Then DownloadData_Photo(String.Empty, Token)
+                    If DownloadVideos And ParseVideoBlock Then DownloadData_Video(String.Empty, Token)
+                    If (DownloadImages Or DownloadVideos) And ParseStoriesBlock Then DownloadData_Stories(Token)
+                End If
+                LoadSavePostsKV(False)
+            Finally
+                MySettings.UpdateResponserData(Responser)
+            End Try
+        End Sub
+        Private Const Header_fb_fr_name_Photo As String = "ProfileCometAppCollectionPhotosRendererPaginationQuery"
+        Private Const Header_fb_fr_name_Video As String = "PagesCometChannelTabAllVideosCardImplPaginationQuery"
+        Private Const Header_fb_fr_name_Stories As String = "StoriesSuspenseContentPaneRootWithEntryPointQuery"
+        Private Const Header_fb_fr_name_SavedPosts As String = "CometSaveDashboardAllItemsPaginationQuery"
+        Private Const DocID_Photo As String = "6684543058255697"
+        Private Const DocID_Video As String = "24545934291687581"
+        Private Const DocID_Stories As String = "6771064226315961"
+        Private Const DocID_SavedPosts As String = "7112228098805003"
+        Private Const Graphql_UrlPattern As String = "https://www.facebook.com/api/graphql?lsd={0}&doc_id={1}&server_timestamps=true&fb_dtsg={3}&fb_api_req_friendly_name={2}&variables={4}"
+        Private Const VideoHtmlUrlPattern As String = "https://www.facebook.com/watch/?v={0}"
+        Private Sub DownloadData_Photo(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Const VarPattern$ = """count"":8,""cursor"":""{0}"",""scale"":1,""id"":""{1}"""
+            Try
+                Dim nextCursor$ = String.Empty
+                Dim newPostsDetected As Boolean = False
+                Dim pUrl$, pUrlBase$
+                Dim pid As PostKV
+
+                ValidateBaseTokens()
+                If Token_Photosby.IsEmptyString Then Throw New ArgumentNullException("Token_Photosby", "Unable to obtain token")
+
+                URL = String.Format(Graphql_UrlPattern, Token_lsd, DocID_Photo, Header_fb_fr_name_Photo,
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly(Token_dtsg),
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(VarPattern, Cursor, Token_Photosby) & "}"))
+
+                ResponserApplyDefs(Header_fb_fr_name_Photo)
+                ThrowAny(Token)
+
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    Using j As EContainer = JsonDocument.Parse(r)
+                        If j.ListExists Then
+                            With j({"data", "node", "pageItems", "edges"})
+                                If .ListExists Then
+                                    ProgressPre.ChangeMax(.Count)
+                                    For Each jNode As EContainer In .Self
+                                        ProgressPre.Perform()
+                                        With jNode
+                                            If Not .Value("cursor").IsEmptyString Then nextCursor = .Value("cursor")
+                                            With .Item({"node"})
+                                                If .ListExists Then
+                                                    pUrl = .Value({"node", "viewer_image"}, "uri")
+                                                    pUrlBase = .Value("url")
+                                                    If Not pUrl.IsEmptyString Then
+                                                        pid = New PostKV(.Value("id"), .Value({"node"}, "id"), PageBlock.Photos)
+                                                        If Not PostKvExists(pid) Then
+                                                            newPostsDetected = True
+                                                            PostsKVIDs.ListAddValue(pid, LNC)
+                                                            _TempPostsList.Add(pid.ID)
+                                                            _TempMediaList.ListAddValue(New UserMedia(pUrl, UTypes.Picture) With {
+                                                                                        .URL_BASE = pUrlBase,
+                                                                                        .File = CreateFileFromUrl(pUrl),
+                                                                                        .Post = pid.ID.IfNullOrEmpty(pid.Code)}, LNC)
+                                                            If Limit > 0 And _TempMediaList.Count >= Limit Then Exit Sub
+                                                        Else
+                                                            Exit Sub
+                                                        End If
+                                                    End If
+                                                End If
+                                            End With
+                                        End With
+                                    Next
+                                End If
+                            End With
+                        End If
+                    End Using
+                End If
+
+                If newPostsDetected And Not nextCursor.IsEmptyString Then DownloadData_Photo(nextCursor, Token)
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (photo) downloading error [{URL}]",, Responser)
+            End Try
+        End Sub
+        Private Sub DownloadData_Video(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Const VarPattern$ = """alwaysIncludeAudioRooms"":true,""count"":6,""cursor"":{0},""pageID"":""{1}"",""scale"":4,""showReactions"":true,""useDefaultActor"":false,""id"":""{1}"""
+            Try
+                Dim nextCursor$ = String.Empty
+                Dim newPostsDetected As Boolean = False
+                Dim pid As PostKV
+
+                If VideoPageID.IsEmptyString Then GetVideoPageID(Token)
+                If VideoPageID.IsEmptyString Then Throw New ArgumentNullException("VideoPageID", "Unable to obtain VideoPageID")
+                ValidateBaseTokens()
+
+                URL = String.Format(Graphql_UrlPattern, Token_lsd, DocID_Video, Header_fb_fr_name_Video,
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly(Token_dtsg),
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(VarPattern, If(Cursor.IsEmptyString, "null", $"""{Cursor}"""), VideoPageID) & "}"))
+
+                ResponserApplyDefs(Header_fb_fr_name_Video)
+                ThrowAny(Token)
+
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    Using j As EContainer = JsonDocument.Parse(r)
+                        If j.ListExists Then
+                            With j({"data", "node", "all_videos", "edges"})
+                                If .ListExists Then
+                                    ProgressPre.ChangeMax(.Count)
+                                    For Each jNode As EContainer In .Self
+                                        ProgressPre.Perform()
+                                        pid = New PostKV(String.Empty, jNode.Value({"node"}, "id"), PageBlock.Videos)
+                                        pid.Code = $"Stories:{pid.ID}"
+                                        nextCursor = jNode.Value("cursor")
+                                        If Not PostKvExists(pid) Then
+                                            newPostsDetected = True
+                                            PostsKVIDs.ListAddValue(pid, LNC)
+                                            _TempPostsList.Add(pid.Code)
+                                            _TempMediaList.ListAddValue(New UserMedia(String.Format(VideoHtmlUrlPattern, pid.ID),
+                                                                                      UTypes.VideoPre) With {.Post = pid.ID}, LNC)
+                                            If Limit > 0 And _TempMediaList.Count >= Limit Then Exit Sub
+                                        Else
+                                            Exit Sub
+                                        End If
+                                    Next
+                                End If
+                            End With
+                        End If
+                    End Using
+                End If
+
+                If newPostsDetected And Not nextCursor.IsEmptyString Then DownloadData_Video(nextCursor, Token)
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (video) downloading error [{URL}]",, Responser)
+            End Try
+        End Sub
+        Private Sub DownloadData_Stories(ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Const VarPattern$ = """UFI2CommentsProvider_commentsKey"":""StoriesSuspenseContentPaneRootWithEntryPointQuery"",""blur"":10,""bucketID"":""{0}"",""displayCommentsContextEnableComment"":true,""displayCommentsContextIsAdPreview"":false,""displayCommentsContextIsAggregatedShare"":false,""displayCommentsContextIsStorySet"":false,""displayCommentsFeedbackContext"":null,""feedbackSource"":65,""feedLocation"":""COMET_MEDIA_VIEWER"",""focusCommentID"":null,""initialBucketID"":""{0}"",""initialLoad"":true,""isInitialLoadFromCommentsNotification"":false,""isStoriesArchive"":false,""isStoryCommentingEnabled"":false,""scale"":1,""shouldDeferLoad"":false,""shouldEnableArmadilloStoryReply"":false,""shouldEnableLiveInStories"":true,""__relay_internal__pv__StoriesIsCommentEnabledrelayprovider"":false,""__relay_internal__pv__StoriesIsContextualReplyDisabledrelayprovider"":false,""__relay_internal__pv__StoriesIsShareToStoryEnabledrelayprovider"":false,""__relay_internal__pv__StoriesRingrelayprovider"":false,""__relay_internal__pv__StoriesLWRVariantrelayprovider"":""www_new_reactions"""
+            Try
+                Dim pUrl$, pUrlBase$
+                Dim pid As PostKV
+                Dim t As UTypes
+                Dim postDate As Date?
+
+                ValidateBaseTokens()
+                If StoryBucket.IsEmptyString Then Throw New ArgumentNullException("StoryBucket", "Unable to obtain StoryBucket")
+
+                URL = String.Format(Graphql_UrlPattern, Token_lsd, DocID_Stories, Header_fb_fr_name_Stories,
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly(Token_dtsg),
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(VarPattern, StoryBucket) & "}"))
+
+                ResponserApplyDefs(Header_fb_fr_name_Stories)
+                ThrowAny(Token)
+
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then r = RegexReplace(r, RParams.DM("[^\r\n]+", 0, EDP.ReturnValue))
+                If Not r.IsEmptyString Then
+                    Using j As EContainer = JsonDocument.Parse(r)
+                        If j.ListExists Then
+                            With j({"data", "bucket", "unified_stories", "edges"})
+                                If .ListExists Then
+                                    ProgressPre.ChangeMax(.Count)
+                                    For Each jNode As EContainer In .Self
+                                        ProgressPre.Perform()
+                                        With jNode({"node"})
+                                            If .ListExists Then
+                                                pid = New PostKV(.Value("id"), "", Sections.Stories)
+                                                With .ItemF({"attachments", 0, "media"})
+                                                    If .ListExists Then
+                                                        pid.ID = .Value("id")
+                                                        pUrl = String.Empty
+                                                        postDate = AConvert(Of Date)(.Value("creation_time"), UnixDate32Provider, Nothing)
+                                                        Select Case .Value("__typename")
+                                                            Case "Photo"
+                                                                t = UTypes.Picture
+                                                                pUrl = .Value({"image"}, "uri")
+                                                            Case "Video"
+                                                                t = UTypes.Video
+                                                                pUrl = .Value("browser_native_hd_url").IfNullOrEmpty(.Value("browser_native_sd_url"))
+                                                        End Select
+                                                        If Not pUrl.IsEmptyString AndAlso Not PostKvExists(pid) Then
+                                                            pUrlBase = $"https://www.facebook.com/stories/{StoryBucket}"
+                                                            PostsKVIDs.Add(pid)
+                                                            _TempMediaList.ListAddValue(New UserMedia(pUrl, t) With {
+                                                                                        .URL_BASE = pUrlBase,
+                                                                                        .File = CreateFileFromUrl(pUrl),
+                                                                                        .SpecialFolder = $"{StoriesFolder} (user)",
+                                                                                        .Post = New UserPost(pid.ID, postDate)}, LNC)
+                                                        End If
+                                                    End If
+                                                End With
+                                            End If
+                                        End With
+                                    Next
+                                End If
+                            End With
+                        End If
+                    End Using
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (stories) downloading error [{URL}]",, Responser)
+            End Try
+        End Sub
+        Private Sub DownloadData_SavedPosts(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Const VarPattern$ = """content_filter"":[],""count"":10,""cursor"":{0},""scale"":1,""use_case"":""SAVE_DEFAULT"""
+            Try
+                Dim nextCursor$ = String.Empty
+                Dim newPostsDetected As Boolean = False
+                Dim pUrl$, videoId$, imgUri$
+                Dim imgFile As SFile
+                Dim pid As PostKV
+
+                ValidateBaseTokens()
+                URL = String.Format(Graphql_UrlPattern, Token_lsd, DocID_SavedPosts, Header_fb_fr_name_SavedPosts,
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly(Token_dtsg),
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(VarPattern, If(Cursor.IsEmptyString, "null", $"""{Cursor}""")) & "}"))
+
+                ResponserApplyDefs(Header_fb_fr_name_SavedPosts)
+                ThrowAny(Token)
+
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    Using j As EContainer = JsonDocument.Parse(r)
+                        If j.ListExists Then
+                            With j({"data", "viewer", "saver_info", "all_saves", "edges"})
+                                If .ListExists Then
+                                    ProgressPre.ChangeMax(.Count)
+                                    For Each jNode As EContainer In .Self
+                                        ProgressPre.Perform()
+                                        nextCursor = jNode.Value("cursor")
+                                        pid = New PostKV("", jNode.Value({"node"}, "id"), Sections.SavedPosts)
+                                        If Not PostKvExists(pid) Then
+                                            PostsKVIDs.Add(pid)
+                                            newPostsDetected = True
+                                            With jNode({"node", "savable"})
+                                                If .ListExists Then
+                                                    pUrl = .Value("savable_permalink")
+                                                    If Not pUrl.IsEmptyString Then
+                                                        Select Case .Value("savable_default_category").StringToLower
+                                                            Case "post_with_photo"
+                                                                imgUri = .Value({"savable_image"}, "uri")
+                                                                If Not imgUri.IsEmptyString Then
+                                                                    imgFile = CreateFileFromUrl(imgUri)
+                                                                    If Not imgFile.Name.IsEmptyString Then
+                                                                        ThrowAny(Token)
+                                                                        _TempMediaList.ListAddList(DownloadData_SavedPosts_ParseImagePost(pUrl, imgFile.Name, Token))
+                                                                    End If
+                                                                End If
+                                                            Case "video"
+                                                                videoId = RegexReplace(pUrl, Regex_VideoIDFromURL)
+                                                                If Not videoId.IsEmptyString Then _
+                                                                   _TempMediaList.ListAddValue(New UserMedia(pUrl, UTypes.VideoPre) With {.Post = videoId}, LNC)
+                                                            Case Else : Continue For
+                                                        End Select
+                                                    End If
+                                                End If
+                                            End With
+                                        End If
+                                    Next
+                                End If
+                            End With
+                        End If
+                    End Using
+                End If
+
+                If newPostsDetected And Not nextCursor.IsEmptyString Then DownloadData_SavedPosts(nextCursor, Token)
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (saved posts) downloading error [{URL}]",, Responser)
+            End Try
+        End Sub
+        Private Function DownloadData_SavedPosts_ParseImagePost(ByVal PostUrl As String, ByVal ImageName As String, ByVal Token As CancellationToken,
+                                                                Optional ByVal Round As Integer = 0) As IEnumerable(Of UserMedia)
+            Dim resp As Responser = HtmlResponserCreate()
+            Try
+                If Round > 0 Then ThrowAny(Token)
+                Dim script$, newUrl$
+                Dim jNode As EContainer, jNode2 As EContainer
+                Dim r$ = resp.GetResponse(PostUrl)
+
+                If Not r.IsEmptyString Then
+                    script = RegexReplace(r, RParams.DMS($"<script type=""application/json""[^\>]*data-sjs>([^<]+?{ImageName}[^<]+)<", 1, EDP.ReturnValue))
+                    If Not script.IsEmptyString Then
+                        Using j As EContainer = JsonDocument.Parse(script)
+                            If j.ListExists Then
+                                jNode = j.Find(Function(jj) Not jj.Name.IsEmptyString AndAlso jj.Name.ToLower = "prefetch_uris_v2", True)
+                                If jNode.ListExists Then
+                                    For Each vNode As EContainer In jNode
+                                        newUrl = RegexReplace(vNode.Value("uri"), Regex_PostHtmlFullPicture)
+                                        If Not newUrl.IsEmptyString Then _
+                                           Return {New UserMedia(newUrl, UTypes.Picture) With {.URL_BASE = PostUrl, .File = CreateFileFromUrl(newUrl)}}
+                                    Next
+                                End If
+                                If Round = 0 Then
+                                    j.SetSourceReferences()
+                                    jNode = j.GetNode(SpecialNode)
+                                    If Not jNode Is Nothing AndAlso Not jNode.Value.IsEmptyString AndAlso Not jNode.Source Is Nothing Then
+                                        With DirectCast(jNode.Source, EContainer)
+                                            If Not .Source Is Nothing Then
+                                                newUrl = DirectCast(.Source, EContainer).Value("url")
+                                                If Not newUrl.IsEmptyString Then
+                                                    Dim __data As IEnumerable(Of UserMedia) =
+                                                        DownloadData_SavedPosts_ParseImagePost(newUrl, CreateFileFromUrl(jNode.Value).Name, Token, Round + 1)
+                                                    If __data.ListExists Then Return __data
+                                                End If
+                                            End If
+                                        End With
+                                    End If
+                                End If
+
+                                jNode = j.Find(Function(jj) Not jj.Name.IsEmptyString AndAlso jj.Name = "viewer_image", True)
+                                If Not jNode Is Nothing AndAlso Not jNode.Source Is Nothing Then
+                                    Dim doRound% = 0
+                                    Do : doRound += 1 : jNode = jNode.Source : Loop While doRound <= 30 AndAlso Not jNode Is Nothing AndAlso Not jNode.Name = "nodes"
+                                    If Not jNode Is Nothing AndAlso jNode.Name = "nodes" AndAlso jNode.Count > 0 Then
+                                        Dim mList As New List(Of UserMedia)
+                                        For Each jNode2 In jNode
+                                            With jNode2
+                                                newUrl = .Value({"media", "viewer_image"}, "uri")
+                                                If Not newUrl.IsEmptyString Then _
+                                                   mList.Add(New UserMedia(newUrl, UTypes.Picture) With {.URL_BASE = PostUrl, .File = CreateFileFromUrl(newUrl)})
+                                            End With
+                                        Next
+                                        Return mList
+                                    End If
+                                End If
+
+                                newUrl = j.GetNode(SpecialNode2).XmlIfNothingValue
+                                If Not newUrl.IsEmptyString Then _
+                                   Return {New UserMedia(newUrl, UTypes.Picture) With {.URL_BASE = PostUrl, .File = CreateFileFromUrl(newUrl)}}
+                            End If
+                        End Using
+                    End If
+                End If
+                Return Nothing
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (saved posts) downloading error [{PostUrl}]",, resp, False)
+                Return Nothing
+            Finally
+                HtmlResponserDispose(resp)
+            End Try
+        End Function
+#End Region
+#Region "ValidateBaseTokens, GetVideoPageID, GetUserTokens"
+        ''' <exception cref="ArgumentNullException"></exception>
+        Private Sub ValidateBaseTokens()
+            If Token_dtsg.IsEmptyString Then Throw New ArgumentNullException("Token_dtsg", "Unable to obtain token")
+            If Token_lsd.IsEmptyString Then Throw New ArgumentNullException("Token_lsd", "Unable to obtain token")
+        End Sub
+        Private Sub GetVideoPageID(ByVal Token As CancellationToken)
+            Dim URL$ = $"{GetProfileUrl()}\videos"
+            Dim resp As Responser = HtmlResponserCreate()
+            Try
+                Dim r$ = resp.GetResponse(URL)
+                If Not r.IsEmptyString Then VideoPageID = RegexReplace(r, Regex_VideoPageID)
+            Catch ex As Exception
+                ProcessException(ex, Token, "get video page ID",, resp)
+            Finally
+                HtmlResponserDispose(resp)
+            End Try
+        End Sub
+        Private Sub GetUserTokens(ByVal Token As CancellationToken)
+            Dim URL$ = If(IsSavedPosts, "https://www.facebook.com/saved", GetProfileUrl())
+            Dim resp As Responser = HtmlResponserCreate()
+            Try
+                Token_dtsg = String.Empty
+                Token_lsd = String.Empty
+                Token_Photosby = String.Empty
+                Dim r$ = resp.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    If Responser.CookiesExists Then Responser.Cookies.Update(resp.Cookies)
+                    Token_dtsg = RegexReplace(r, Regex_UserToken_dtsg)
+                    Token_lsd = RegexReplace(r, Regex_UserToken_lsd)
+                    Token_Photosby = RegexReplace(r, Regex_Photos_by)
+                    If StoryBucket.IsEmptyString Then StoryBucket = RegexReplace(r, Regex_StoryBucket)
+                    If ID.IsEmptyString Then
+                        ID = RegexReplace(r, Regex_UserID)
+                        If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
+                    End If
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, "get user token",, resp)
+            Finally
+                HtmlResponserDispose(resp)
+            End Try
+        End Sub
+#End Region
+#Region "Responser options"
+        Private Sub ResponserApplyDefs(ByVal __fb_friendly_name As String)
+            With Responser
+                .Headers.Add(ThreadsNet.UserData.Header_FB_LSD, Token_lsd)
+                .Headers.Add(DeclaredNames.Header_FB_FRIENDLY_NAME, __fb_friendly_name)
+                .Method = "POST"
+                .Accept = "*/*"
+                .Referer = GetProfileUrl()
+            End With
+        End Sub
+        Private Function HtmlResponserCreate() As Responser
+            Dim r As Responser = Responser.Copy
+            With r
+                .Accept = CStr(AConvert(Of String)(MySettings.Header_Accept.Value, String.Empty))
+                .Referer = Nothing
+                .Method = "GET"
+                With .Headers
+                    .Clear()
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Authority, "www.facebook.com"))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchDest, "document"))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode, "navigate"))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchSite, "none"))
+                    .Add("Sec-Fetch-User", "?1")
+                    .Add("Upgrade-Insecure-Requests", 1)
+                    Dim h$ = Responser.Headers.Value(IG.Header_Browser)
+                    If Not h.IsEmptyString Then .Add(IG.Header_Browser, h)
+                    h = Responser.Headers.Value(IG.Header_BrowserExt)
+                    If Not h.IsEmptyString Then .Add(IG.Header_BrowserExt, h)
+                    h = .Value(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform))
+                    If Not h.IsEmptyString Then .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform, h))
+                    If ACheck(MySettings.HH_PLATFORM_VER.Value) Then _
+                       .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatformVersion, MySettings.HH_PLATFORM_VER.Value))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaMobile, "?0"))
+                    .Add("Sec-Ch-Ua-Model", "")
+                End With
+            End With
+            Return r
+        End Function
+        Private Sub HtmlResponserDispose(ByVal r As Responser)
+            If Not r Is Nothing Then
+                Responser.Cookies.Update(r.Cookies)
+                r.Dispose()
+            End If
+        End Sub
+#End Region
+#Region "ReparseMissing"
+        Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
+            Dim rList As New List(Of Integer)
+            Dim resp As Responser = HtmlResponserCreate()
+            Try
+                If ContentMissingExists Then
+                    Dim m As UserMedia
+                    Dim result As Boolean
+                    ProgressPre.ChangeMax(_ContentList.Count)
+                    For i% = 0 To _ContentList.Count - 1
+                        ProgressPre.Perform()
+                        m = _ContentList(i)
+                        If (m.State = UStates.Missing And (m.Type = UTypes.Video Or m.Type = UTypes.VideoPre)) AndAlso Not m.URL_BASE.IsEmptyString Then
+                            ThrowAny(Token)
+                            result = False
+                            m = ReparseSingleVideo(m, resp, result)
+                            If result Then
+                                rList.Add(i)
+                                m.State = UStates.Missing
+                                _TempMediaList.ListAddValue(m, LNC)
+                            End If
+                        End If
+                    Next
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, "missing data downloading error")
+            Finally
+                If rList.Count > 0 Then
+                    For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
+                    rList.Clear()
+                End If
+                HtmlResponserDispose(resp)
+            End Try
+        End Sub
+#End Region
+#Region "ReparseVideo"
+        Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Dim resp As Responser = HtmlResponserCreate()
+            Try
+                If _TempMediaList.Count > 0 AndAlso _TempMediaList.Exists(Function(mm) mm.Type = UTypes.VideoPre) Then
+                    ProgressPre.ChangeMax(_TempMediaList.Count)
+                    Dim m As UserMedia
+                    Dim result As Boolean
+                    For i% = 0 To _TempMediaList.Count - 1
+                        m = _TempMediaList(i)
+                        If m.Type = UTypes.VideoPre Then
+                            ThrowAny(Token)
+                            result = False
+                            m = ReparseSingleVideo(m, resp, result)
+                            If Not result Then m.State = UStates.Missing
+                            _TempMediaList(i) = m
+                        End If
+                        ProgressPre.Perform()
+                    Next
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, $"video reparsing error [{URL}]",, resp)
+            Finally
+                HtmlResponserDispose(resp)
+            End Try
+        End Sub
+        Protected Function ReparseSingleVideo(ByVal m As UserMedia, ByVal resp As Responser, ByRef result As Boolean) As UserMedia
+            Const nameSD$ = "browser_native_sd_url"
+            Const nameHD$ = "browser_native_hd_url"
+            Const pattern$ = "<script type=""application/json""[^\>]*data-sjs>([^<]+?{0}[^<]+)<"
+            Dim URL$ = String.Empty
+            Dim j As EContainer = Nothing
+            Try
+                Dim r$, script$, __url$
+                Dim jNode As EContainer
+                Dim jf As Predicate(Of EContainer) = Function(ee) Not ee.Name.IsEmptyString AndAlso (ee.Name.ToLower = nameSD Or ee.Name.ToLower = nameHD)
+                Dim re As RParams = RParams.DMS("", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+                If m.Post.ID.IsEmptyString Then
+                    URL = m.URL_BASE
+                Else
+                    URL = String.Format(VideoHtmlUrlPattern, m.Post.ID)
+                End If
+                r = resp.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    re.Pattern = String.Format(pattern, nameHD)
+                    script = RegexReplace(r, re)
+                    If script.IsEmptyString Then
+                        re.Pattern = String.Format(pattern, nameSD)
+                        script = RegexReplace(r, re)
+                    End If
+                    If Not script.IsEmptyString Then
+                        j = JsonDocument.Parse(script)
+                        If j.ListExists Then
+                            j.SetSourceReferences()
+                            jNode = j.Find(jf, True)
+                            If Not jNode Is Nothing Then
+                                With DirectCast(jNode.Source, EContainer)
+                                    __url = .Value(nameHD).IfNullOrEmpty(.Value(nameSD))
+                                    If Not __url.IsEmptyString Then
+                                        m.URL = __url
+                                        m.URL_BASE = URL
+                                        m.Type = UTypes.Video
+                                        m.File = CreateFileFromUrl(__url)
+                                        m.Post.Date = AConvert(Of Date)(.Value("publish_time"), UnixDate32Provider, Nothing)
+                                        result = True
+                                        Return m
+                                    End If
+                                End With
+                            End If
+                        End If
+                    End If
+                End If
+            Catch ex As Exception
+            End Try
+            j.DisposeIfReady
+            result = False
+            Return m
+        End Function
+#End Region
+#Region "CreateFileFromUrl"
+        Protected Overrides Function CreateFileFromUrl(ByVal URL As String) As SFile
+            If Not URL.IsEmptyString Then
+                Dim f$ = RegexReplace(URL, Regex_FileName)
+                If Not f.IsEmptyString Then
+                    Return f
+                Else
+                    Dim ff As New SFile(URL)
+                    If Not ff.Extension.IsEmptyString Then
+                        If ff.Length > 4 Then ff.Extension = ff.Extension.Split("?").FirstOrDefault
+                        ff.Extension = ff.Extension.StringRemoveWinForbiddenSymbols
+                    End If
+                    ff.Name = ff.Name.StringRemoveWinForbiddenSymbols
+                    Return ff
+                End If
+            End If
+            Return String.Empty
+        End Function
+#End Region
+#Region "DownloadContent"
+        Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
+            DownloadContentDefault(Token)
+        End Sub
+#End Region
+#Region "DownloadSingleObject"
+        Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
+            _ContentList.Add(New UserMedia(Data.URL, UTypes.VideoPre) With {.Post = CStr(AConvert(Of String)(Data.URL, Regex_VideoIDFromURL, String.Empty))})
+            ReparseMissing(Token)
+        End Sub
+#End Region
+#Region "ThrowAny"
+        Friend Overrides Sub ThrowAny(ByVal Token As CancellationToken)
+            ThrowAnyImpl(Token)
+        End Sub
+#End Region
+#Region "Exception"
+        Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
+                                                          Optional ByVal EObj As Object = Nothing) As Integer
+            Return 0
+        End Function
+#End Region
+    End Class
+End Namespace

SCrawler/API/Facebook/UserExchangeOptions.vb

@@ -0,0 +1,32 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.Plugin.Attributes
+Namespace API.Facebook
+    Friend Class UserExchangeOptions
+        <PSetting(NameOf(SiteSettings.ParsePhotoBlock), NameOf(MySettings))>
+        Friend Property ParsePhotoBlock As Boolean = True
+        <PSetting(NameOf(SiteSettings.ParseVideoBlock), NameOf(MySettings))>
+        Friend Property ParseVideoBlock As Boolean = True
+        <PSetting(NameOf(SiteSettings.ParseStoriesBlock), NameOf(MySettings))>
+        Friend Property ParseStoriesBlock As Boolean = True
+        Private ReadOnly Property MySettings As SiteSettings
+        Friend Sub New(ByVal u As UserData)
+            MySettings = u.HostCollection.Default.Source
+            ParsePhotoBlock = u.ParsePhotoBlock
+            ParseVideoBlock = u.ParseVideoBlock
+            ParseStoriesBlock = u.ParseStoriesBlock
+        End Sub
+        Friend Sub New(ByVal s As SiteSettings)
+            MySettings = s
+            ParsePhotoBlock = s.ParsePhotoBlock.Value
+            ParseVideoBlock = s.ParseVideoBlock.Value
+            ParseStoriesBlock = s.ParseStoriesBlock.Value
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Gfycat/Envir.vb

@@ -39,8 +39,8 @@ Namespace API.Gfycat
             If Not urlVideo.IsEmptyString Then
                 If urlVideo.Contains("redgifs.com") Then
                     _IsRedGifs = True
-                    DirectCast(Settings(RedGifs.RedGifsSiteKey).Source, RedGifs.SiteSettings).UpdateTokenIfRequired()
-                    Dim newData As IYouTubeMediaContainer = Settings(RedGifs.RedGifsSiteKey).GetSingleMediaInstance(urlVideo, Data.File)
+                    DirectCast(Settings(RedGifs.RedGifsSiteKey).Default.Source, RedGifs.SiteSettings).UpdateTokenIfRequired()
+                    Dim newData As IYouTubeMediaContainer = Settings(RedGifs.RedGifsSiteKey).Default.GetSingleMediaInstance(urlVideo, Data.File)
                     If Not newData Is Nothing Then
                         newData.Progress = Data.Progress
                         newData.Download(Data.UseCookies, Token)
@@ -49,7 +49,7 @@ Namespace API.Gfycat
                         With DirectCast(Data, YouTubeMediaContainerBase)
                             .Site = RedGifs.RedGifsSite
                             .SiteKey = RedGifs.RedGifsSiteKey
-                            .SiteIcon = Settings(RedGifs.RedGifsSiteKey).Source.Image
+                            .SiteIcon = Settings(RedGifs.RedGifsSiteKey).Default.Source.Image
                         End With
                     Else
                         Throw New Exception($"Unable to get RedGifs instance{vbCr}{Data.URL}{vbCr}{urlVideo}")

SCrawler/API/Instagram/SiteSettings.vb

@@ -10,9 +10,6 @@ Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Imports PersonalUtilities.Forms
-Imports PersonalUtilities.Functions.XML
-Imports PersonalUtilities.Functions.XML.Objects
-Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Cookies
@@ -21,18 +18,6 @@ Namespace API.Instagram
     <Manifest(InstagramSiteKey), SeparatedTasks(1), SavedPosts, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
-#Region "Images"
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.InstagramIcon_32
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.InstagramPic_76
-            End Get
-        End Property
-#End Region
 #Region "Providers"
         Private Class TimersChecker : Inherits FieldsCheckerProviderBase
             Private ReadOnly LVProvider As New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral}
@@ -78,23 +63,23 @@ Namespace API.Instagram
         Friend Const Header_Browser As String = "Sec-Ch-Ua"
         Friend Const Header_BrowserExt As String = "Sec-Ch-Ua-Full-Version-List"
         Friend Const Header_Platform As String = "Sec-Ch-Ua-Platform-Version"
-        <PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash for tagged posts", IsAuth:=True), PXML("InstaHash"), ControlNumber(0)>
+        <PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash for tagged posts", IsAuth:=True), PXML("InstaHash"), ControlNumber(0), PClonable(Clone:=False)>
         Friend ReadOnly Property HashTagged As PropertyValue
-        <PropertyOption(ControlText:="x-csrftoken", IsAuth:=True, AllowNull:=False), ControlNumber(2)>
+        <PropertyOption(ControlText:="x-csrftoken", IsAuth:=True, AllowNull:=False), ControlNumber(2), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_CSRF_TOKEN As PropertyValue
-        <PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3)>
+        <PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3), PClonable(Clone:=False)>
         Friend Property HH_IG_APP_ID As PropertyValue
-        <PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=True), ControlNumber(4)>
+        <PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=True), ControlNumber(4), PClonable(Clone:=False)>
         Friend Property HH_ASBD_ID As PropertyValue
-        <PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=True), ControlNumber(5)>
+        <PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=True), ControlNumber(5), PClonable(Clone:=False)>
         Friend Property HH_IG_WWW_CLAIM As PropertyValue
-        <PropertyOption(ControlText:="sec-ch-ua", IsAuth:=True, AllowNull:=True), ControlNumber(6)>
+        <PropertyOption(ControlText:="sec-ch-ua", IsAuth:=True, AllowNull:=True), ControlNumber(6), PClonable>
         Private Property HH_BROWSER As PropertyValue
-        <PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:="sec-ch-ua-full-version-list", IsAuth:=True, AllowNull:=True), ControlNumber(7)>
+        <PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:="sec-ch-ua-full-version-list", IsAuth:=True, AllowNull:=True), ControlNumber(7), PClonable>
         Private Property HH_BROWSER_EXT As PropertyValue
-        <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, AllowNull:=True), ControlNumber(8)>
+        <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, AllowNull:=True), ControlNumber(8), PClonable>
         Private Property HH_PLATFORM As PropertyValue
-        <PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=True), ControlNumber(9)>
+        <PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=True), ControlNumber(9), PClonable>
         Private Property HH_USER_AGENT As PropertyValue
         Friend Overrides Function BaseAuthExists() As Boolean
             Return Responser.CookiesExists And ACheck(HH_IG_APP_ID.Value) And ACheck(HH_CSRF_TOKEN.Value)
@@ -124,68 +109,69 @@ Namespace API.Instagram
         End Sub
 #End Region
 #Region "Download properties"
-        <PropertyOption(ControlText:="Request timer", AllowNull:=False), PXML("RequestsWaitTimer"), ControlNumber(20)>
+        <PropertyOption(ControlText:="Request timer", AllowNull:=False), PXML("RequestsWaitTimer"), ControlNumber(20), PClonable>
         Friend ReadOnly Property RequestsWaitTimer As PropertyValue
         <Provider(NameOf(RequestsWaitTimer), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerProvider As IFormatProvider
-        <PropertyOption(ControlText:="Request timer counter", AllowNull:=False, LeftOffset:=120), PXML("RequestsWaitTimerTaskCount"), ControlNumber(21)>
+        <PropertyOption(ControlText:="Request timer counter", AllowNull:=False, LeftOffset:=120), PXML("RequestsWaitTimerTaskCount"), ControlNumber(21), PClonable>
         Friend ReadOnly Property RequestsWaitTimerTaskCount As PropertyValue
         <Provider(NameOf(RequestsWaitTimerTaskCount), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerTaskCountProvider As IFormatProvider
-        <PropertyOption(ControlText:="Posts limit timer", AllowNull:=False), PXML("SleepTimerOnPostsLimit"), ControlNumber(22)>
+        <PropertyOption(ControlText:="Posts limit timer", AllowNull:=False), PXML("SleepTimerOnPostsLimit"), ControlNumber(22), PClonable>
         Friend ReadOnly Property SleepTimerOnPostsLimit As PropertyValue
         <Provider(NameOf(SleepTimerOnPostsLimit), FieldsChecker:=True)>
         Private ReadOnly Property SleepTimerOnPostsLimitProvider As IFormatProvider
-        <PropertyOption(ControlText:="Get timeline", ControlToolTip:="Default value for new users"), PXML, ControlNumber(23)>
+        <PropertyOption(ControlText:="Get timeline", ControlToolTip:="Default value for new users"), PXML, ControlNumber(23), PClonable>
         Friend ReadOnly Property GetTimeline As PropertyValue
-        <PropertyOption(ControlText:="Get stories", ControlToolTip:="Default value for new users"), PXML, ControlNumber(24)>
+        <PropertyOption(ControlText:="Get stories", ControlToolTip:="Default value for new users"), PXML, ControlNumber(24), PClonable>
         Friend ReadOnly Property GetStories As PropertyValue
-        <PropertyOption(ControlText:="Get stories: user", ControlToolTip:="Default value for new users"), PXML, ControlNumber(25)>
+        <PropertyOption(ControlText:="Get stories: user", ControlToolTip:="Default value for new users"), PXML, ControlNumber(25), PClonable>
         Friend ReadOnly Property GetStoriesUser As PropertyValue
-        <PropertyOption(ControlText:="Get tagged photos", ControlToolTip:="Default value for new users"), PXML, ControlNumber(26)>
+        <PropertyOption(ControlText:="Get tagged photos", ControlToolTip:="Default value for new users"), PXML, ControlNumber(26), PClonable>
         Friend ReadOnly Property GetTagged As PropertyValue
         <PropertyOption(ControlText:="Tagged notify limit",
                         ControlToolTip:="If the number of tagged posts exceeds this number you will be notified." & vbCr &
-                        "-1 to disable"), PXML, ControlNumber(27)>
+                        "-1 to disable"), PXML, ControlNumber(27), PClonable>
         Friend ReadOnly Property TaggedNotifyLimit As PropertyValue
         <Provider(NameOf(TaggedNotifyLimit), FieldsChecker:=True)>
         Private ReadOnly Property TaggedNotifyLimitProvider As IFormatProvider
 #End Region
 #Region "Download ready"
-        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download timeline"), PXML, ControlNumber(10)>
+        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download timeline"), PXML, ControlNumber(10), PClonable>
         Friend ReadOnly Property DownloadTimeline As PropertyValue
-        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download stories"), PXML, ControlNumber(11)>
+        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download stories"), PXML, ControlNumber(11), PClonable>
         Friend ReadOnly Property DownloadStories As PropertyValue
-        <PropertyOption(ControlText:="Download stories: user", ControlToolTip:="Download stories (user)"), PXML, ControlNumber(12)>
+        <PropertyOption(ControlText:="Download stories: user", ControlToolTip:="Download stories (user)"), PXML, ControlNumber(12), PClonable>
         Friend ReadOnly Property DownloadStoriesUser As PropertyValue
-        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged posts"), PXML, ControlNumber(13)>
+        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged posts"), PXML, ControlNumber(13), PClonable>
         Friend ReadOnly Property DownloadTagged As PropertyValue
 #End Region
 #Region "429 bypass"
-        Private ReadOnly Property DownloadingErrorDate As XMLValue(Of Date)
+        <PXML("InstagramDownloadingErrorDate")>
+        Private ReadOnly Property DownloadingErrorDate As PropertyValue
         Friend Property LastApplyingValue As Integer? = Nothing
         Friend ReadOnly Property ReadyForDownload As Boolean
             Get
                 If SkipUntilNextSession Then Return False
                 With DownloadingErrorDate
-                    If .ValueF.Exists Then
-                        Return .ValueF.Value.AddMinutes(If(LastApplyingValue, 10)) < Now
+                    If ACheck(Of Date)(.Value) Then
+                        Return CDate(.Value).AddMinutes(If(LastApplyingValue, 10)) < Now
                     Else
                         Return True
                     End If
                 End With
             End Get
         End Property
-        Private ReadOnly Property LastDownloadDate As XMLValue(Of Date)
-        Private ReadOnly Property LastRequestsCount As XMLValue(Of Integer)
+        <PXML> Private ReadOnly Property LastDownloadDate As PropertyValue
+        <PXML> Private ReadOnly Property LastRequestsCount As PropertyValue
         <PropertyOption(IsInformationLabel:=True), ControlNumber(100)>
         Private Property LastRequestsCountLabel As PropertyValue
         Private ReadOnly LastRequestsCountLabelStr As Func(Of Integer, String) = Function(r) $"Number of spent requests: {r.NumToGroupIntegral}"
         Private TooManyRequestsReadyForCatch As Boolean = True
         Friend Function GetWaitDate() As Date
             With DownloadingErrorDate
-                If .ValueF.Exists Then
-                    Return .ValueF.Value.AddMinutes(If(LastApplyingValue, 10))
+                If ACheck(Of Date)(.Value) Then
+                    Return CDate(.Value).AddMinutes(If(LastApplyingValue, 10))
                 Else
                     Return Now
                 End If
@@ -194,7 +180,7 @@ Namespace API.Instagram
         Friend Sub TooManyRequests(ByVal Catched As Boolean)
             With DownloadingErrorDate
                 If Catched Then
-                    If Not .ValueF.Exists Then
+                    If Not ACheck(Of Date)(.Value) Then
                         .Value = Now
                         If TooManyRequestsReadyForCatch Then
                             LastApplyingValue = If(LastApplyingValue, 0) + 10
@@ -203,7 +189,7 @@ Namespace API.Instagram
                         End If
                     End If
                 Else
-                    .ValueF = Nothing
+                    .Value = Nothing
                     LastApplyingValue = Nothing
                     TooManyRequestsReadyForCatch = True
                 End If
@@ -212,8 +198,8 @@ Namespace API.Instagram
 #End Region
 #End Region
 #Region "Initializer"
-        Friend Sub New(ByRef _XML As XmlFile, ByVal GlobalPath As SFile)
-            MyBase.New(InstagramSite, "instagram.com")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New(InstagramSite, "instagram.com", AccName, Temp, My.Resources.SiteResources.InstagramIcon_32, My.Resources.SiteResources.InstagramPic_76)
 
             Dim app_id$ = String.Empty
             Dim www_claim$ = String.Empty
@@ -226,7 +212,7 @@ Namespace API.Instagram
 
             With Responser
                 .Accept = "*/*"
-                useragent = .UserAgent
+                If .UserAgentExists Then useragent = .UserAgent Else .UserAgent = String.Empty
                 With .Headers
                     If .Count > 0 Then
                         token = .Value(Header_CSRF_TOKEN)
@@ -250,8 +236,6 @@ Namespace API.Instagram
                 .CookiesExtractedAutoSave = False
             End With
 
-            Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString}
-
             HashTagged = New PropertyValue(String.Empty, GetType(String))
             HH_CSRF_TOKEN = New PropertyValue(token, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_CSRF_TOKEN), v))
             HH_IG_APP_ID = New PropertyValue(app_id, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_IG_APP_ID), v))
@@ -281,12 +265,11 @@ Namespace API.Instagram
             TaggedNotifyLimit = New PropertyValue(200)
             TaggedNotifyLimitProvider = New TaggedNotifyLimitChecker
 
-            DownloadingErrorDate = New XMLValue(Of Date) With {.Provider = New XMLValueConversionProvider(Function(ss, nn, vv, dd) AConvert(Of String)(vv, AModes.Var, Nothing))}
-            DownloadingErrorDate.SetExtended("InstagramDownloadingErrorDate", Now.AddYears(-10), _XML, n)
-            LastDownloadDate = New XMLValue(Of Date)("LastDownloadDate", Now.AddDays(-1), _XML, n)
-            LastRequestsCount = New XMLValue(Of Integer)("LastRequestsCount", 0, _XML, n)
+            DownloadingErrorDate = New PropertyValue(Nothing, GetType(Date))
+            LastDownloadDate = New PropertyValue(Now.AddDays(-1))
+            LastRequestsCount = New PropertyValue(0)
             LastRequestsCountLabel = New PropertyValue(LastRequestsCountLabelStr.Invoke(LastRequestsCount.Value))
-            AddHandler LastRequestsCount.ValueChanged, Sub(sender, e) LastRequestsCountLabel.Value = LastRequestsCountLabelStr.Invoke(DirectCast(sender, XMLValue(Of Integer)).ValueF.Value)
+            LastRequestsCount.OnChangeFunction = Sub(vv) LastRequestsCountLabel.Value = LastRequestsCountLabelStr.Invoke(vv)
 
             _AllowUserAgentUpdate = False
             UrlPatternUser = "https://www.instagram.com/{0}/"
@@ -329,7 +312,7 @@ Namespace API.Instagram
         Private _NextTagged As Boolean = True
         Friend Overrides Sub DownloadStarted(ByVal What As Download)
             ActiveJobs += 1
-            If LastDownloadDate.Value.AddMinutes(120) < Now Or Not ACheck(HH_IG_WWW_CLAIM.Value) Then HH_IG_WWW_CLAIM.Value = "0"
+            If CDate(LastDownloadDate.Value).AddMinutes(120) < Now Or Not ACheck(HH_IG_WWW_CLAIM.Value) Then HH_IG_WWW_CLAIM.Value = "0"
         End Sub
         Friend Overrides Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download)
             With DirectCast(User, UserData)
@@ -337,8 +320,8 @@ Namespace API.Instagram
                     .WaitNotificationMode = _NextWNM
                     .TaggedCheckSession = _NextTagged
                 End If
-                If LastDownloadDate.Value.AddMinutes(60) > Now Then
-                    .RequestsCount = LastRequestsCount
+                If CDate(LastDownloadDate.Value).AddMinutes(60) > Now Then
+                    .RequestsCount = LastRequestsCount.Value
                 Else
                     LastRequestsCount.Value = 0
                     .RequestsCount = 0

SCrawler/API/Instagram/UserData.vb

@@ -71,7 +71,7 @@ Namespace API.Instagram
                 Return DirectCast(HOST.Source, SiteSettings)
             End Get
         End Property
-        Private ReadOnly PostsKVIDs As List(Of PostKV)
+        Protected ReadOnly PostsKVIDs As List(Of PostKV)
         Private ReadOnly PostsToReparse As List(Of PostKV)
         Private LastCursor As String = String.Empty
         Private FirstLoadingDone As Boolean = False
@@ -175,7 +175,7 @@ Namespace API.Instagram
                 End If
             End If
         End Sub
-        Private Overloads Function PostKvExists(ByVal pkv As PostKV) As Boolean
+        Protected Overloads Function PostKvExists(ByVal pkv As PostKV) As Boolean
             Return PostKvExists(pkv.ID, False, pkv.Section) OrElse PostKvExists(pkv.Code, True, pkv.Section)
         End Function
         Private Overloads Function PostKvExists(ByVal PostCodeId As String, ByVal IsCode As Boolean, ByVal Section As Sections) As Boolean
@@ -297,7 +297,7 @@ Namespace API.Instagram
             Declarations.UpdateResponser(e, Responser)
         End Sub
         Protected Enum Sections : Timeline : Tagged : Stories : UserStories : SavedPosts : End Enum
-        Private Const StoriesFolder As String = "Stories"
+        Protected Const StoriesFolder As String = "Stories"
         Private Const TaggedFolder As String = "Tagged"
 #Region "429 bypass"
         Private Const MaxPostsCount As Integer = 200
@@ -973,7 +973,8 @@ Namespace API.Instagram
             Try
                 Dim r$ = Responser.GetResponse($"https://i.instagram.com/api/v1/highlights/{ID}/highlights_tray/",, EDP.ThrowException)
                 If Not r.IsEmptyString Then
-                    Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing()("tray").XmlIfNothing
+                    Dim ee As New ErrorsDescriber(EDP.ReturnValue) With {.DeclaredMessage = New MMessage($"{ToStringForLog()}:")}
+                    Using j As EContainer = JsonDocument.Parse(r, ee).XmlIfNothing()("tray").XmlIfNothing
                         If j.Count > 0 Then Return j.Select(Function(jj) jj.Value("id").Replace("highlight:", String.Empty)).ListIfNothing
                     End Using
                 End If
@@ -1010,7 +1011,7 @@ Namespace API.Instagram
                                                           Optional ByVal s As Object = Nothing) As Integer
             If Responser.StatusCode = HttpStatusCode.NotFound Then '404
                 If Not UserNameRequested AndAlso GetUserNameById() Then Return 1 Else UserExists = False
-            ElseIf Responser.StatusCode = HttpStatusCode.BadRequest Then '400
+            ElseIf Responser.StatusCode = HttpStatusCode.BadRequest Or Responser.StatusCode = HttpStatusCode.Unauthorized Then '400, 401
                 HasError = True
                 MyMainLOG = $"Instagram credentials have expired [{CInt(Responser.StatusCode)}]: {ToStringForLog()} [{s}]"
                 DisableSection(s)

SCrawler/API/JustForFans/M3U8.vb

@@ -126,70 +126,6 @@ Namespace API.JustForFans
                                         $"API.JustForFans.M3U8.GetFiles({IIf(IsAudio, "audio", "video")}):{vbCr}URL: {URL}{vbCr}File: {File}")
             End Try
         End Sub
-        'TODELETE: JFF.M3U8.GetFiles_OLD 20231008
-        'Private Sub GetFiles_OLD(ByVal URL As String, ByRef File As SFile, ByVal IsAudio As Boolean)
-        '    Try
-        '        Dim r$ = Responser.GetResponse(URL)
-        '        If Not r.IsEmptyString Then
-        '            Dim data As List(Of RegexMatchStruct) = RegexFields(Of RegexMatchStruct)(r, {REGEX_PLS_FILES}, {1, 2}, EDP.ReturnValue)
-        '            If data.ListExists Then
-        '                Dim appender$ = URL.Replace(URL.Split("/").LastOrDefault, String.Empty)
-        '                With (From d As RegexMatchStruct In data
-        '                      Where Not d.Arr(0).IfNullOrEmpty(d.Arr(1)).IsEmptyString
-        '                      Select M3U8Base.CreateUrl(appender, d.Arr(0).IfNullOrEmpty(d.Arr(1)).Trim)).ToList
-        '                    If .ListExists Then
-        '                        File = $"{Cache.RootDirectory.PathWithSeparator}{IIf(IsAudio, "AUDIO.aac", "VIDEO.mp4")}"
-        '                        Dim tmpCache As CacheKeeper = Cache.NewInstance
-        '                        Dim tmpFile As SFile = .Item(0)
-        '                        If tmpFile.Extension.IsEmptyString Then tmpFile.Extension = "ts"
-        '                        tmpFile.Path = tmpCache.RootDirectory.Path
-        '                        tmpFile.Separator = "\"
-
-        '                        Dim cFile As SFile = tmpFile
-        '                        cFile.Name = "all"
-
-        '                        tmpCache.Validate()
-
-        '                        Using bat As New TextSaver
-        '                            Using b As New BatchExecutor(True) With {.Encoding = Settings.CMDEncoding}
-        '                                AddHandler b.OutputDataReceived, AddressOf Batch_OutputDataReceived
-        '                                bat.AppendLine($"chcp {BatchExecutor.UnicodeEncoding}")
-        '                                bat.AppendLine(BatchExecutor.GetDirectoryCommand(tmpCache))
-        '                                ProgressChangeMax(.Count * 2 + 1)
-        '                                Using w As New WebClient
-        '                                    For i = 0 To .Count - 1
-        '                                        tmpFile.Name = $"ConPart_{i}"
-        '                                        Thrower.ThrowAny()
-        '                                        'Responser.DownloadFile(.Item(i), tmpFile)
-        '                                        w.DownloadFile(.Item(i), tmpFile)
-        '                                        ProgressPerform()
-        '                                        tmpCache.AddFile(tmpFile, True)
-        '                                        bat.AppendLine($"type {tmpFile.File} >> {cFile.File}")
-        '                                    Next
-        '                                End Using
-
-        '                                bat.AppendLine($"""{Settings.FfmpegFile}"" -i {cFile.File} -c copy ""{File}""")
-
-        '                                Dim batFile As SFile = bat.SaveAs($"{tmpCache.RootDirectory.PathWithSeparator}command.bat")
-
-        '                                b.Execute($"""{batFile}""")
-
-        '                                If Not File.Exists Then File = Nothing
-        '                            End Using
-        '                        End Using
-        '                    End If
-        '                End With
-        '            End If
-        '        End If
-        '    Catch oex As OperationCanceledException
-        '        Throw oex
-        '    Catch dex As ObjectDisposedException
-        '        Throw dex
-        '    Catch ex As Exception
-        '        ErrorsDescriber.Execute(EDP.SendToLog + EDP.ThrowException, ex,
-        '                                $"API.JustForFans.M3U8.GetFiles({IIf(IsAudio, "audio", "video")}):{vbCr}URL: {URL}{vbCr}File: {File}")
-        '    End Try
-        'End Sub
         Private Async Sub Batch_OutputDataReceived(ByVal Sender As Object, ByVal e As DataReceivedEventArgs)
             Await Task.Run(Sub() If Not e.Data.IsEmptyString AndAlso e.Data.Contains("] Opening") Then ProgressPerform())
         End Sub

SCrawler/API/JustForFans/SiteSettings.vb

@@ -15,32 +15,22 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.JustForFans
     <Manifest("AndyProgram_JustForFans"), SavedPosts, SeparatedTasks(1)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.JFFIcon_64
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.JFFPic_76
-            End Get
-        End Property
         Friend Const UserHash4_CookieName As String = "userhash4"
-        <PropertyOption(ControlText:="User ID", AllowNull:=False), PXML>
+        <PropertyOption(ControlText:="User ID", AllowNull:=False), PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property UserID As PropertyValue
-        <PropertyOption, PXML>
+        <PropertyOption, PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property UserHash4 As PropertyValue
-        <PropertyOption(ControlText:="Accept", ControlToolTip:="Header 'Accept'")>
+        <PropertyOption(ControlText:="Accept", ControlToolTip:="Header 'Accept'"), PClonable>
         Friend ReadOnly Property HeaderAccept As PropertyValue
-        <PropertyOption> Friend ReadOnly Property UserAgent As PropertyValue
+        <PropertyOption, PClonable> Friend ReadOnly Property UserAgent As PropertyValue
         Private Sub UpdateHeader(ByVal HeaderName As String, ByVal HeaderValue As String)
             Select Case HeaderName
                 Case NameOf(HeaderAccept) : If HeaderValue.IsEmptyString Then Responser.Accept = Nothing Else Responser.Accept = HeaderValue
                 Case NameOf(UserAgent) : If Not HeaderValue.IsEmptyString Then Responser.UserAgent = HeaderValue
             End Select
         End Sub
-        Friend Sub New()
-            MyBase.New("JustForFans", "justfor.fans")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("JustForFans", "justfor.fans", AccName, Temp, My.Resources.SiteResources.JFFIcon_64, My.Resources.SiteResources.JFFPic_76)
 
             With Responser
                 .CookiesExtractMode = Responser.CookiesExtractModes.Any
@@ -53,7 +43,7 @@ Namespace API.JustForFans
             UserID = New PropertyValue(String.Empty, GetType(String))
             UserHash4 = New PropertyValue(String.Empty, GetType(String))
             HeaderAccept = New PropertyValue(Responser.Accept.Value, GetType(String), Sub(v) UpdateHeader(NameOf(HeaderAccept), v))
-            UserAgent = New PropertyValue(Responser.UserAgent, GetType(String), Sub(v) UpdateHeader(NameOf(UserAgent), v))
+            UserAgent = New PropertyValue(If(Responser.UserAgentExists, Responser.UserAgent, String.Empty), GetType(String), Sub(v) UpdateHeader(NameOf(UserAgent), v))
 
             _AllowUserAgentUpdate = False
             UserRegex = RParams.DMS("https://justfor.fans/([^/\?]+)", 1, EDP.ReturnValue)

SCrawler/API/JustForFans/UserData.vb

@@ -168,8 +168,6 @@ Namespace API.JustForFans
 #Region "Initializer"
         Friend Sub New()
             UseInternalM3U8Function = True
-            'TODELETE: UseResponserClient 20231008
-            'UseResponserClient = True
         End Sub
 #End Region
 #Region "Download functions"
@@ -297,7 +295,7 @@ Namespace API.JustForFans
                     InitializeFileSerial()
                     Dim r$
                     Dim m As UserMedia
-                    Dim stateRefill As Func(Of UserMedia, Integer, UserMedia) = Function(ByVal input As UserMedia, ii As Integer) As UserMedia
+                    Dim stateRefill As Func(Of UserMedia, Integer, UserMedia) = Function(ByVal input As UserMedia, ByVal ii As Integer) As UserMedia
                                                                                     input.State = UserMedia.States.Missing
                                                                                     input.Attempts = m.Attempts
                                                                                     Return input

SCrawler/API/LPSG/SiteSettings.vb

@@ -12,18 +12,8 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.LPSG
     <Manifest("AndyProgram_LPSG")>
     Friend Class SiteSettings : Inherits Base.SiteSettingsBase
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.LPSGIcon_48
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.LPSGPic_32
-            End Get
-        End Property
-        Friend Sub New()
-            MyBase.New("LPSG", "www.lpsg.com")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("LPSG", "www.lpsg.com", AccName, Temp, My.Resources.SiteResources.LPSGIcon_48, My.Resources.SiteResources.LPSGPic_32)
             UrlPatternUser = "https://www.lpsg.com/threads/{0}/"
             UserRegex = RParams.DMS(".+?lpsg.com/threads/[^/]+?\.(\d+)", 1, EDP.ReturnValue)
         End Sub

SCrawler/API/Mastodon/MastodonDomains.vb

@@ -1,65 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Imports SCrawler.Plugin
-Imports SCrawler.API.Base
-Imports PersonalUtilities.Functions.XML
-Namespace API.Mastodon
-    Friend Class MastodonDomains : Inherits DomainsContainer
-        Friend ReadOnly Property Credentials As List(Of Credentials)
-        Friend ReadOnly Property CredentialsTemp As List(Of Credentials)
-        Private ReadOnly CredentialsFile As SFile = $"{SettingsFolderName}\Responser_Mastodon_DomainsCredentials.xml"
-        Friend Sub New(ByVal _Instance As ISiteSettings, ByVal DefaultValue As String)
-            MyBase.New(_Instance, DefaultValue)
-            Credentials = New List(Of Credentials)
-            CredentialsTemp = New List(Of Credentials)
-            If CredentialsFile.Exists Then
-                Using x As New XmlFile(CredentialsFile,, False) With {.AllowSameNames = True, .XmlReadOnly = True}
-                    x.LoadData()
-                    If x.Count > 0 Then Credentials.ListAddList(x, LAP.IgnoreICopier)
-                End Using
-            End If
-        End Sub
-        Friend Overrides Function Apply() As Boolean
-            If Changed Then
-                Credentials.Clear()
-                If CredentialsTemp.Count > 0 Then Credentials.AddRange(CredentialsTemp)
-                CredentialsTemp.Clear()
-            End If
-            Return MyBase.Apply()
-        End Function
-        Friend Overrides Sub Save()
-            If Credentials.Count > 0 Then
-                Using x As New XmlFile With {.AllowSameNames = True}
-                    x.AddRange(Credentials)
-                    x.Name = "DomainsCredentials"
-                    x.Save(CredentialsFile)
-                End Using
-            Else
-                CredentialsFile.Delete(,, EDP.None)
-            End If
-            MyBase.Save()
-        End Sub
-        Friend Overrides Sub Reset()
-            CredentialsTemp.Clear()
-            MyBase.Reset()
-        End Sub
-        Friend Overrides Sub OpenSettingsForm()
-            Using f As New SettingsForm(Instance)
-                f.ShowDialog()
-                If f.DialogResult = DialogResult.OK Then
-                    Changed = True
-                    CredentialsTemp.Clear()
-                    If f.MyCredentials.Count > 0 Then CredentialsTemp.AddRange(f.MyCredentials)
-                    DomainsTemp.Clear()
-                    If f.MyDomains.Count > 0 Then DomainsTemp.ListAddList(f.MyDomains, LAP.NotContainsOnly)
-                End If
-            End Using
-        End Sub
-    End Class
-End Namespace

SCrawler/API/Mastodon/SettingsForm.Designer.vb

@@ -1,165 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Namespace API.Mastodon
-    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
-    Partial Friend Class SettingsForm : Inherits System.Windows.Forms.Form
-        <System.Diagnostics.DebuggerNonUserCode()>
-        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
-            Try
-                If disposing AndAlso components IsNot Nothing Then
-                    components.Dispose()
-                End If
-            Finally
-                MyBase.Dispose(disposing)
-            End Try
-        End Sub
-        Private components As System.ComponentModel.IContainer
-        <System.Diagnostics.DebuggerStepThrough()>
-        Private Sub InitializeComponent()
-            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
-            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
-            Dim ActionButton1 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(SettingsForm))
-            Dim ActionButton2 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton3 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton4 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton5 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton6 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Me.CMB_DOMAINS = New PersonalUtilities.Forms.Controls.ComboBoxExtended()
-            Me.TXT_AUTH = New PersonalUtilities.Forms.Controls.TextBoxExtended()
-            Me.TXT_TOKEN = New PersonalUtilities.Forms.Controls.TextBoxExtended()
-            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
-            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
-            CONTAINER_MAIN.ContentPanel.SuspendLayout()
-            CONTAINER_MAIN.SuspendLayout()
-            TP_MAIN.SuspendLayout()
-            CType(Me.CMB_DOMAINS, System.ComponentModel.ISupportInitialize).BeginInit()
-            CType(Me.TXT_AUTH, System.ComponentModel.ISupportInitialize).BeginInit()
-            CType(Me.TXT_TOKEN, System.ComponentModel.ISupportInitialize).BeginInit()
-            Me.SuspendLayout()
-            '
-            'CONTAINER_MAIN
-            '
-            '
-            'CONTAINER_MAIN.ContentPanel
-            '
-            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
-            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(384, 361)
-            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
-            CONTAINER_MAIN.LeftToolStripPanelVisible = False
-            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
-            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
-            CONTAINER_MAIN.RightToolStripPanelVisible = False
-            CONTAINER_MAIN.Size = New System.Drawing.Size(384, 361)
-            CONTAINER_MAIN.TabIndex = 0
-            CONTAINER_MAIN.TopToolStripPanelVisible = False
-            '
-            'TP_MAIN
-            '
-            TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
-            TP_MAIN.ColumnCount = 1
-            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
-            TP_MAIN.Controls.Add(Me.CMB_DOMAINS, 0, 0)
-            TP_MAIN.Controls.Add(Me.TXT_AUTH, 0, 1)
-            TP_MAIN.Controls.Add(Me.TXT_TOKEN, 0, 2)
-            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
-            TP_MAIN.Location = New System.Drawing.Point(0, 0)
-            TP_MAIN.Name = "TP_MAIN"
-            TP_MAIN.RowCount = 3
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
-            TP_MAIN.Size = New System.Drawing.Size(384, 361)
-            TP_MAIN.TabIndex = 0
-            '
-            'CMB_DOMAINS
-            '
-            ActionButton1.BackgroundImage = CType(resources.GetObject("ActionButton1.BackgroundImage"), System.Drawing.Image)
-            ActionButton1.Name = "Add"
-            ActionButton1.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Add
-            ActionButton2.BackgroundImage = CType(resources.GetObject("ActionButton2.BackgroundImage"), System.Drawing.Image)
-            ActionButton2.Name = "Delete"
-            ActionButton2.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Delete
-            ActionButton3.BackgroundImage = CType(resources.GetObject("ActionButton3.BackgroundImage"), System.Drawing.Image)
-            ActionButton3.Name = "Clear"
-            ActionButton3.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            ActionButton4.BackgroundImage = CType(resources.GetObject("ActionButton4.BackgroundImage"), System.Drawing.Image)
-            ActionButton4.Name = "ArrowDown"
-            ActionButton4.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.ArrowDown
-            ActionButton4.Visible = False
-            Me.CMB_DOMAINS.Buttons.Add(ActionButton1)
-            Me.CMB_DOMAINS.Buttons.Add(ActionButton2)
-            Me.CMB_DOMAINS.Buttons.Add(ActionButton3)
-            Me.CMB_DOMAINS.Buttons.Add(ActionButton4)
-            Me.CMB_DOMAINS.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CMB_DOMAINS.ListDropDownStyle = PersonalUtilities.Forms.Controls.ComboBoxExtended.ListMode.Simple
-            Me.CMB_DOMAINS.Location = New System.Drawing.Point(4, 4)
-            Me.CMB_DOMAINS.Name = "CMB_DOMAINS"
-            Me.CMB_DOMAINS.Size = New System.Drawing.Size(378, 296)
-            Me.CMB_DOMAINS.TabIndex = 0
-            '
-            'TXT_AUTH
-            '
-            ActionButton5.BackgroundImage = CType(resources.GetObject("ActionButton5.BackgroundImage"), System.Drawing.Image)
-            ActionButton5.Name = "Clear"
-            ActionButton5.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_AUTH.Buttons.Add(ActionButton5)
-            Me.TXT_AUTH.CaptionText = "Auth"
-            Me.TXT_AUTH.CaptionToolTipEnabled = True
-            Me.TXT_AUTH.CaptionToolTipText = "Bearer token"
-            Me.TXT_AUTH.CaptionWidth = 50.0R
-            Me.TXT_AUTH.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.TXT_AUTH.Location = New System.Drawing.Point(4, 306)
-            Me.TXT_AUTH.Name = "TXT_AUTH"
-            Me.TXT_AUTH.Size = New System.Drawing.Size(376, 22)
-            Me.TXT_AUTH.TabIndex = 1
-            '
-            'TXT_TOKEN
-            '
-            ActionButton6.BackgroundImage = CType(resources.GetObject("ActionButton6.BackgroundImage"), System.Drawing.Image)
-            ActionButton6.Name = "Clear"
-            ActionButton6.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_TOKEN.Buttons.Add(ActionButton6)
-            Me.TXT_TOKEN.CaptionText = "Token"
-            Me.TXT_TOKEN.CaptionToolTipEnabled = True
-            Me.TXT_TOKEN.CaptionToolTipText = "csrf token"
-            Me.TXT_TOKEN.CaptionWidth = 50.0R
-            Me.TXT_TOKEN.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.TXT_TOKEN.Location = New System.Drawing.Point(4, 335)
-            Me.TXT_TOKEN.Name = "TXT_TOKEN"
-            Me.TXT_TOKEN.Size = New System.Drawing.Size(376, 22)
-            Me.TXT_TOKEN.TabIndex = 2
-            '
-            'SettingsForm
-            '
-            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
-            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
-            Me.ClientSize = New System.Drawing.Size(384, 361)
-            Me.Controls.Add(CONTAINER_MAIN)
-            Me.Icon = Global.SCrawler.My.Resources.SiteResources.MastodonIcon_48
-            Me.MinimumSize = New System.Drawing.Size(400, 400)
-            Me.Name = "SettingsForm"
-            Me.ShowInTaskbar = False
-            Me.Text = "Mastodon domains"
-            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
-            CONTAINER_MAIN.ResumeLayout(False)
-            CONTAINER_MAIN.PerformLayout()
-            TP_MAIN.ResumeLayout(False)
-            CType(Me.CMB_DOMAINS, System.ComponentModel.ISupportInitialize).EndInit()
-            CType(Me.TXT_AUTH, System.ComponentModel.ISupportInitialize).EndInit()
-            CType(Me.TXT_TOKEN, System.ComponentModel.ISupportInitialize).EndInit()
-            Me.ResumeLayout(False)
-
-        End Sub
-        Private WithEvents CMB_DOMAINS As PersonalUtilities.Forms.Controls.ComboBoxExtended
-        Private WithEvents TXT_AUTH As PersonalUtilities.Forms.Controls.TextBoxExtended
-        Private WithEvents TXT_TOKEN As PersonalUtilities.Forms.Controls.TextBoxExtended
-    End Class
-End Namespace

SCrawler/API/Mastodon/SettingsForm.resx

@@ -1,292 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<root>
-  <!-- 
-    Microsoft ResX Schema 
-    
-    Version 2.0
-    
-    The primary goals of this format is to allow a simple XML format 
-    that is mostly human readable. The generation and parsing of the 
-    various data types are done through the TypeConverter classes 
-    associated with the data types.
-    
-    Example:
-    
-    ... ado.net/XML headers & schema ...
-    <resheader name="resmimetype">text/microsoft-resx</resheader>
-    <resheader name="version">2.0</resheader>
-    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
-    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
-    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
-    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
-    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
-        <value>[base64 mime encoded serialized .NET Framework object]</value>
-    </data>
-    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
-        <comment>This is a comment</comment>
-    </data>
-                
-    There are any number of "resheader" rows that contain simple 
-    name/value pairs.
-    
-    Each data row contains a name, and value. The row also contains a 
-    type or mimetype. Type corresponds to a .NET class that support 
-    text/value conversion through the TypeConverter architecture. 
-    Classes that don't support this are serialized and stored with the 
-    mimetype set.
-    
-    The mimetype is used for serialized objects, and tells the 
-    ResXResourceReader how to depersist the object. This is currently not 
-    extensible. For a given mimetype the value must be set accordingly:
-    
-    Note - application/x-microsoft.net.object.binary.base64 is the format 
-    that the ResXResourceWriter will generate, however the reader can 
-    read any of the formats listed below.
-    
-    mimetype: application/x-microsoft.net.object.binary.base64
-    value   : The object must be serialized with 
-            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
-            : and then encoded with base64 encoding.
-    
-    mimetype: application/x-microsoft.net.object.soap.base64
-    value   : The object must be serialized with 
-            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
-            : and then encoded with base64 encoding.
-
-    mimetype: application/x-microsoft.net.object.bytearray.base64
-    value   : The object must be serialized into a byte array 
-            : using a System.ComponentModel.TypeConverter
-            : and then encoded with base64 encoding.
-    -->
-  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
-    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
-    <xsd:element name="root" msdata:IsDataSet="true">
-      <xsd:complexType>
-        <xsd:choice maxOccurs="unbounded">
-          <xsd:element name="metadata">
-            <xsd:complexType>
-              <xsd:sequence>
-                <xsd:element name="value" type="xsd:string" minOccurs="0" />
-              </xsd:sequence>
-              <xsd:attribute name="name" use="required" type="xsd:string" />
-              <xsd:attribute name="type" type="xsd:string" />
-              <xsd:attribute name="mimetype" type="xsd:string" />
-              <xsd:attribute ref="xml:space" />
-            </xsd:complexType>
-          </xsd:element>
-          <xsd:element name="assembly">
-            <xsd:complexType>
-              <xsd:attribute name="alias" type="xsd:string" />
-              <xsd:attribute name="name" type="xsd:string" />
-            </xsd:complexType>
-          </xsd:element>
-          <xsd:element name="data">
-            <xsd:complexType>
-              <xsd:sequence>
-                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
-                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
-              </xsd:sequence>
-              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
-              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
-              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
-              <xsd:attribute ref="xml:space" />
-            </xsd:complexType>
-          </xsd:element>
-          <xsd:element name="resheader">
-            <xsd:complexType>
-              <xsd:sequence>
-                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
-              </xsd:sequence>
-              <xsd:attribute name="name" type="xsd:string" use="required" />
-            </xsd:complexType>
-          </xsd:element>
-        </xsd:choice>
-      </xsd:complexType>
-    </xsd:element>
-  </xsd:schema>
-  <resheader name="resmimetype">
-    <value>text/microsoft-resx</value>
-  </resheader>
-  <resheader name="version">
-    <value>2.0</value>
-  </resheader>
-  <resheader name="reader">
-    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
-  </resheader>
-  <resheader name="writer">
-    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
-  </resheader>
-  <metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
-    <value>False</value>
-  </metadata>
-  <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
-    <value>False</value>
-  </metadata>
-  <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
-  <data name="ActionButton1.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
-        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAADmUlE
-        QVRIS62WWWxMURjHL220JW1HausmlFrDFKUhnUGH6bRFzJ2idImlC0Vp2mlji1A8iNhCPIjIRES8EU+W
-        h2oEtbSDTk3HNNM7S01VKsXjkb/vXBo3k1Ee7sMvmZzzzf//ne/+z50RAAxL1MUIG4G/YAv3HSVhF5Vw
-        IYNdz3LadVj9RgdTB+HQYYPHIJuE1ocSdlEJFzG+1bPRLQLinglIeCkg+XUkKvz56hnkOfQs/rmA8S9H
-        YEp7FDI64tAQtKhnsMapZ7zzNHsUFnbGY4VzIk70l6hnIH4wsDR7NBZ3apDrSqL5T8eFgUr1DLZ78lim
-        Q4N8VzK29MxEpZSBa4M16hnU+c3M9CEFpdJsVHsXos63DDcHrf9nQEXD5VymwW/5USLNwl5vJhp7dTgW
-        NML2pR7jbsUMS+KdMTa5Q8NQxinfBU4dRFcOyjy52OtbhwOBDTgZLKPPmTgY0ON4MBdNfSbYBupxY8Aq
-        G10dqMG5/nIc7ytGQ6CQRliAamkTN/g1Ai4e95Qy3iogpX0UtBRDnhRzdxq2SXOxz5eFQ70rScCEU335
-        ssGxj0YS06HSm4GN3ekwdE2C1hGH1LZR0JDOJof5jwHvnIvzTa0jlooTYfktvt+fhcOBHDQFTWRgxJGP
-        ObAGsulZLMLWnjlY756K5c4JmNcRi6T2SGheCIihS2l5ozAo6NRhMolnUAcGV6IcwwqvFrX+JTjYuwKH
-        SfRAYDms/mzs9y1GFe2VSnOw1j0FejqpLN4WCX4ZufiIBwLMLxQGm12rsLQzgWKYgmLPLNTQw6ynpDSS
-        IBet8y+TqaVRVdFIeJrWuCcj+/0EzH43BomvIhBLI45uFiDcJ+6QwROFwa6+Amb9bGFNg6Xs9Ncd7Oy3
-        Knb2eyU7/20nu9y/m136tIvEl6BC0qKoZwby3alo9JVhj7T5R7m/kJVIIityi8zyXmTiW+I10SqyIQNb
-        uIgNwYuuf25kFd75KPKkI49OmUWnrfYWyXv/wBb2cijhhVf6a9lGei65XclYRDd6mj0GWz2iLBJaH0rY
-        RSVc5Eywmhm7kuQXHX+bJlBStrh+zTi0PpSwi0q4yNFAOVvgiEcKJWUsxZn/NhT+znlofShhF5VwkRpv
-        MUtti4KGYjj6sYCIh5QSu4oG27stjItHU+cjeQzvkcFzFQ2KnSKLoc4FukDCXeI2GbSoaFD4ziyPxNxK
-        0AUyNxOP1DOwcaG/8I+/LRB+At7psBnyDBG0AAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton2.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAABGdBTUEAALGPC/xhBQAAABl0RVh0U29m
-        dHdhcmUAQWRvYmUgSW1hZ2VSZWFkeXHJZTwAAAVoSURBVEhLhZVrTJNXGMdfrtNSQIoadKRz2o0CorU3
-        WkDIVBRaaGNbwAteh+AARRQlitEYTTRekiX7sH3YPmyZH9wtziybigLRCWTaCW5sCBWhlrb0Ci9zSxbo
-        2f+UliGX7SS/tO85z/k9T57zXhhCCPO7Wh3VIhB83JKQ0Nu4bNlHm5YseZ1hmHC69n+Y5HLFcz7/ft/S
-        pY+vr1hhwL4oEBJcZ0x793If5uZ+1VNfT/qvXCHP6+p8tzMymqRxcW8hMGKqbDo9MlmWddu2AfbiRTJ6
-        +TIZKC52fyAUVi2JiYkLJmGaBYIPnx4+TPrOnCH9p08TC4LNx46RWwrF/ZXR0W/PleRZZuY669atZvbS
-        JcJiL9vQQEZPnSKmwkLPjcTE97GPB8KZlvh4C5X31dWRgRMniAVBtvPnyWB9ve+2XP7jmtjYpOlJTOnp
-        G60lJRZaOZWPQs4ePUpGUZh3xw7SnJDQhT0KEM3c5fOv9paVkX4kMAPL8ePEig1D584RG9rVpFS2rY6J
-        EQaTmKTSjbbiYsvIhQuERTGjKIrFvtHaWjK8fz9plsudexYu/BLxKsBj9ALBGzel0vt9e/b4XiBoENhQ
-        zRDOxIWWOY4cIS0KRZs4Nja5QyLJtRoM1pGzZ/0tYVExi/ayNTVkBPJ76enuJA7nM4j3gVWAHjgTIYqL
-        E96SStvMu3YR64EDxF5dTYYOHSJOJPNA5Kiu9rUrlZ1mrdbCnjzpr5jFGotYtqpqQi6TuVM4nKvwlYHU
-        gDzU31OMSGl8fPJtsbjVsn27z15RQRzAVVlJ3BB4kcx78CAZQbUjVIxrFtd+OdrbmpHhEXG5VE4rTwHz
-        wMRdFDw4jEgFj5dyRyRqsxYVEcfu3cQFPPv2ES8qHEbCYRzgsFZLvO+8Q7xKJXGDVoXCK46Ovob95YBW
-        Ph/8+xwE/wSTyHi81OZVq9qsGs2Ye8sW4srPJy6JhDgTE4kzOpo4IyKIMyyMOLhcX9Py5R4lj0cPtAKs
-        BBwwKfc7p174J5BEhHY9FIk6bBDaIRuiQkDFfsLDSbdU+pdBKPwe8e+BNDBD7vdNn6BYd+6stK5da7bP
-        nz9TDujcoEAw1lJY+CyFz9dCHDubnDJjwltRccS5fr3TjurnlIMBYE5NJY8Nhq7SrCwREsz6xL9y4S4v
-        b3Bt2uSyR0XNkDvQe9ouKu8HvaGh5FfQIxL5OgyG30qUStqmGUkm/3jKy0+48vLcs1XuiI8nL/Ly/rYl
-        JfmovCcgN4JW+l8iGe8oKuoqzcyckSQob3CpVB47l+sXv9KWxYtJt0r1x9ns7HZjQYHNnJxMfoH0EXgA
-        7oFm0CmTjRsNhs6Na9bQF+Tkq57xlJXVu9Rqz9Bs8kWLSG9BwcsqieQONlXnpaaWdul0z7rR+6C8CTSC
-        m8Aol4+36/XGT7VaevCRIIRx6/WWoQULZq2cyveLxY0IrAT0IHm1OTmZT3Q6U2da2qT8B/Ad+BZ05OSM
-        GXW6p4hdBiIZZ1FRt5vPn6vyuwiqCsj9Xyq6qXbDBkWnXm/6OS3NN1X+dUgIeZSdPXZPoxlEXC6IY9pL
-        S7faNBqXC9Iplf95YBb5ZF+RpGbdunQcbO/D1avJ9YC8LT19/Iv8/BeqpKRPEDORAGNeY3HxSYtG43Eq
-        FL5etfpljUzWhPlZ5VOTlGVliR+hHUbs+0mpHP9GpRqM5XAuY20zmGgRRohYKIx9rNd/3qfTOa7l5uLu
-        C63BvARw6fp0eRCMyBslJe8+2bx58EFhoVMlFNJvgQ4kgggQEgykvV0ApEAd+J3z8Z8KxmuA3pr0zikA
-        b4LJZ2FqYBigFdOPNf0NC679Fxi0OPr+XxiAJgwURph/AJfOQQebMR8TAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton3.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton4.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAYAAAD0eNT6AAAABGdBTUEAALGPC/xhBQAAE65JREFUeF7t
-        3X2sJWddB/DdLi2lQG2hdOHuvfM887J7Cxca4ELTQMDWKigIFpBAEAgi9g+CJpJo9Q8NJhgBiYZIYspL
-        GlAKCkhEC4KgQlsLQkqhKi/lrYWWlxaw3dLddrerz/Q89+7dc2fbfTn3npf5fJJv2rS758z85nnOzJz5
-        nZktAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMK3O3r79wVUIz65jfGNVxI/VIX69CvGO9M//a9P+e8o3B/8v
-        vKn9s+3fyX8dAJgmaWd+fl3E96Wd/E9XdvZHkfbvXNa+Rn45AGCS3bvjj/E/h3box5OrmxjPyy8PAEyS
-        XXO7zqhCeH/HDnwUOdCE+J6zdux4eH47YIrEGE8uy/Ls9Bnx/LooL0oH9b9Th/I1TVG+rCqKC+q6Xsh/
-        FJgmO8vy6WknfdPQTnsjckMdwlPy2wITLO3wF6si/lGas1ekuXvX0Fzuyg9S3psOCl6qDwimQB3ji9Ok
-        3btmEm907kpnEa/Mbw9Mlq1pB/6cdHZ/ZcfcPZrcXoXyrVVVFfl1gUmSdsS/libqPUMTd5NSvjktwrbB
-        kgDjVi1UT26K+Nnu+XrMuaud60uPWHpIfhtg3JqyfEaanHcPTdZNTRPCPy4uLj40LxIwBudt2fKAtOP/
-        0zQnN+5koIg3tpca81sC49J+LZcm5a3rJulYEq6LSV40YBOFEB6V5uFV6+flRiTsSwf9r81vDYzBCSO4
-        vjfq/KAuiqfm5QM2QRPjuWnubUbz71DCn6W33zpYCmDT1EX5m92Tcuy5q47xFXkxgQ3UduqnOXfn0Bzc
-        xJSvz4sCbIb2pzlp8v1w/WScnKSzkjekRT1hsMTAKC0vL5/Ydud3zb1NT1FelBcL2GiDm3d0TMTJy0ea
-        pjk1LzYwAu3NvtLc+uTQXBtn7tYYCJtja/vQno5JOJFpQrzWb4hhNJoQnpjm1Q3D82wCcnNRFKfnxQQ2
-        Qttk1zH5JjzhFmcIcHzyzb6O5aFem5J0sP/OvKjARmg7b7sm3xRkT3vDorwawJHb1t6Ep2NOTVoOtDch
-        yssMjFr6IPh8x8SbnsT4lrQamgPhCMzPzz+sifHjnXNpMnN5XnRglJaWlk5KE2z/0ISbxnzQQ0bgvlXz
-        1ePSXPnG0NyZ+DRF8Zi8CsCo7Azh0V0TbkrzRc2B0G3wIJ9429CcmZLce4MgYJTyff87JtzU5uayLM/J
-        qwcM7vD5+jQ3DgzNlWnKDXldgFFJZwW/2jHZpj1727uZ5VWE3mofqJXmw4eG5sdUpqqqXXm1gFGoQnhJ
-        12SbgRxoYvzjtIruK04vxRjPSvPgK0PzYmqTPqtemVcNGIU6xgu7JtusJH1ovH9ubu6UvLrQC2ncPyuN
-        /58Mz4fpTvnmvHrAKJQL5dO6J9ssJXxucWFhLq8yzLKtaUf5h2ncb9zz+8eUKsYP53UERmHX/PyOrsk2
-        g7nJDUWYZUuPWHpIE8oPdIz92UiMn86rCoxIOmOYta8KD5uftk2Peb1hZtTzdVOHcF3HmJ+ZVCF+Ia8u
-        MCppcl0+PNlmOG1zYPtYYc2BzIQ0np+ZxvWPh8b5LObqvMrAqEzRo4BHmctijCfnEsBUqkP5u2ksz8Kd
-        PI8g5SfyagOj0jbIpQk2c01DR5Brmh3NfC4DTI324LWO8V0dY3pm48mAsEGm7OEgo0sRb9wZ4+NzGWDi
-        lWUZ0ti9Zt1YnvUU8fdyCYBRmsFbAh9xqhDvqEN4Xi4FTKz8s93vD4/hPiSdpJyXywCMWPtrgKuGJ12P
-        ck/6gPmDXAuYOHVR/lY6UN3XMXb7kDv17MAGqhaqJ6WJ1sdegDUJ726a5oG5JDB2917vL+Kl3eO1N/lQ
-        LgewUdIO8E0dk69vubosy+25JDA2bYNuFeJnOsZovxLjhbkkwEZZXl4+0QfOvfl2Ogg4O5cFNl1dFE9N
-        4/B7Q+Oyj7mh/VzKZQE2UtM0j6iL+LWOidizhN3OPBiHuigvSmPwrvVjsn9pQnh1LguwGQa3Fo3fHp6M
-        Pcw97c1WcllgQy0tLZ2UDr7/qmMc9jJNiF/WkwNjMHhQ0GzfX/yIU8RLfRCxkdq+kzTfrugcf/3MgZ1l
-        +fRcHmCztU8Yq2P8h47J2cdcpTmQjdCE8IQ0vnzjdkjKP8nlAcZoWxXin3dP0n4l1eGb9UL92FwXOG51
-        Ub48ja09w2Otz2nvTJpKs21QIWDs0lnKb6TJqTEphN3NQvncXBY4VtvSju4N3WOs17l6cXHxoblGwKRo
-        r8mlHeAtHZO2b9mfDohem8sCR2XX3K4z0hj65NCYklSTGONpuUzApNlVFFWaqP81NHF7mvD2tnM7lwbu
-        V/vwqTR2vrV+LPU7VSjf4ff+MAU0B65NeWVd12fm0sBhpTnzosHDp7rGUV8T9lVFvDiXCJgSrmEezDea
-        onhMrgsM25rmyuvSODkwNG56nvZyYvi5XCNg2mgOXM3tVVH9ci4L3KtpmlN9W7Y+VYhfiEkuEzCt8n3L
-        fzA8yXuY/b7OZEVZlovt3ew6xknf8965ublTcpmAaac5cG3C2zQ09Vv7bVAaC/+7fmz0Og6QYVZpDlyT
-        GD/dPlgpl4b+2Nru5NIYuGfdmOhxmhB/VBblL+QaATNKc+DBfH1nCI/OdWHGtTewSdv874fGgIT4xfYb
-        wlwmYNZpDlzNbVUIz85lYUblJ2i6BDacGP/u7O3bH5zLBPSF5sDV7K+L+Nu5LMyYtJP7xbSNfzy0zfue
-        A+03gak8WwdVAnpHc+CaxHiJ5sCZsnK9f/+6bd3v3JZ2/r+SawT0mebAg0kfjB93v/Pp136t3X693bWN
-        e56v6nsBhmkOXE24Ph0EnJXrwpSp63qhDuXnu7dtn1P+U1VVP5PLBHAozYGDtD+LchvU6TN4Iqa+lqGs
-        XO8/YVAlgMPQHLiSsC+dNb0ml4UJVxflRWm73b1+O/Y5YXcVwvNziQDun+bANYnxkvO2bHlALg0TJsZ4
-        cl3ESzu3Xa8Trm+KYimXCeDIaQ48mKqIH9McOHl2zc/vaIr42a5t1vN8tCiK03OZAI6J5sCVFPFr7QNk
-        cl0Ys3yp6nvrtlO/s3K9f9ugSgDHSXPgILk58PxcFsYkX+93J8s1qUK8oynKF+YSAYyO5sCVhH3pgOjV
-        uSxsoqZpHpjq//bu7dLjFPHGND+Xc5kARk9z4JrE+JZUEl+1bpLFhYW5VPf/WLcd5N/ruj4zlwlg42gO
-        PCQfdXOVjdeE8MRU6xuGai9uXw2MgebA1YTrFkMoc10YsaYoX5rqfOf6uvc6e9LO/xW5RACbT3Pgam5N
-        B0Q/m8vCCLT3XnCQ2ZXwnWqhenIuE8D4aA5czV3OykZj19yuM1I9PzlUXwnhirIst+cyAYyf5sA1GTQH
-        uu/6MdoZ4+NTHb+1rq59j+v9wKTSHHhIPtI0zam5NByhNH5enGr306Fa9j1720ttuUQAE0tz4Epi/FJM
-        cl24b8ZNd25KdTk31whg8mkOXEm4pX1EbS4LHebn5x+WdnIf765fr3NVCOFRuUwA00Nz4Gr21kX58lwW
-        1qjmq8el+nxjqF4S4yVLS0sn5TIBTB/NgWuiOfAQTVE+J9XltnV16nXCvqqIF+cSAUw3zYGH5INnb9/+
-        4Fyavtra7uRSLe4Zqk3f88MmxvNyjQBmhiavg/liVVVFrkuvLC4uPjSt/4eG6iEhXlOWZchlApg9mgNX
-        c3P6wD8nl6UXqvlqZ1rv/xmqQ+/ThPJv5ufnH5TLBDC7NAeuZm97n/tclplWhfCstL4/GVr/nsf1fqCH
-        NAeu5kB7aSSVZFabA13v786tVVFckGsE0C+aAw8mnSG/f25u7pRcmpnQbt8mlB/oWt8+pwnxWk+PBNAc
-        uJoqxC/MSnNgs7BQ1yFc17WePc97Z+1AD+C4aA5czU3T/qjXdED3zLQePx5ar75nf77ev3VQJQBWaQ5c
-        zZ4qhJfkskyVuigvapvbOtapt2lC/FFTls/IJQKgi+bA1aw0B07FGWOM8eQ6xnd1rEe/E+OX2jGdywTA
-        fdEcuCYx/u2kXzNudjTz6az/c53L3+NUMX7YI6EBjp7mwIO5pt3J5rpMlHKhfFpavu8PLW/fM+s/7QTY
-        eJoDV/PduiiWc1kmwuB6f7x7aDn7ntvrGC/MJQLgeGgOHKQK8Y46hOflsoxN0zQPbEJ8Z9cy9jxf3RnC
-        o3OZABgFzYGrab9efl0qyViaAxcXFubSgchnOpar77k8xnhaLhMAo6Q5cE2K+L7NfoBMHcJT0nvfvG5Z
-        +h3X+wE2iebAg7m6LMvtuS4bKl/v14txSMLuKoQX5BIBsBk0B64kfCfV4om5LCN33pYtD3DA1ZVwfb1Q
-        PzaXCYDNpDlwJWH3RnSe75rbdUZ6/X9b/369z0eLojg9lwmAcdAcuJqV5sCRaEJ4QnrNbw+9R9+zcr1/
-        26BKAIyV5sBDcll7W95cmmPSPocgvc6dQ6/b9+xpivJluUQATBDNgQdz1TE2B6phV4p446TdhAmAIens
-        9ZXpQ1tz4KA58Am5LPfrrB07Hp7+zr90v1Z/UxXxU3Vdn5nLBMAk0xy4krC7WSifm8tyWGVZnl2F+M3u
-        1+hxYrxkeXn5xFwmAKaB5sDV7E9nsRfnsqyTdnIvGtxiuPPv9jV7Ul1ekUsEwLTRHHgwVSjfsbS0dFIu
-        TWtr+6uB9P8ODP/Znue7ZVmek2sEwBTT2Laa8sr2enb7jHoHRl0JV8QYH5nHDQCzwJ0DV/P1tKO7vuO/
-        9zpVKN/qej/AjNIcKB3Z24TyVXmIADCrNAfKmtzUxHhuHhoAzDrNgZJyVQjhUXlIANAjmgN7m/Du471d
-        MgBTzp0D+5Sw777uiQBAz2gO7EPCLSnn500OAAOaA2c615RlGfKmBoBDaQ6cvTQhvmd+fv5BeRMDwGFp
-        DpyJuN4PwDHQHDjVubUqigvypgSAo6M5cPrShHjtYghl3oQAcGw0B05Rivi+ubm5U/KmA4Djozlw4rM/
-        X+/fOthiADA6mgMnME2IP2rK8hl5GwHAxtAcOFH5SozxrLxpAGBjaQ4cf6oYP9w0zal5kwDA5tAcOLYc
-        aC/FpE1wwmBLAMAm0xy46bk91fvCXH4AGCvNgZuRIn6tKYrH5JoDwGTQHLihuTzGeFouNQBMFs2BI4/r
-        /QBMB82Bo0rYXYXwglxWAJh8mgOPN+H6eqF+bC4nAEwVzYHHkiL+c1EUp+caAsB00hx4FInxLalk2waV
-        A4AppznwfrOnLsqX53IBwOzQHHiYFPHGaqF6Ui4TAMwezYGHpirip+q6PjOXBwBmmubANjFesry8fGKu
-        CQD0Q4+bA/dWMf56LgMA9E8PmwO/W5blOXn1AaC/+tMcWF4ZY3xkXm0AYOabA2O8ZGlp6aS8ugDAGrPY
-        HLi3CeWr8voBAIczQ82BN6UDmnPzagEA92f6mwPLz1dVVeTVAQCO1LQ2B1Yh/PX8/PyD8moAAEdrupoD
-        w76qiBfnRQcAjtMUNAeGW1LOz8sLAIzKBDcHXlOWZciLCQCM2gQ2B142Nzd3Sl48AGCjTEhz4H7X+wFg
-        k425OfDWqqh+Pi8KALDJtqWDgDemHfKBoR30hqUJ8dqY5PcHAMalKcrnpJ3z94Z31qNO+/t+1/sBYIKk
-        k/LT6hD+Mu2oR/4rgXTW/+X02r+U3woAmDTtz/GaIv5F2nH/ZHhHfpS5J+Vf01n/S9LLbhu8OgAw0dpb
-        8TYL5XPTmfvb0o78v/MOvWtHvybtzXzKT1Qx/n5d1wv5pQCAaXXvAUFRLLXd+3WMFzZF+cKUl7X/rIri
-        gsWFhbn8RwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-        AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA6LEtW/4flgYiLD1qeX0A
-        AAAASUVORK5CYII=
-</value>
-  </data>
-  <data name="ActionButton5.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton6.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-</root>

SCrawler/API/Mastodon/SettingsForm.vb

@@ -1,154 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Imports PersonalUtilities.Forms
-Imports PersonalUtilities.Forms.Controls.Base
-Namespace API.Mastodon
-    Friend Class SettingsForm
-        Private WithEvents MyDefs As DefaultFormOptions
-        Friend ReadOnly Property MyCredentials As List(Of Credentials)
-        Friend ReadOnly Property MyDomains As List(Of String)
-        Friend Sub New(ByVal s As SiteSettings)
-            InitializeComponent()
-            MyCredentials = New List(Of Credentials)
-            If s.Domains.Credentials.Count > 0 Then MyCredentials.AddRange(s.Domains.Credentials)
-            MyDomains = New List(Of String)
-            MyDomains.ListAddList(s.Domains)
-            MyDefs = New DefaultFormOptions(Me, Settings.Design)
-        End Sub
-        Private Sub MyForm_Load(sender As Object, e As EventArgs) Handles Me.Load
-            With MyDefs
-                .MyView = New FormView(Me, Settings.Design, "MastodonSettingsForm")
-                .MyView.Import()
-                .MyView.SetFormSize()
-                .AddOkCancelToolbar()
-                RefillList()
-                .EndLoaderOperations()
-            End With
-        End Sub
-        Private Sub SettingsForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed
-            MyCredentials.Clear()
-            MyDomains.Clear()
-        End Sub
-        Private Sub RefillList()
-            CMB_DOMAINS.Items.Clear()
-            If MyDomains.Count > 0 Then
-                MyDomains.Sort()
-                CMB_DOMAINS.BeginUpdate()
-                CMB_DOMAINS.Items.AddRange(MyDomains.Select(Function(d) New ListItem(d)))
-                CMB_DOMAINS.EndUpdate(True)
-            End If
-        End Sub
-        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
-            ApplyCredentials()
-            If MyCredentials.Count > 0 Then MyCredentials.RemoveAll(Function(c) c.Domain.IsEmptyString Or c.Bearer.IsEmptyString Or c.Csrf.IsEmptyString)
-            If MyDomains.Count > 0 Then
-                If MyCredentials.Count > 0 Then
-                    MyCredentials.RemoveAll(Function(c) Not MyDomains.Contains(c.Domain))
-                Else
-                    MyCredentials.Clear()
-                End If
-            End If
-            MyDefs.CloseForm()
-        End Sub
-        Private Sub CMB_DOMAINS_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As ActionButtonEventArgs) Handles CMB_DOMAINS.ActionOnButtonClick
-            Try
-                Dim d$
-                Dim i% = -1
-                Select Case e.DefaultButton
-                    Case ActionButton.DefaultButtons.Add
-                        d = InputBoxE("Enter a new domain using the pattern [mastodon.social]:", "New domain")
-                        If Not d.IsEmptyString Then
-                            If MyDomains.Count > 0 Then i = MyDomains.IndexOf(d)
-                            If i >= 0 Then
-                                MsgBoxE({$"Domain '{d}' already exists", "Add domain"}, vbExclamation)
-                                If i <= CMB_DOMAINS.Count - 1 Then CMB_DOMAINS.SelectedIndex = i
-                            Else
-                                ApplyCredentials()
-                                ClearCredentials()
-                                MyDomains.Add(d)
-                                _Suspended = True
-                                RefillList()
-                                _Suspended = False
-                                i = MyDomains.IndexOf(d)
-                                If i.ValueBetween(0, CMB_DOMAINS.Count - 1) Then
-                                    CMB_DOMAINS.SelectedIndex = i
-                                Else
-                                    _LatestSelected = -1
-                                    _CurrentCredentialsIndex = -1
-                                    _CurrentDomain = String.Empty
-                                End If
-                            End If
-                        End If
-                    Case ActionButton.DefaultButtons.Delete
-                        If _LatestSelected >= 0 Then
-                            d = CMB_DOMAINS.Items(_LatestSelected).Value(0)
-                            If Not d.IsEmptyString AndAlso MsgBoxE({$"Are you sure you want to delete the [{d}] domain?",
-                                                                   "Removing domains"}, vbYesNo) = vbYes Then
-                                i = MyDomains.IndexOf(d)
-                                Dim l% = _LatestSelected
-                                If i >= 0 Then
-                                    ClearCredentials()
-                                    MyDomains.RemoveAt(i)
-                                    _Suspended = True
-                                    RefillList()
-                                    _Suspended = False
-                                    If (l - 1).ValueBetween(0, CMB_DOMAINS.Count - 1) Then
-                                        CMB_DOMAINS.SelectedIndex = l - 1
-                                    Else
-                                        _LatestSelected = -1
-                                        _CurrentCredentialsIndex = -1
-                                        _CurrentDomain = String.Empty
-                                    End If
-                                End If
-                            End If
-                        End If
-                End Select
-            Catch ex As Exception
-                ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "API.Mastodon.SettingsForm.ActionButtonClick")
-            End Try
-        End Sub
-        Private _LatestSelected As Integer = -1
-        Private _CurrentCredentialsIndex As Integer = -1
-        Private _CurrentDomain As String = String.Empty
-        Private _Suspended As Boolean = False
-        Private Sub CMB_DOMAINS_ActionSelectedItemChanged(ByVal Sender As Object, ByVal e As EventArgs, ByVal Item As ListViewItem) Handles CMB_DOMAINS.ActionSelectedItemChanged
-            If Not MyDefs.Initializing And Not _Suspended Then
-                Dim DropCredentials As Boolean = True
-                If Not Item Is Nothing Then
-                    ApplyCredentials()
-                    _LatestSelected = Item.Index
-                    _CurrentDomain = Item.Text
-                    If MyCredentials.Count > 0 And Not _CurrentDomain.IsEmptyString Then
-                        _CurrentCredentialsIndex = MyCredentials.IndexOf(_CurrentDomain)
-                        If _CurrentCredentialsIndex >= 0 Then
-                            With MyCredentials(_CurrentCredentialsIndex) : TXT_AUTH.Text = .Bearer : TXT_TOKEN.Text = .Csrf : End With
-                            DropCredentials = False
-                        End If
-                    Else
-                        _CurrentCredentialsIndex = -1
-                    End If
-                End If
-                If DropCredentials Then ClearCredentials()
-            End If
-        End Sub
-        Private Sub ClearCredentials()
-            TXT_AUTH.Clear()
-            TXT_TOKEN.Clear()
-        End Sub
-        Private Sub ApplyCredentials()
-            Try
-                If _LatestSelected >= 0 And Not _CurrentDomain.IsEmptyString Then
-                    Dim c As New Credentials With {.Domain = _CurrentDomain, .Bearer = TXT_AUTH.Text, .Csrf = TXT_TOKEN.Text}
-                    If _CurrentCredentialsIndex.ValueBetween(0, MyCredentials.Count - 1) Then MyCredentials(_CurrentCredentialsIndex) = c Else MyCredentials.Add(c)
-                End If
-            Catch ex As Exception
-            End Try
-        End Sub
-    End Class
-End Namespace

SCrawler/API/Mastodon/SiteSettings.vb

@@ -18,29 +18,34 @@ Namespace API.Mastodon
     <Manifest(MastodonSiteKey), SavedPosts, SpecialForm(True), SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.MastodonIcon_48
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.MastodonPic_48
-            End Get
-        End Property
 #Region "Domains"
-        <PXML("Domains")> Private ReadOnly Property SiteDomains As PropertyValue
-        Friend ReadOnly Property Domains As MastodonDomains
-        <PXML> Private ReadOnly Property DomainsLastUpdateDate As PropertyValue
+        <PXML("Domains"), PClonable> Private ReadOnly Property SiteDomains As PropertyValue
+        Private Shadows ReadOnly Property DefaultInstance As SiteSettings
+            Get
+                Return MyBase.DefaultInstance
+            End Get
+        End Property
+        Private ReadOnly _Domains As DomainsContainer
+        Friend ReadOnly Property Domains As DomainsContainer
+            Get
+                Return If(DefaultInstance?.Domains, _Domains)
+            End Get
+        End Property
+        <PXML("DomainsLastUpdateDate")> Private ReadOnly Property Base_DomainsLastUpdateDate As PropertyValue
+        Private ReadOnly Property DomainsLastUpdateDate As PropertyValue
+            Get
+                Return If(DefaultInstance?.DomainsLastUpdateDate, Base_DomainsLastUpdateDate)
+            End Get
+        End Property
 #End Region
 #Region "Auth"
         <PropertyOption(IsAuth:=True, AllowNull:=False, ControlText:="My domain",
-                        ControlToolTip:="Your account domain without 'https://' (for example, 'mastodon.social')"), PXML>
+                        ControlToolTip:="Your account domain without 'https://' (for example, 'mastodon.social')"), PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property MyDomain As PropertyValue
         <PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Authorization",
-                        ControlToolTip:="Set authorization from [authorization] response header. This field must start from [Bearer] key word")>
+                        ControlToolTip:="Set authorization from [authorization] response header. This field must start from [Bearer] key word"), PClonable(Clone:=False)>
         Friend ReadOnly Property Auth As PropertyValue
-        <PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header")>
+        <PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header"), PClonable(Clone:=False)>
         Friend ReadOnly Property Token As PropertyValue
         Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object)
             If Not PropName.IsEmptyString Then
@@ -58,29 +63,29 @@ Namespace API.Mastodon
         End Sub
 #End Region
 #Region "Other properties"
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsDownloadCaption), PXML>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsDownloadCaption), PXML, PClonable>
         Friend ReadOnly Property GifsDownload As PropertyValue
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip), PXML>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsSpecialFolderCaption, ControlToolTip:=DN.GifsSpecialFolderToolTip), PXML, PClonable>
         Friend ReadOnly Property GifsSpecialFolder As PropertyValue
-        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip), PXML>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.GifsPrefixCaption, ControlToolTip:=DN.GifsPrefixToolTip), PXML, PClonable>
         Friend ReadOnly Property GifsPrefix As PropertyValue
         <Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)>
         Private ReadOnly Property GifStringChecker As IFormatProvider
-        <PropertyOption(IsAuth:=False, ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip), PXML>
+        <PropertyOption(IsAuth:=False, ControlText:=DN.UseMD5ComparisonCaption, ControlToolTip:=DN.UseMD5ComparisonToolTip), PXML, PClonable>
         Friend ReadOnly Property UseMD5Comparison As PropertyValue
         <PropertyOption(IsAuth:=False, ControlText:="User related to my domain",
-                        ControlToolTip:="Open user profiles and user posts through my domain."), PXML>
+                        ControlToolTip:="Open user profiles and user posts through my domain."), PXML, PClonable>
         Friend ReadOnly Property UserRelatedToMyDomain As PropertyValue
 #End Region
 #End Region
 #Region "Initializer"
-        Friend Sub New()
-            MyBase.New("Mastodon", "mastodon.social")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("Mastodon", "mastodon.social", AccName, Temp, My.Resources.SiteResources.MastodonIcon_48, My.Resources.SiteResources.MastodonPic_48)
 
-            Domains = New MastodonDomains(Me, "mastodon.social")
+            _Domains = New DomainsContainer(Me, "mastodon.social")
             SiteDomains = New PropertyValue(Domains.DomainsDefault, GetType(String))
             Domains.DestinationProp = SiteDomains
-            DomainsLastUpdateDate = New PropertyValue(Now.AddYears(-1))
+            Base_DomainsLastUpdateDate = New PropertyValue(Now.AddYears(-1))
 
             Auth = New PropertyValue(Responser.Headers.Value(DN.Header_Authorization), GetType(String), Sub(v) ChangeResponserFields(NameOf(Auth), v))
             Token = New PropertyValue(Responser.Headers.Value(DN.Header_CSRFToken), GetType(String), Sub(v) ChangeResponserFields(NameOf(Token), v))
@@ -182,20 +187,32 @@ Namespace API.Mastodon
 #End Region
 #Region "IsMyUser, IsMyImageVideo"
         Private Const UserRegexDefault As String = "https?://{0}/@([^/@]+)@?([^/]*)"
-        Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
+        Friend Overloads Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
             If Domains.Count > 0 Then
-                Dim l As List(Of String)
+                Dim e As ExchangeOptions
+                If ACheck(MyDomain.Value) Then
+                    e = IsMyUser(UserURL, MyDomain.Value)
+                    If Not e.SiteName.IsEmptyString Then Return e
+                End If
                 For Each domain$ In Domains
-                    UserRegex.Pattern = String.Format(UserRegexDefault, domain)
-                    l = RegexReplace(UserURL, UserRegex)
-                    If l.ListExists(2) Then Return New ExchangeOptions(Site, $"{l(2).IfNullOrEmpty(domain)}@{l(1)}")
+                    e = IsMyUser(UserURL, domain)
+                    If Not e.SiteName.IsEmptyString Then Return e
                 Next
             End If
             Return Nothing
         End Function
+        Private Overloads Function IsMyUser(ByVal UserURL As String, ByVal Domain As String) As ExchangeOptions
+            UserRegex.Pattern = String.Format(UserRegexDefault, Domain)
+            Dim l As List(Of String) = RegexReplace(UserURL, UserRegex)
+            If l.ListExists(2) Then Return New ExchangeOptions(Site, $"{l(2).IfNullOrEmpty(Domain)}@{l(1)}") Else Return Nothing
+        End Function
         Friend Overrides Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions
             If Not URL.IsEmptyString And Domains.Count > 0 Then
-                If Domains.Domains.Exists(Function(d) URL.Contains(d)) Then Return New ExchangeOptions(Site, URL) With {.Exists = True}
+                Dim urlDomain$ = RegexReplace(URL, RParams.DM("[^/]+", 1, EDP.ReturnValue, String.Empty))
+                If Not urlDomain.IsEmptyString Then
+                    urlDomain = urlDomain.StringToLower
+                    If Domains.Domains.Exists(Function(d) urlDomain = d.StringToLower) Then Return New ExchangeOptions(Site, URL) With {.Exists = True}
+                End If
             End If
             Return Nothing
         End Function
@@ -222,6 +239,12 @@ Namespace API.Mastodon
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[API.Mastodon.SiteSettings.UpdateServersList]")
             End Try
         End Sub
+#End Region
+#Region "IDisposable Support"
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            If Not disposedValue And disposing Then _Domains.Dispose()
+            MyBase.Dispose(disposing)
+        End Sub
 #End Region
     End Class
 End Namespace

SCrawler/API/Mastodon/UserData.vb

@@ -18,7 +18,6 @@ Namespace API.Mastodon
     Friend Class UserData : Inherits Twitter.UserData
 #Region "XML names"
         Private Const Name_UserDomain As String = "UserDomain"
-        Private Const Name_TrueName As String = "TrueName"
 #End Region
 #Region "Declarations"
         Private _UserDomain As String = String.Empty
@@ -38,22 +37,7 @@ Namespace API.Mastodon
         End Property
         Private MyCredentials As Credentials
         Private Sub ResetCredentials()
-            MyCredentials = Nothing
-            With MySettings
-                Dim setDef As Boolean = True
-                If Not IsSavedPosts Then
-                    If ACheck(.MyDomain.Value) AndAlso UserDomain = .MyDomain.Value Then
-                        setDef = True
-                    ElseIf .Domains.Credentials.Count > 0 Then
-                        Dim i% = .Domains.Credentials.IndexOf(UserDomain)
-                        If i >= 0 Then
-                            MyCredentials = .Domains.Credentials(i)
-                            setDef = Not MyCredentials.Exists
-                        End If
-                    End If
-                End If
-                If setDef Then MyCredentials = New Credentials With {.Domain = UserDomain, .Bearer = MySettings.Auth.Value, .Csrf = MySettings.Token.Value}
-            End With
+            MyCredentials = New Credentials With {.Domain = MySettings.MyDomain.Value, .Bearer = MySettings.Auth.Value, .Csrf = MySettings.Token.Value}
             With MyCredentials
                 Responser.Headers.Add(DeclaredNames.Header_Authorization, .Bearer)
                 Responser.Headers.Add(DeclaredNames.Header_CSRFToken, .Csrf)

SCrawler/API/OnlyFans/SiteSettings.vb

@@ -16,23 +16,11 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.OnlyFans
     <Manifest("AndyProgram_OnlyFans"), SavedPosts, SpecialForm(False), SeparatedTasks(1)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
-#Region "Icon"
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.OnlyFansIcon_32
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.OnlyFansPic_32
-            End Get
-        End Property
-#End Region
 #Region "Declarations"
 #Region "Options"
-        <PropertyOption(ControlText:="Download highlights", ControlToolTip:="Download profile highlights if they exists"), PXML>
+        <PropertyOption(ControlText:="Download highlights", ControlToolTip:="Download profile highlights if they exists"), PXML, PClonable>
         Friend Property DownloadHighlights As PropertyValue
-        <PropertyOption(ControlText:="Download chat", ControlToolTip:="Download unlocked chat media"), PXML>
+        <PropertyOption(ControlText:="Download chat", ControlToolTip:="Download unlocked chat media"), PXML, PClonable>
         Friend Property DownloadChatMedia As PropertyValue
 #End Region
 #Region "Headers"
@@ -40,15 +28,15 @@ Namespace API.OnlyFans
         Private Const HeaderUserID As String = "User-Id"
         Private Const HeaderXBC As String = "X-Bc"
         Private Const HeaderAppToken As String = "App-Token"
-        <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False)>
+        <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_USER_ID As PropertyValue
-        <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False)>
+        <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False), PClonable(Clone:=False)>
         Private ReadOnly Property HH_X_BC As PropertyValue
-        <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False)>
+        <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False), PClonable(Clone:=False)>
         Private ReadOnly Property HH_APP_TOKEN As PropertyValue
-        <PropertyOption(ControlText:=HeaderBrowser, ControlToolTip:="Can be null", AllowNull:=True)>
+        <PropertyOption(ControlText:=HeaderBrowser, ControlToolTip:="Can be null", AllowNull:=True), PClonable>
         Private ReadOnly Property HH_BROWSER As PropertyValue
-        <PropertyOption(AllowNull:=False)>
+        <PropertyOption(AllowNull:=False), PClonable>
         Private ReadOnly Property UserAgent As PropertyValue
         Private Sub UpdateHeader(ByVal PropertyName As String, ByVal Value As String)
             Dim hName$ = String.Empty
@@ -79,21 +67,21 @@ Namespace API.OnlyFans
         End Property
         <PropertyOption(ControlText:="Use old authorization rules",
                         ControlToolTip:="Use old dynamic rules (from 'DATAHOARDERS') or new ones (from 'DIGITALCRIMINALS')." & vbCr &
-                                        "Change this value only if you know what you are doing."), PXML>
+                                        "Change this value only if you know what you are doing."), PXML, PClonable>
         Friend ReadOnly Property UseOldAuthRules As PropertyValue
-        <PropertyOption(ControlText:="Dynamic rules update", ControlToolTip:="'Dynamic rules' update interval (minutes). Default: 1440", LeftOffset:=110), PXML>
+        <PropertyOption(ControlText:="Dynamic rules update", ControlToolTip:="'Dynamic rules' update interval (minutes). Default: 1440", LeftOffset:=110), PXML, PClonable>
         Friend ReadOnly Property DynamicRulesUpdateInterval As PropertyValue
         <Provider(NameOf(DynamicRulesUpdateInterval), FieldsChecker:=True)>
         Private ReadOnly Property DynamicRulesUpdateIntervalProvider As IFormatProvider
         <PropertyOption(ControlText:="Dynamic rules",
                         ControlToolTip:="Overwrite 'Dynamic rules' with this URL" & vbCr &
-                                        "Change this value only if you know what you are doing."), PXML>
+                                        "Change this value only if you know what you are doing."), PXML, PClonable>
         Friend ReadOnly Property DynamicRules As PropertyValue
 #End Region
 #End Region
 #Region "Initializer"
-        Friend Sub New()
-            MyBase.New("OnlyFans", ".onlyfans.com")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("OnlyFans", ".onlyfans.com", AccName, Temp, My.Resources.SiteResources.OnlyFansIcon_32, My.Resources.SiteResources.OnlyFansPic_32)
 
             With Responser
                 .Accept = "application/json, text/plain, */*"

SCrawler/API/OnlyFans/UserData.vb

@@ -380,7 +380,7 @@ Namespace API.OnlyFans
             Try
                 If ContentMissingExists Then
                     Dim m As UserMedia
-                    Dim stateRefill As Func(Of UserMedia, Integer, UserMedia) = Function(ByVal input As UserMedia, ii As Integer) As UserMedia
+                    Dim stateRefill As Func(Of UserMedia, Integer, UserMedia) = Function(ByVal input As UserMedia, ByVal ii As Integer) As UserMedia
                                                                                     input.State = UStates.Missing
                                                                                     input.Attempts = m.Attempts
                                                                                     Return input
@@ -444,42 +444,56 @@ Namespace API.OnlyFans
                 Return f
             End Get
         End Property
-        Private Function UpdateSignature(ByVal Path As String, Optional ByVal ForceUpdateAuth As Boolean = False) As Boolean
+        Private Function UpdateSignature(ByVal Path As String, Optional ByVal ForceUpdateAuth As Boolean = False,
+                                         Optional ByVal Round As Integer = 0) As Boolean
             Try
                 If UpdateAuthFile(ForceUpdateAuth) Then
                     Const nullMsg$ = "The auth parameter is null"
-                    Dim j As EContainer = JsonDocument.Parse(AuthFile.GetText)
-                    Dim pattern$ = j.Value("format")
-                    If pattern.IsEmptyString Then Throw New ArgumentNullException("format", nullMsg)
-                    pattern = pattern.Replace("{}", "{0}").Replace("{:x}", "{1:x}")
+                    Dim j As EContainer
+                    Try
+                        j = JsonDocument.Parse(AuthFile.GetText)
+                    Catch jex As Exception
+                        If Round = 0 Then
+                            AuthFile.Delete()
+                            UpdateAuthFile(True)
+                            Return UpdateSignature(Path, ForceUpdateAuth, Round + 1)
+                        Else
+                            MySettings.SessionAborted = True
+                            Return False
+                        End If
+                    End Try
+                    If Not j Is Nothing Then
+                        Dim pattern$ = j.Value("format")
+                        If pattern.IsEmptyString Then Throw New ArgumentNullException("format", nullMsg)
+                        pattern = pattern.Replace("{}", "{0}").Replace("{:x}", "{1:x}")
 
-                    Dim li%() = j("checksum_indexes").Select(Function(e) CInt(e(0).Value)).ToArray
+                        Dim li%() = j("checksum_indexes").Select(Function(e) CInt(e(0).Value)).ToArray
 
-                    If Not li.ListExists Then Throw New ArgumentNullException("checksum_indexes", nullMsg)
-                    If j.Value("static_param").IsEmptyString Then Throw New ArgumentNullException("static_param", nullMsg)
-                    If j.Value("checksum_constant").IsEmptyString Then Throw New ArgumentNullException("checksum_constant", nullMsg)
+                        If Not li.ListExists Then Throw New ArgumentNullException("checksum_indexes", nullMsg)
+                        If j.Value("static_param").IsEmptyString Then Throw New ArgumentNullException("static_param", nullMsg)
+                        If j.Value("checksum_constant").IsEmptyString Then Throw New ArgumentNullException("checksum_constant", nullMsg)
 
-                    Dim t$ = ADateTime.ConvertToUnix64(Now.ToUniversalTime).ToString
-                    Dim h$ = String.Join(vbLf, j.Value("static_param"), t, Path, MySettings.HH_USER_ID.Value.ToString)
+                        Dim t$ = ADateTime.ConvertToUnix64(Now.ToUniversalTime).ToString
+                        Dim h$ = String.Join(vbLf, j.Value("static_param"), t, Path, MySettings.HH_USER_ID.Value.ToString)
 
-                    Dim hash$ = GetHashSha1(h)
-                    Dim hashBytes() As Byte = System.Text.Encoding.ASCII.GetBytes(hash)
-                    Dim hashSum% = li.Sum(Function(i) hashBytes(i)) + CInt(j.Value("checksum_constant"))
-                    Dim sign$ = String.Format(pattern, hash, Math.Abs(hashSum))
+                        Dim hash$ = GetHashSha1(h)
+                        Dim hashBytes() As Byte = System.Text.Encoding.ASCII.GetBytes(hash)
+                        Dim hashSum% = li.Sum(Function(i) hashBytes(i)) + CInt(j.Value("checksum_constant"))
+                        Dim sign$ = String.Format(pattern, hash, Math.Abs(hashSum))
 
-                    '#If DEBUG Then
-                    'Debug.WriteLine(sign)
-                    'Debug.WriteLine(t)
-                    '#End If
+                        '#If DEBUG Then
+                        'Debug.WriteLine(sign)
+                        'Debug.WriteLine(t)
+                        '#End If
 
-                    Responser.Headers.Add(HeaderSign, sign)
-                    Responser.Headers.Add(HeaderTime, t)
+                        Responser.Headers.Add(HeaderSign, sign)
+                        Responser.Headers.Add(HeaderTime, t)
 
-                    j.Dispose()
-                    Return True
-                Else
-                    Return False
+                        j.Dispose()
+                        Return True
+                    End If
                 End If
+                Return False
             Catch ex As Exception
                 Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"{ToStringForLog()}: UpdateSignature", False)
             End Try
@@ -526,7 +540,7 @@ Namespace API.OnlyFans
         Private _DownloadingException_AuthFileUpdate As Boolean = False
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer
-            If Responser.StatusCode = Net.HttpStatusCode.BadRequest Then
+            If Responser.StatusCode = Net.HttpStatusCode.BadRequest Then '400
                 If Not _DownloadingException_AuthFileUpdate AndAlso UpdateAuthFile(True) Then
                     _DownloadingException_AuthFileUpdate = True
                     Return 2
@@ -535,13 +549,17 @@ Namespace API.OnlyFans
                     MyMainLOG = $"{ToStringForLog()}: OnlyFans credentials expired"
                     Return 1
                 End If
-            ElseIf Responser.StatusCode = Net.HttpStatusCode.NotFound Then
+            ElseIf Responser.StatusCode = Net.HttpStatusCode.NotFound Then '404
                 UserExists = False
                 Return 1
-            ElseIf Responser.StatusCode = Net.HttpStatusCode.GatewayTimeout Or Responser.StatusCode = 429 Then
+            ElseIf Responser.StatusCode = Net.HttpStatusCode.GatewayTimeout Or Responser.StatusCode = 429 Then '504, 429
                 If Responser.StatusCode = 429 Then MyMainLOG = $"[429] OnlyFans too many requests ({ToStringForLog()})"
                 MySettings.SessionAborted = True
                 Return 1
+            ElseIf Responser.StatusCode = Net.HttpStatusCode.Unauthorized Then '401
+                MySettings.SessionAborted = True
+                MyMainLOG = $"{ToStringForLog()}: OnlyFans credentials expired"
+                Return 1
             Else
                 Return 0
             End If

SCrawler/API/PathPlugin/SiteSettings.vb

@@ -12,19 +12,8 @@ Imports SCrawler.Plugin.Attributes
 Namespace API.PathPlugin
     <Manifest(PluginKey)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
-        Private ReadOnly _Icon As Icon = Nothing
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return _Icon
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return PersonalUtilities.My.Resources.FolderOpenPic_Orange_16
-            End Get
-        End Property
         Friend Sub New()
-            MyBase.New(PluginName)
+            MyBase.New(PluginName,, PersonalUtilities.My.Resources.FolderOpenPic_Orange_16)
             _Icon = PersonalUtilities.Tools.ImageRenderer.GetIcon(PersonalUtilities.My.Resources.FolderOpenPic_Orange_16, EDP.ReturnValue)
         End Sub
         Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider

SCrawler/API/Pinterest/SiteSettings.vb

@@ -9,33 +9,22 @@
 Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
-Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Pinterest
     <Manifest("AndyProgram_Pinterest"), SavedPosts, SeparatedTasks>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.PinterestIcon_32
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.PinterestPic_48
-            End Get
-        End Property
         <PropertyOption(ControlText:=DeclaredNames.ConcurrentDownloadsCaption,
-                        ControlToolTip:=DeclaredNames.ConcurrentDownloadsToolTip, AllowNull:=False, LeftOffset:=120), PXML, TaskCounter>
+                        ControlToolTip:=DeclaredNames.ConcurrentDownloadsToolTip, AllowNull:=False, LeftOffset:=120), PXML, TaskCounter, PClonable>
         Friend ReadOnly Property ConcurrentDownloads As PropertyValue
         <Provider(NameOf(ConcurrentDownloads), FieldsChecker:=True)>
         Private ReadOnly Property MyConcurrentDownloadsProvider As IFormatProvider
-        <PropertyOption(ControlText:=DeclaredNames.SavedPostsUserNameCaption, ControlToolTip:=DeclaredNames.SavedPostsUserNameToolTip), PXML>
+        <PropertyOption(ControlText:=DeclaredNames.SavedPostsUserNameCaption, ControlToolTip:=DeclaredNames.SavedPostsUserNameToolTip), PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property SavedPostsUserName As PropertyValue
 #End Region
 #Region "Initializer"
-        Friend Sub New()
-            MyBase.New("Pinterest", "pinterest.com")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("Pinterest", "pinterest.com", AccName, Temp, My.Resources.SiteResources.PinterestIcon_32, My.Resources.SiteResources.PinterestPic_48)
             SavedPostsUserName = New PropertyValue(String.Empty, GetType(String))
             ConcurrentDownloads = New PropertyValue(1)
             MyConcurrentDownloadsProvider = New ConcurrentDownloadsProvider

SCrawler/API/PornHub/Declarations.vb

@@ -24,6 +24,7 @@ Namespace API.PornHub
                                                                 0, RegexOptions.Singleline, RegexReturn.List, EDP.ReturnValue, UnicodeHexConverter)
         Friend ReadOnly RegexVideo_Video_VideoKey As RParams = RParams.DMS("viewkey=([\w\d]+)", 1, EDP.ReturnValue)
         Friend ReadOnly RegexVideoPageTitle As RParams = RParams.DMS("meta (property|name)=""[^:]+?:title"" content=""([^""]+)""", 2, EDP.ReturnValue)
+        Friend ReadOnly RegexDataToken As RParams = RParams.DMS("data-token=""([^""]+)", 1, EDP.ReturnValue)
 #End Region
 #Region "Declarations M3U8"
         Friend ReadOnly Regex_M3U8_FilesList As RParams = RParams.DM("RESOLUTION=\d+x(\d+).*?[\r\n]*?(.+?m3u8.*)", 0, RegexReturn.List, EDP.ReturnValue)

SCrawler/API/PornHub/SiteSettings.vb

@@ -15,40 +15,30 @@ Namespace API.PornHub
     <Manifest("AndyProgram_PornHub"), SavedPosts, SpecialForm(False), SeparatedTasks(1)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
-        Friend Overrides ReadOnly Property Icon As Icon
-            Get
-                Return My.Resources.SiteResources.PornHubIcon_16
-            End Get
-        End Property
-        Friend Overrides ReadOnly Property Image As Image
-            Get
-                Return My.Resources.SiteResources.PornHubPic_16
-            End Get
-        End Property
-        <PropertyOption(ControlText:="Download UHD", ControlToolTip:="Download UHD (4K) content"), PXML>
+        <PropertyOption(ControlText:="Download UHD", ControlToolTip:="Download UHD (4K) content"), PXML, PClonable>
         Friend Property DownloadUHD As PropertyValue
-        <PropertyOption(ControlText:="Download uploaded", ControlToolTip:="Download uploaded videos"), PXML>
+        <PropertyOption(ControlText:="Download uploaded", ControlToolTip:="Download uploaded videos"), PXML, PClonable>
         Friend Property DownloadUploaded As PropertyValue
-        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged videos"), PXML>
+        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged videos"), PXML, PClonable>
         Friend Property DownloadTagged As PropertyValue
-        <PropertyOption(ControlText:="Download private", ControlToolTip:="Download private videos"), PXML>
+        <PropertyOption(ControlText:="Download private", ControlToolTip:="Download private videos"), PXML, PClonable>
         Friend Property DownloadPrivate As PropertyValue
-        <PropertyOption(ControlText:="Download favorite", ControlToolTip:="Download favorite videos"), PXML>
+        <PropertyOption(ControlText:="Download favorite", ControlToolTip:="Download favorite videos"), PXML, PClonable>
         Friend Property DownloadFavorite As PropertyValue
-        <PropertyOption(ControlText:="Download GIF", ControlToolTip:="Default for new users", ThreeStates:=True), PXML>
+        <PropertyOption(ControlText:="Download GIF", ControlToolTip:="Default for new users", ThreeStates:=True), PXML, PClonable>
         Friend ReadOnly Property DownloadGifs As PropertyValue
-        <PropertyOption(ControlText:="Download GIFs as mp4", ControlToolTip:="Download gifs in 'mp4' format instead of native 'webm'"), PXML>
+        <PropertyOption(ControlText:="Download GIFs as mp4", ControlToolTip:="Download gifs in 'mp4' format instead of native 'webm'"), PXML, PClonable>
         Friend ReadOnly Property DownloadGifsAsMp4 As PropertyValue
         <PropertyOption(ControlText:="Photo ModelHub only",
                         ControlToolTip:="Download photo only from ModelHub. Prornstar photos hosted on PornHub itself will not be downloaded." & vbCr &
-                                        "Attention! Downloading photos hosted on PornHub is a very heavy job."), PXML>
+                                        "Attention! Downloading photos hosted on PornHub is a very heavy job."), PXML, PClonable>
         Friend ReadOnly Property DownloadPhotoOnlyFromModelHub As PropertyValue
-        <PropertyOption(ControlText:=DeclaredNames.SavedPostsUserNameCaption, ControlToolTip:=DeclaredNames.SavedPostsUserNameToolTip), PXML>
+        <PropertyOption(ControlText:=DeclaredNames.SavedPostsUserNameCaption, ControlToolTip:=DeclaredNames.SavedPostsUserNameToolTip), PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property SavedPostsUserName As PropertyValue
 #End Region
 #Region "Initializer"
-        Friend Sub New()
-            MyBase.New("PornHub", "pornhub.com")
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("PornHub", "pornhub.com", AccName, Temp, My.Resources.SiteResources.PornHubIcon_16, My.Resources.SiteResources.PornHubPic_16)
             With Responser : .CurlSslNoRevoke = True : .CurlInsecure = True : End With
 
             DownloadUHD = New PropertyValue(False)

SCrawler/API/PornHub/UserData.vb

@@ -29,7 +29,7 @@ Namespace API.PornHub
         Private Const Name_DownloadFavorite As String = "DownloadFavorite"
         Private Const Name_DownloadGifs As String = "DownloadGifs"
         Private Const Name_DownloadPhotoOnlyFromModelHub As String = "DownloadPhotoOnlyFromModelHub"
-        Private Const Name_IsUser As String = "IsUser"
+        <Obsolete> Private Const Name_IsUser As String = "IsUser"
 #End Region
 #Region "Structures"
         Private Structure FlashVar : Implements IRegExCreator
@@ -117,6 +117,8 @@ Namespace API.PornHub
         Private Const PersonTypeUser As String = "users"
         Private Const PersonTypePornstar As String = "pornstar"
         Private Const PersonTypeCannel As String = "channels"
+        Private Const PersonTypePlaylist As String = "playlist"
+        Private Const PlaylistsLabelName As String = "Playlist"
 #End Region
 #Region "Person"
         Friend Property PersonType As String
@@ -133,7 +135,7 @@ Namespace API.PornHub
 #Region "Advanced fields"
         Friend Overrides ReadOnly Property FeedIsUser As Boolean
             Get
-                Return IsUser
+                Return IsUser Or SiteMode = SiteModes.Playlists
             End Get
         End Property
         Private Property PhotoPageModel As PhotoPageModels = PhotoPageModels.Undefined
@@ -144,12 +146,12 @@ Namespace API.PornHub
         Friend Property DownloadFavorite As Boolean = False
         Friend Property DownloadGifs As Boolean
         Friend Property DownloadPhotoOnlyFromModelHub As Boolean = True
-        Private _IsUser As Boolean = True
         Friend Overrides ReadOnly Property IsUser As Boolean
             Get
-                Return _IsUser
+                Return SiteMode = SiteModes.User
             End Get
         End Property
+        Friend Property SiteMode As SiteModes = SiteModes.User
         Friend Property QueryString As String
             Get
                 If IsUser Then
@@ -164,7 +166,7 @@ Namespace API.PornHub
         End Property
         Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
             Get
-                Return {SearchRequestLabelName}
+                Return {SearchRequestLabelName, PlaylistsLabelName}
             End Get
         End Property
 #End Region
@@ -192,38 +194,43 @@ Namespace API.PornHub
                 Return DirectCast(HOST.Source, SiteSettings)
             End Get
         End Property
-        Private ReadOnly LastPageIDs As List(Of String)
 #End Region
 #Region "Initializer"
         Friend Sub New()
-            LastPageIDs = New List(Of String)
             UseInternalM3U8Function = True
             UseClientTokens = True
+            SessionPosts = New List(Of String)
         End Sub
 #End Region
 #Region "Loader"
         Private Function UpdateUserOptions(Optional ByVal Force As Boolean = False, Optional ByVal NewUrl As String = Nothing) As Boolean
 
-            If Not Force OrElse (Not IsUser AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
+            If Not Force OrElse (Not IsUser AndAlso Not SiteMode = SiteModes.Playlists AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
                 Dim eObj As Plugin.ExchangeOptions = Nothing
                 If Force Then eObj = MySettings.IsMyUser(NewUrl)
                 If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And Not Name.IsEmptyString And NameTrue.IsEmptyString) Then
                     If Not If(Force, eObj.Options, Options).IsEmptyString Then
-                        If IsUser And Force Then
+                        If (IsUser Or SiteMode = SiteModes.Playlists) And Force Then
                             Return False
                         Else
-                            _IsUser = False
+                            SiteMode = SiteModes.Search
                             Options = If(Force, eObj.Options, Options)
-                            NameTrue = Options
+                            If Options.ToLower.StartsWith(PersonTypePlaylist) Then
+                                SiteMode = SiteModes.Playlists
+                                NameTrue = Options.ToLower.Replace(PersonTypePlaylist, String.Empty).StringTrim.TrimStart("/")
+                            Else
+                                NameTrue = Options
+                            End If
                             If Not Force Then
-                                Settings.Labels.Add(SearchRequestLabelName)
-                                Labels.ListAddValue(SearchRequestLabelName, LNC)
+                                Dim l$ = IIf(SiteMode = SiteModes.Playlists, PlaylistsLabelName, SearchRequestLabelName)
+                                Settings.Labels.Add(l)
+                                Labels.ListAddValue(l, LNC)
                                 Labels.Sort()
                                 Return True
                             End If
                         End If
                     Else
-                        _IsUser = True
+                        SiteMode = SiteModes.User
                         Dim n$() = Name.Split("_")
                         If n.ListExists(2) Then
                             NameTrue = Name.Replace($"{n(0)}_", String.Empty)
@@ -247,7 +254,14 @@ Namespace API.PornHub
                     DownloadFavorite = .Value(Name_DownloadFavorite).FromXML(Of Boolean)(False)
                     DownloadGifs = .Value(Name_DownloadGifs).FromXML(Of Integer)(False)
                     DownloadPhotoOnlyFromModelHub = .Value(Name_DownloadPhotoOnlyFromModelHub).FromXML(Of Boolean)(True)
-                    _IsUser = .Value(Name_IsUser).FromXML(Of Boolean)(True)
+                    If .Contains(Name_SiteMode) Then
+                        SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
+                    Else
+                        'TODELETE: PornHub 'IsUser' 20231113
+#Disable Warning BC40008
+                        SiteMode = IIf(.Value(Name_IsUser).FromXML(Of Boolean)(True), SiteModes.User, SiteModes.Search)
+#Enable Warning
+                    End If
                     UpdateUserOptions()
                 Else
                     If UpdateUserOptions() Then .Value(Name_LabelsName) = LabelsString
@@ -261,7 +275,7 @@ Namespace API.PornHub
                     .Add(Name_DownloadFavorite, DownloadFavorite.BoolToInteger)
                     .Add(Name_DownloadGifs, DownloadGifs.BoolToInteger)
                     .Add(Name_DownloadPhotoOnlyFromModelHub, DownloadPhotoOnlyFromModelHub.BoolToInteger)
-                    .Add(Name_IsUser, IsUser.BoolToInteger)
+                    .Add(Name_SiteMode, CInt(SiteMode))
 
                     'Debug.WriteLine(GetNonUserUrl(0))
                     'Debug.WriteLine(GetNonUserUrl(2))
@@ -271,9 +285,16 @@ Namespace API.PornHub
 #End Region
 #Region "Downloading"
 #Region "Download override"
+        Private Const PlayListUrlPattern As String = "https://www.pornhub.com/playlist/viewChunked?id={0}&token={1}&page={2}"
+        Private PlaylistToken As String = String.Empty
+        Private ReadOnly SessionPosts As List(Of String)
+        Private _PageVideosRepeat As Integer = 0
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
+                PlaylistToken = String.Empty
                 Responser.ResetStatus()
+                _PageVideosRepeat = 0
+                SessionPosts.Clear()
 
                 If IsSavedPosts Then
                     PersonType = PersonTypeUser
@@ -283,28 +304,32 @@ Namespace API.PornHub
                 Dim limit% = If(DownloadTopCount, -1)
                 If DownloadVideos Then
 
-                    If IsSavedPosts Or Not IsUser Or PersonType = PersonTypeUser Then
+                    If SiteMode = SiteModes.Playlists Then
+                        Responser.Mode = Responser.Modes.Default
+                        GetPlaylistToken(Token)
+                        DownloadUserVideos(1, VideoTypes.Favorite, False, Token)
+                    ElseIf IsSavedPosts Or Not IsUser Or PersonType = PersonTypeUser Then
                         DownloadUserVideos(1, VideoTypes.Favorite, False, Token)
                     Else
                         If DownloadUploaded Then
-                            LastPageIDs.Clear()
+                            SessionPosts.Clear()
                             DownloadUserVideos(1, VideoTypes.Uploaded, False, Token)
                         End If
                         If DownloadTagged Then
-                            LastPageIDs.Clear()
+                            SessionPosts.Clear()
                             Dim lBefore% = _TempMediaList.Count
                             DownloadUserVideos(1, VideoTypes.Tagged, False, Token)
                             If PersonType = PersonTypePornstar And lBefore = _TempMediaList.Count Then
-                                LastPageIDs.Clear()
+                                SessionPosts.Clear()
                                 DownloadUserVideos(1, VideoTypes.Tagged, True, Token)
                             End If
                         End If
                         If DownloadPrivate Then
-                            LastPageIDs.Clear()
+                            SessionPosts.Clear()
                             DownloadUserVideos(1, VideoTypes.Private, False, Token)
                         End If
                         If DownloadFavorite Then
-                            LastPageIDs.Clear()
+                            SessionPosts.Clear()
                             DownloadUserVideos(1, VideoTypes.Favorite, False, Token)
                         End If
                     End If
@@ -348,6 +373,7 @@ Namespace API.PornHub
                 Dim specFolder$ = String.Empty
                 Dim tryNextPage As Boolean = False
                 Dim limit% = If(DownloadTopCount, -1)
+                Dim cBefore% = _TempMediaList.Count
                 If IsUser Then
                     URL = $"https://www.pornhub.com/{PersonType}/{NameTrue}"
                     If Type = VideoTypes.Uploaded Then
@@ -365,6 +391,9 @@ Namespace API.PornHub
                         Throw New ArgumentException($"Type '{Type}' is not implemented in the video download function", "Type")
                     End If
                     If Page > 1 Then URL &= $"?page={Page}"
+                ElseIf SiteMode = SiteModes.Playlists Then
+                    If PlaylistToken.IsEmptyString Then Throw New ArgumentNullException("PlaylistToken", "Unable to get 'PlaylistToken'")
+                    URL = String.Format(PlayListUrlPattern, NameTrue, PlaylistToken, Page)
                 Else
                     URL = GetNonUserUrl(Page)
                 End If
@@ -375,7 +404,7 @@ Namespace API.PornHub
                 Dim r$ = Responser.GetResponse(URL)
                 If Not r.IsEmptyString Then
                     Dim l As List(Of UserVideo) = RegexFields(Of UserVideo)(r, {RegexUserVideos}, {6, 7, 3, 10})
-                    If l.ListExists Then l = l.ListTake(3, l.Count).ToList
+                    If l.ListExists And Not SiteMode = SiteModes.Playlists Then l = l.ListTake(3, l.Count).ToList
                     If l.ListExists Then
                         If IsUser Then
                             If Type = VideoTypes.Favorite Then
@@ -389,23 +418,35 @@ Namespace API.PornHub
                             Dim lBefore% = l.Count
                             Dim nonLastPageDetected As Boolean = False
                             Dim newLastPageIDs As New List(Of String)
+                            Dim pageRepeatSet As Boolean = False, prevPostsFound As Boolean = False, newPostsFound As Boolean = False
                             l.RemoveAll(Function(ByVal uv As UserVideo) As Boolean
+                                            newLastPageIDs.Add(uv.ID)
                                             If Not _TempPostsList.Contains(uv.ID) Then
                                                 _TempPostsList.Add(uv.ID)
-                                                newLastPageIDs.Add(uv.ID)
+                                                newPostsFound = True
                                                 Return False
+                                            ElseIf SessionPosts.Count > 0 AndAlso SessionPosts.Contains(uv.id) Then
+                                                prevPostsFound = True
+                                                Return True
                                             Else
-                                                If Not LastPageIDs.Contains(uv.ID) Then nonLastPageDetected = True
+                                                If Not pageRepeatSet And Not newPostsFound Then pageRepeatSet = True : _PageVideosRepeat += 1
                                                 'Debug.WriteLine($"[REMOVED]: {uv.Title}")
                                                 Return True
                                             End If
                                         End Function)
                             'Debug.WriteLineIf(l.Count > 0, l.Select(Function(ll) ll.Title).ListToString(vbNewLine))
+                            If prevPostsFound And Not pageRepeatSet And Not newPostsFound Then pageRepeatSet = True : _PageVideosRepeat += 1
+                            If prevPostsFound And newPostsFound And pageRepeatSet Then _PageVideosRepeat -= 1
                             If l.Count > 0 Then _TempMediaList.ListAddList(l.Select(Function(uv) uv.ToUserMedia(specFolder)))
-                            LastPageIDs.Clear()
-                            If newLastPageIDs.Count > 0 Then LastPageIDs.AddRange(newLastPageIDs) : newLastPageIDs.Clear()
-                            If l.Count > 0 AndAlso (l.Count = lBefore Or Not nonLastPageDetected) AndAlso
-                               Not (limit > 0 And _TempMediaList.Count >= limit) Then tryNextPage = True
+                            SessionPosts.ListAddList(newLastPageIDs, LNC)
+                            newLastPageIDs.Clear()
+
+                            If limit > 0 And _TempMediaList.Count >= limit Then Exit Sub
+                            If _PageVideosRepeat < 2 And
+                               ((Not IsUser And prevPostsFound And Not newPostsFound And Page < 1000) Or
+                               (Not cBefore = _TempMediaList.Count And (IsUser Or Page < 1000))) Then tryNextPage = True
+
+                            l.Clear()
                         End If
                     End If
                 End If
@@ -419,6 +460,15 @@ Namespace API.PornHub
                 ProgressPre.Perform()
             End Try
         End Sub
+        Private Sub GetPlaylistToken(ByVal Token As CancellationToken)
+            Dim URL$ = GetNonUserUrl(0)
+            Try
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then PlaylistToken = RegexReplace(r, RegexDataToken)
+            Catch ex As Exception
+                ProcessException(ex, Token, $"token getting error [{URL}]")
+            End Try
+        End Sub
 #End Region
 #Region "Download GIF"
         Private Sub DownloadUserGifs(ByVal Token As CancellationToken)
@@ -931,7 +981,7 @@ Namespace API.PornHub
 #End Region
 #Region "IDisposable Support"
         Protected Overrides Sub Dispose(ByVal disposing As Boolean)
-            If Not disposedValue And disposing Then LastPageIDs.Clear()
+            If Not disposedValue And disposing Then SessionPosts.Clear()
             MyBase.Dispose(disposing)
         End Sub
 #End Region

SCrawler/API/Reddit/Channel.vb

@@ -18,7 +18,7 @@ Imports Period = SCrawler.API.Reddit.IRedditView.Period
 Namespace API.Reddit
     Friend Class Channel : Implements ICollection(Of UserPost), IEquatable(Of Channel), IComparable(Of Channel),
             IRangeSwitcherContainer(Of UserPost), ILoaderSaver, IMyEnumerator(Of UserPost), IChannelLimits, IRedditView, IDisposable
-#Region "XML Nodes' Names"
+#Region "XML Names"
         Private Const Name_Name As String = "Name"
         Private Const Name_ID As String = "ID"
         Private Const Name_Date As String = "Date"
@@ -88,10 +88,14 @@ Namespace API.Reddit
         End Property
         Friend Property ViewMode As View = View.New Implements IRedditView.ViewMode
         Friend Property ViewPeriod As Period = Period.All Implements IRedditView.ViewPeriod
+        Friend Property RedGifsAccount As String = String.Empty Implements IRedditView.RedGifsAccount
+        Friend Property RedditAccount As String = String.Empty Implements IRedditView.RedditAccount
         Friend Sub SetView(ByVal Options As IRedditView) Implements IRedditView.SetView
             If Not Options Is Nothing Then
                 ViewMode = Options.ViewMode
                 ViewPeriod = Options.ViewPeriod
+                RedditAccount = Options.RedditAccount
+                RedGifsAccount = Options.RedGifsAccount
             End If
         End Sub
 #Region "Statistics support"
@@ -215,7 +219,17 @@ Namespace API.Reddit
         End Sub
         Friend Property AutoGetLimits As Boolean = True Implements IChannelLimits.AutoGetLimits
 #End Region
+        Private _HOST As SettingsHost
         Friend ReadOnly Property HOST As SettingsHost
+            Get
+                _HOST = Settings(RedditSiteKey, RedditAccount)
+                If _HOST Is Nothing Then
+                    MyMainLOG = $"Reddit account '{RedditAccount}' for channel '{Name}' not found in the accounts. The default account will be used."
+                    _HOST = Settings(RedditSiteKey).Default
+                End If
+                Return _HOST
+            End Get
+        End Property
         Friend Sub New()
             Posts = New List(Of UserPost)
             PostsLatest = New List(Of UserPost)
@@ -223,7 +237,6 @@ Namespace API.Reddit
             CountOfAddedUsers = New List(Of Integer)
             CountOfLoadedPostsPerSession = New List(Of Integer)
             ChannelExistentUserNames = New List(Of String)
-            HOST = Settings(RedditSiteKey)
         End Sub
         Friend Sub New(ByVal f As SFile)
             Me.New
@@ -350,6 +363,8 @@ Namespace API.Reddit
                         ID = x.Value(Name_ID)
                         ViewMode = x.Value(Name_ViewMode).FromXML(Of Integer)(CInt(View.[New]))
                         ViewPeriod = x.Value(Name_ViewPeriod).FromXML(Of Integer)(CInt(Period.All))
+                        RedGifsAccount = x.Value(Name_RedGifsAccount)
+                        RedditAccount = x.Value(Name_RedditAccount)
                         If FilePosts.Exists Then PostsNames.ListAddList(FilePosts.GetText.StringToList(Of String)("|"), LNC)
                         LatestParsedDate = AConvert(Of Date)(x.Value(Name_Date), DateTimeDefaultProvider, Nothing)
                         CountOfAddedUsers.ListAddList(x.Value(Name_UsersAdded).StringToList(Of Integer)("|"), lc)
@@ -388,6 +403,8 @@ Namespace API.Reddit
                     x.Add(Name_UsersAdded, CountOfAddedUsers.ListToString("|"))
                     x.Add(Name_PostsDownloaded, CountOfLoadedPostsPerSession.ListToString("|"))
                     x.Add(Name_UsersExistent, ChannelExistentUserNames.ListToString("|"))
+                    x.Add(Name_RedGifsAccount, RedGifsAccount)
+                    x.Add(Name_RedditAccount, RedditAccount)
                     If Posts.Count > 0 Or PostsLatest.Count > 0 Then
                         Dim tmpPostList As List(Of UserPost) = Nothing
                         tmpPostList.ListAddList(Posts).ListAddList(PostsLatest)

SCrawler/API/Reddit/IRedditView.vb

@@ -23,17 +23,25 @@ Namespace API.Reddit
         End Enum
         Property ViewMode As View
         Property ViewPeriod As Period
+        Property RedGifsAccount As String
+        Property RedditAccount As String
         Sub SetView(ByVal Options As IRedditView)
     End Interface
     Friend Class RedditViewExchange : Implements IRedditView
         Friend Const Name_ViewMode As String = "ViewMode"
         Friend Const Name_ViewPeriod As String = "ViewPeriod"
+        Friend Const Name_RedGifsAccount As String = "RedGifsAccount"
+        Friend Const Name_RedditAccount As String = "RedditAccount"
         Friend Property ViewMode As IRedditView.View Implements IRedditView.ViewMode
         Friend Property ViewPeriod As IRedditView.Period Implements IRedditView.ViewPeriod
+        Friend Property RedGifsAccount As String Implements IRedditView.RedGifsAccount
+        Friend Property RedditAccount As String Implements IRedditView.RedditAccount
         Friend Sub SetView(ByVal Options As IRedditView) Implements IRedditView.SetView
             If Not Options Is Nothing Then
                 ViewMode = Options.ViewMode
                 ViewPeriod = Options.ViewPeriod
+                RedGifsAccount = Options.RedGifsAccount
+                RedditAccount = Options.RedditAccount
             End If
         End Sub
     End Class