2025.2.25.0

PluginProvider
IPluginContentProvider: add 'NameTrue' property

YT
YouTubeSettings: remove the 'CreateDescriptionFiles_AddUploadDate' property
PlayListParserForm: add 'RDAMP' as default value when initializing form
YouTubeMediaContainerBase: fix line breaks

SCrawler
API.Base: add 'EditorExchangeOptionsBase' class
API.Base.GDL: move functions to Pinterest.UserData
API.Base.IUserData: add 'NameTrue' property
API.Base.SiteSettingsBase: update the 'GetUserUrl' function to use the 'NameTrue' property; update 'UserOptions' function
API.Base.UserDataBase: add 'NameTrue' property; add 'SimpleDownloadAvatar' function to get rid of the  same functions of other classes; Update 'UserDescriptionUpdate' function
API.Base.InternalSettingsForm: calculate max offset

ADD API.Bluesky

API.Facebook: add 'Reels' downloads; fix video downloading
API.Instagram: add inheritance 'EditorExchangeOptionsBase'; remove 'GetUserUrl' function from 'SiteSettings'; update 'UserData' class to new environment; add saving 'heic' file along with 'jpg'
API.LPSG.UserData: simplify 403 error
API.Mastodon: update classes to new environment
API.OnlyFans: add 'AppTokenDefault'; disable cookies update; update 'UserData' class to new environment
API.Pinterest: add sub-boards downloading; update download functions
API.PornHub: fix photo & video downloading; remove 'ModelHub' support
API.Reddit: fix token update; update 'UserData' class to new environment
API.ThisVid: update 'UserData' class to new environment
API.ThreadsNet: fix data download; update classes to new environment; fix 'UserID' extraction; add the ability to manually change the UserName
API.TikTok: update classes to new environment
API.Twitter: update classes to new environment; add sleep timers to fully download large profiles; add 'CookiesUpdate' hidden property
API.Xhamster: update classes to new environment
API.XVIDEOS: update classes to new environment

Feed: add the ability to invert selection; open post URL when double-clicking on subscription image
FeedSpecialCollection: update 'FeedsComparer'
GlobalSettingsForm: remove 'UserAgent' from the 'Basis' tab
UserDataHost: update class to new environment
SettingsCLS: set the 'UserSiteNameAsFriendly' property to 'True' by default; disable 'DownDetector'; add 'UsersListProtected' property

Changelog.md

@@ -1,3 +1,35 @@
+# 2025.2.25.0
+
+*2025-02-25*
+
+- Added
+  - Sites:
+    - **Bluesky**
+    - Facebook: **`Reels` downloads**
+    - OnlyFans: default value for `App-Token`
+    - Pinterest: **sub-boards downloading**
+    - Threads: ability to manually change `UserName`
+    - Twitter:
+      - new icon support
+      - **sleep timers to fully download large profiles**
+  - Feed:
+    - ability to invert selection
+    - open post URL when double-clicking on subscription image
+  - Minor improvements
+- Updated
+  - yt-dlp up to version **2025.02.19**
+  - gallery-dl up to version **1.28.5**
+- PluginProvider
+  - `IPluginContentProvider`: added property `NameTrue`
+- Fixed
+  - Sites:
+    - Facebook: videos are not downloading
+    - LPSG: simplified 403 error
+    - PornHub: photos & videos are not downloading
+    - Reddit: **token does not update automatically**
+    - Threads: **data is not downloading**
+  - Minor bugs
+
 # 2025.1.12.0
 
 *2025-01-12*

FAQ.md

@@ -63,6 +63,7 @@ I strongly recommend you to **regularly** create backup copies of the settings f
 
 - Reddit: don't use credentials at all or configure [OAuth](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-get-reddit-credentials). **Reddit profiles can be downloaded without any credentials at all. Subreddits require OAuth! If nothing downloads, use OAuth!** Don't use OAuth token to download saved posts (use cookies only).
 - **META** (**Instagram**, Threads, Facebook): you need **cookies** and fill in **all fields**
+- **Instagram [TIPS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-tips)**
 - **Instagram saved posts**: I don't consider questions like "I have 10k saved posts and only 1000 were downloaded". Download posts, remove them from saved posts, delete the `Saved posts` **settings folder**, repeat.
 - TikTok: works via yt-dlp. If something doesn't download, we need to wait until yt-dlp fixes it. TikTok doesn't require cookies to download.
 - Porn sites: **COOKIES**!

README.md

@@ -38,6 +38,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
   - Reddit images, galleries of images, videos, saved posts;
   - Redgifs images and videos (https://www.redgifs.com/);
   - Twitter images and videos, saved (bookmarked) posts, likes, communities;
+  - Bluesky images and videos;
   - OnlyFans images and videos, saved (bookmarked) posts, stories;
   - JustForFans images and videos, saved (bookmarked) posts;
   - Mastodon images and videos, saved (bookmarked) posts;
@@ -78,6 +79,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **YouTube Music**
 - **Reddit**
 - **Twitter**
+- **Bluesky**
 - **OnlyFans** *(partial support)*[^1]
 - **Instagram**
 - **Threads**
@@ -131,6 +133,7 @@ First, the program downloads the full profile. After the program downloads only
   - **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
   - [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
   - [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
+  - [Bluesky](https://github.com/AAndyProgram/SCrawler/wiki/Settings#bluesky)
   - [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
   - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
   - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
@@ -215,16 +218,4 @@ F5-->[*]
 
 Discord server: https://discord.gg/uFNUXvFFmg
 
-<!--
-[e-mail](mailto:andyprogram@proton.me): andyprogram@proton.me
-
-Matrix (Element): https://matrix.to/#/@andyprogram:matrix.org
-
-Discord (contact the developer): andyprogram
-
-Discord server: https://discord.gg/uFNUXvFFmg
-
-[Wire](https://account.wire.com/user-profile/?id=93985052-cf2c-4b72-ac75-bbe3231cf544): @andyprogram
--->
-
 [^1]: Partial support means that I don't have personal accounts on paid porn sites because I don't pay for porn. If this site has stopped downloading and you want me to fix it, please be ready to give me access to an account with at least one active subscription. Otherwise, the download from this site will not be fixed.

SCrawler.PluginProvider/Interfaces/IPluginContentProvider.vb

@@ -17,6 +17,7 @@ Namespace Plugin
         Property Settings As ISiteSettings
         Property AccountName As String
         Property Name As String
+        Property NameTrue As String
         Property ID As String
         Property Options As String
         Property ParseUserMediaOnly As Boolean

SCrawler.PluginProvider/My Project/AssemblyInfo.vb

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
 <Assembly: AssemblyDescription("Plugin provider for SCrawler")>
 <Assembly: AssemblyCompany("AndyProgram")>
 <Assembly: AssemblyProduct("SCrawler.PluginProvider")>
-<Assembly: AssemblyCopyright("Copyright © 2024")>
+<Assembly: AssemblyCopyright("Copyright © 2025")>
 <Assembly: AssemblyTrademark("AndyProgram")>
 
 <Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2024.5.18.0")>
+<Assembly: AssemblyVersion("2025.2.25.0")>
-<Assembly: AssemblyFileVersion("2024.5.18.0")>
+<Assembly: AssemblyFileVersion("2025.2.25.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Base/YouTubeSettings.vb

@@ -168,9 +168,6 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible, XMLVN({"Info"}), Category("Info"), DisplayName("Create description files"),
             Description("Create video description files. Default: false.")>
         Public ReadOnly Property CreateDescriptionFiles As XMLValue(Of Boolean)
-        <Browsable(True), GridVisible, XMLVN({"Info"}, True), Category("Info"), DisplayName("Create description files: add upload date"),
-            Description("Add the upload date to the top of the description file. Default: true.")>
-        Public ReadOnly Property CreateDescriptionFiles_AddUploadDate As XMLValue(Of Boolean)
         <Browsable(True), GridVisible, XMLVN({"Info"}, True), Category("Info"), DisplayName("Create description files: create without description"),
             Description("Create a description file with the upload date, even if the description does not exist. Default: true.")>
         Public ReadOnly Property CreateDescriptionFiles_CreateWithNoDescription As XMLValue(Of Boolean)

SCrawler.YouTube/Controls/PlayListParserForm.Designer.vb

@@ -121,9 +121,10 @@ Namespace API.YouTube.Controls
             Me.TXT_LIMIT.Location = New System.Drawing.Point(3, 3)
             Me.TXT_LIMIT.Name = "TXT_LIMIT"
             Me.TXT_LIMIT.PlaceholderEnabled = True
-            Me.TXT_LIMIT.PlaceholderText = "e.g. ABCDE"
+            Me.TXT_LIMIT.PlaceholderText = "e.g. RDAMP"
             Me.TXT_LIMIT.Size = New System.Drawing.Size(378, 22)
             Me.TXT_LIMIT.TabIndex = 2
+            Me.TXT_LIMIT.Text = "RDAMP"
             '
             'CONTAINER_MAIN
             '

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2025.1.12.0")>
+<Assembly: AssemblyVersion("2025.2.25.0")>
-<Assembly: AssemblyFileVersion("2025.1.12.0")>
+<Assembly: AssemblyFileVersion("2025.2.25.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Objects/YouTubeMediaContainerBase.vb

@@ -1215,12 +1215,12 @@ Namespace API.YouTube.Objects
                             End If
 
                             With MyYouTubeSettings
-                                If .CreateDescriptionFiles And (Not Description.IsEmptyString Or
+                                If .CreateDescriptionFiles And (Not Description.IsEmptyString Or .CreateDescriptionFiles_CreateWithNoDescription) Then
-                                                                (.CreateDescriptionFiles_CreateWithNoDescription And .CreateDescriptionFiles_AddUploadDate)) Then
                                     Dim fileDesr As SFile = File
                                     fileDesr.Extension = "txt"
                                     Using fileDesrText As New TextSaver(fileDesr)
-                                        If .CreateDescriptionFiles_AddUploadDate Then fileDesrText.Append($"Uploaded: {DateAdded:yyyy-MM-dd HH:mm:ss}")
+                                        fileDesrText.Append($"Uploaded: {DateAdded:yyyy-MM-dd HH:mm:ss}")
+                                        fileDesrText.AppendLine()
                                         fileDesrText.AppendLine($"URL: {URL}")
                                         fileDesrText.AppendLine($"Channel name: {AccountName}")
                                         fileDesrText.AppendLine($"Channel ID: {UserID}")

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2025.1.12.0")>
+<Assembly: AssemblyVersion("2025.2.25.0")>
-<Assembly: AssemblyFileVersion("2025.1.12.0")>
+<Assembly: AssemblyFileVersion("2025.2.25.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/API/Base/EditorExchangeOptionsBase.vb

@@ -0,0 +1,22 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.Plugin.Attributes
+Imports DN = SCrawler.API.Base.DeclaredNames
+Namespace API.Base
+    Friend Class EditorExchangeOptionsBase
+        Friend Overridable Property SiteKey As String
+        <PSetting(Address:=SettingAddress.User, Caption:=DN.UserNameChangeCaption, ToolTip:=DN.UserNameChangeToolTip)>
+        Friend Overridable Property UserName As String = String.Empty
+        Friend Sub New(ByVal u As UserDataBase)
+            UserName = u.NameTrue(True)
+        End Sub
+        Friend Sub New()
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Base/GDL.vb

@@ -6,66 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
-Imports PersonalUtilities.Tools
-Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Base.GDL
-    Friend Module Declarations
-        Private Structure GDLURL : Implements IRegExCreator
-            Private _URL As String
-            Friend ReadOnly Property URL As String
-                Get
-                    Return _URL
-                End Get
-            End Property
-            Public Shared Widening Operator CType(ByVal u As String) As GDLURL
-                Return New GDLURL With {._URL = u}
-            End Operator
-            Public Shared Widening Operator CType(ByVal u As GDLURL) As String
-                Return u.URL
-            End Operator
-            Private Function CreateFromArray(ByVal ParamsArray() As String) As Object Implements IRegExCreator.CreateFromArray
-                If ParamsArray.ListExists(2) Then
-                    Dim u$ = ParamsArray(0).StringTrim.StringTrimEnd("/"), u2$
-                    If Not u.IsEmptyString Then
-                        u2 = ParamsArray(1).StringTrim
-                        If Not u2.IsEmptyString AndAlso u2.StartsWith("GET", StringComparison.OrdinalIgnoreCase) Then
-                            u2 = u2.Remove(0, 3).StringTrim.StringTrimStart("/")
-                            If Not u2.IsEmptyString Then _URL = $"{u}/{u2}"
-                        End If
-                    End If
-                End If
-                Return Me
-            End Function
-            Public Shared Operator =(ByVal x As GDLURL, ByVal y As GDLURL) As Boolean
-                Return x.URL = y.URL
-            End Operator
-            Public Shared Operator <>(ByVal x As GDLURL, ByVal y As GDLURL) As Boolean
-                Return Not x.URL = y.URL
-            End Operator
-            Public Overrides Function ToString() As String
-                Return URL
-            End Function
-            Public Overrides Function Equals(ByVal Obj As Object) As Boolean
-                Return URL = CType(Obj, String)
-            End Function
-        End Structure
-        Private ReadOnly Property GdlUrlPattern As RParams = RParams.DM(GDLBatch.UrlLibStart.Replace("[", "\[").Replace("]", "\]") &
-                                                                        "([^""]+?)""(GET [^""]+)""", 0, EDP.ReturnValue)
-        Friend Function GetUrlsFromGalleryDl(ByVal Batch As BatchExecutor, ByVal Command As String) As List(Of String)
-            Dim urls As New List(Of String)
-            Dim u As GDLURL
-            With Batch
-                .Execute(Command)
-                If .ErrorOutputData.Count > 0 Then
-                    For Each eValue$ In .ErrorOutputData
-                        u = RegexFields(Of GDLURL)(eValue, {GdlUrlPattern}, {1, 2}, EDP.ReturnValue).ListIfNothing.FirstOrDefault
-                        If Not u.URL.IsEmptyString Then urls.ListAddValue(u, LNC)
-                    Next
-                End If
-            End With
-            Return urls
-        End Function
-    End Module
     Friend Class GDLBatch : Inherits TokenBatch
         Friend Const UrlLibStart As String = "[urllib3.connectionpool][debug]"
         Friend Const UrlTextStart As String = UrlLibStart & " https"

SCrawler/API/Base/IUserData.vb

@@ -18,6 +18,7 @@ Namespace API.Base
         End Enum
         ReadOnly Property Site As String
         ReadOnly Property Name As String
+        Property NameTrue As String
         Property ID As String
         Property Options As String
         Property FriendlyName As String

SCrawler/API/Base/SiteSettingsBase.vb

@@ -33,7 +33,7 @@ Namespace API.Base
         End Property
         Friend Property AccountName As String Implements ISiteSettings.AccountName
         Friend Property Temporary As Boolean = False Implements ISiteSettings.Temporary
-        Friend Property DefaultInstance As ISiteSettings = Nothing Implements ISiteSettings.DefaultInstance
+        Friend Overridable Property DefaultInstance As ISiteSettings = Nothing Implements ISiteSettings.DefaultInstance
         Protected _UserAgentDefault As String = String.Empty
         Friend Overridable Property UserAgentDefault As String Implements ISiteSettings.UserAgentDefault
             Get
@@ -55,6 +55,11 @@ Namespace API.Base
         Friend Overridable ReadOnly Property Responser As Responser
         Private _UserOptionsExists As Boolean = False
         Private _UserOptionsType As Type = Nothing
+        Protected Overridable Function UserOptionsValid(ByVal Options As Object) As Boolean
+            Return True
+        End Function
+        Protected Overridable Sub UserOptionsSetParameters(ByRef Options As Object)
+        End Sub
         Protected Property UserOptionsType As Type
             Get
                 Return _UserOptionsType
@@ -243,7 +248,7 @@ Namespace API.Base
 #Region "User info"
         Protected UrlPatternUser As String = String.Empty
         Friend Overridable Function GetUserUrl(ByVal User As IPluginContentProvider) As String Implements ISiteSettings.GetUserUrl
-            If Not UrlPatternUser.IsEmptyString Then Return String.Format(UrlPatternUser, User.Name)
+            If Not UrlPatternUser.IsEmptyString Then Return String.Format(UrlPatternUser, User.NameTrue.IfNullOrEmpty(User.Name))
             Return String.Empty
         End Function
         Private Function ISiteSettings_GetUserPostUrl(ByVal User As IPluginContentProvider, ByVal Media As IUserMedia) As String Implements ISiteSettings.GetUserPostUrl
@@ -380,11 +385,40 @@ Namespace API.Base
         End Sub
         Friend Overridable Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
             If _UserOptionsExists Then
-                If Options Is Nothing OrElse Not Options.GetType Is _UserOptionsType Then
+                If Options Is Nothing OrElse (Not Options.GetType Is _UserOptionsType OrElse Not UserOptionsValid(Options)) Then
-                    Options = AConvert(Me, AModes.Var, _UserOptionsType,, True, Nothing)
+                    Dim args% = 0
+                    Dim constructor As ConstructorInfo = Nothing
+                    With _UserOptionsType.GetTypeInfo.DeclaredConstructors
+                        If .ListExists Then
+                            With .Where(Function(ByVal c As ConstructorInfo) As Boolean
+                                            With c.GetParameters
+                                                If .ListExists Then
+                                                    If .Count = 1 Then
+                                                        Return .Self()(0).ParameterType.GetInterfaces.ListIfNothing.Where(Function(i) i Is Me.GetType).Count = 1
+                                                    Else
+                                                        Return False
+                                                    End If
+                                                Else
+                                                    Return True
+                                                End If
+                                            End With
+                                            Return If(c.GetParameters?.Count, 0).ValueBetween(0, 1)
+                                        End Function)
+                                If .ListExists Then
+                                    args = .Max(Of Integer)(Function(c) If(c.GetParameters?.Count, 0))
+                                    constructor = .First(Function(c) If(c.GetParameters?.Count, 0) = args)
+                                End If
+                            End With
+                        End If
+                    End With
+                    If Not constructor Is Nothing Then
+                        If args > 0 AndAlso Not constructor.GetParameters()(0).ParameterType Is GetType(ISiteSettings) Then Throw New Exception
+                        If args = 0 Then Options = constructor.Invoke(Nothing) Else Options = constructor.Invoke({Me})
+                    End If
                     If Options Is Nothing Then Options = Activator.CreateInstance(_UserOptionsType)
+                    If Not Options Is Nothing Then UserOptionsSetParameters(Options)
                 End If
-                If OpenForm Then
+                If Not Options Is Nothing And OpenForm Then
                     Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
                 End If
             Else

SCrawler/API/Base/UserDataBase.vb

@@ -178,6 +178,8 @@ Namespace API.Base
 #Region "Additional names"
         Protected Const Name_SiteMode As String = "SiteMode"
         Protected Const Name_TrueName As String = "TrueName"
+        'TODELETE Name_TrueName2
+        <Obsolete> Protected Const Name_TrueName2 As String = "NameTrue"
         Protected Const Name_Arguments As String = "Arguments"
 #End Region
 #End Region
@@ -278,6 +280,21 @@ Namespace API.Base
                 Return User.Name
             End Get
         End Property
+        Private _NameTrue As String = String.Empty
+        Friend Overridable Overloads Property NameTrue As String Implements IUserData.NameTrue, IPluginContentProvider.NameTrue
+            Get
+                Return NameTrue(False)
+            End Get
+            Set(ByVal NewName As String)
+                If Not _NameTrue = NewName Then EnvirChanged(NewName)
+                _NameTrue = NewName
+            End Set
+        End Property
+        Friend Overloads ReadOnly Property NameTrue(ByVal Exact As Boolean) As String
+            Get
+                Return If(Exact, _NameTrue, _NameTrue.IfNullOrEmpty(Name))
+            End Get
+        End Property
         Friend Overridable Property ID As String = String.Empty Implements IUserData.ID, IPluginContentProvider.ID
         Protected _FriendlyName As String = String.Empty
         Friend Overridable Property FriendlyName As String Implements IUserData.FriendlyName
@@ -348,12 +365,20 @@ Namespace API.Base
         Protected Function UserDescriptionNeedToUpdate() As Boolean
             Return (UserDescription.IsEmptyString Or _DescriptionEveryTime) And Not _DescriptionChecked
         End Function
-        Protected Sub UserDescriptionUpdate(ByVal Descr As String)
+        Protected Sub UserDescriptionUpdate(ByVal Descr As String, Optional ByVal Force As Boolean = False,
-            If UserDescriptionNeedToUpdate() Then
+                                            Optional ByVal InsertFirst As Boolean = False, Optional ByVal AppendDate As Boolean = False)
+            If UserDescriptionNeedToUpdate() Or Force Then
+                If AppendDate Then Descr = $"{Now.ToStringDateDef}: {Descr}"
                 If UserDescription.IsEmptyString Then
                     UserDescription = Descr
+                    _ForceSaveUserInfo = True
                 ElseIf Not UserDescription.Contains(Descr) Then
-                    UserDescription &= $"{vbNewLine}----{vbNewLine}{Descr}"
+                    If InsertFirst Then
+                        UserDescription = $"{Descr}{vbNewLine}{UserDescription}"
+                    Else
+                        UserDescription &= $"{vbNewLine}----{vbNewLine}{Descr}"
+                    End If
+                    _ForceSaveUserInfo = True
                 End If
                 _DescriptionChecked = True
             End If
@@ -907,6 +932,10 @@ BlockNullPicture:
                     FileExists = True
                     Using x As New XmlFile(MyFileSettings) With {.XmlReadOnly = True}
                         If User.Name.IsEmptyString Then User.Name = x.Value(Name_UserName)
+                        _NameTrue = x.Value(Name_TrueName)
+#Disable Warning BC40008
+                        If _NameTrue.IsEmptyString AndAlso x.Contains(Name_TrueName2) Then _NameTrue = x.Value(Name_TrueName2)
+#Enable Warning
                         UserExists = x.Value(Name_UserExists).FromXML(Of Boolean)(True)
                         UserSuspended = x.Value(Name_UserSuspended).FromXML(Of Boolean)(False)
                         ID = x.Value(Name_UserID)
@@ -967,6 +996,7 @@ BlockNullPicture:
                     x.Add(Name_Plugin, HOST.Key)
                     x.Add(Name_AccountName, AccountName)
                     x.Add(Name_UserName, User.Name)
+                    x.Add(Name_TrueName, _NameTrue)
                     x.Add(Name_Model_User, CInt(UserModel))
                     x.Add(Name_Model_Collection, CInt(CollectionModel))
                     x.Add(Name_SpecialPath, User.SpecialPath)
@@ -1162,6 +1192,7 @@ BlockNullPicture:
                 Select Case Caller
                     Case NameOf(UserExists) : If Not _EnvirUserExists = CBool(NewValue) Then _EnvirChanged = True : _EnvirInvokeUserUpdated = True
                     Case NameOf(UserSuspended) : If Not _EnvirUserSuspended = CBool(NewValue) Then _EnvirChanged = True : _EnvirInvokeUserUpdated = True
+                    Case NameOf(NameTrue) : _EnvirChanged = True : _EnvirInvokeUserUpdated = True : _ForceSaveUserInfo = True : _ForceSaveUserInfoOnException = True
                     Case Else : _EnvirChanged = True
                 End Select
             End If
@@ -1282,9 +1313,9 @@ BlockNullPicture:
                 UpdateUserInformation_Ex()
                 If Not exit_ex.Silent Then
                     If exit_ex.SimpleLogLine Then
-                        MyMainLOG = $"{ToStringForLog()}: downloading interrupted (exit) ({exit_ex.Message})"
+                        LogError(Nothing, $"downloading interrupted (exit) ({exit_ex.Message})")
                     Else
-                        ErrorsDescriber.Execute(EDP.SendToLog, exit_ex, $"{ToStringForLog()}: downloading interrupted (exit)")
+                        LogError(exit_ex, "downloading interrupted (exit)")
                     End If
                 End If
                 If _EnvirInvokeUserUpdated Then OnUserUpdated()
@@ -1846,6 +1877,31 @@ BlockNullPicture:
         Protected Overridable Function CreateFileFromUrl(ByVal URL As String) As SFile
             Return New SFile(URL)
         End Function
+        Protected Overridable Function SimpleDownloadAvatar(ByVal ImageAddress As String, Optional ByVal FileCreateFunc As Func(Of String, SFile) = Nothing,
+                                                            Optional ByVal e As ErrorsDescriber = Nothing) As SFile
+            Try
+                If Not ImageAddress.IsEmptyString Then
+                    Dim f As SFile
+                    If FileCreateFunc Is Nothing Then
+                        f = CreateFileFromUrl(ImageAddress)
+                    Else
+                        f = FileCreateFunc.Invoke(ImageAddress)
+                    End If
+                    If Not f.Name.IsEmptyString Then f.Name = f.Name.StringRemoveWinForbiddenSymbols.StringTrim
+                    If Not f.Name.IsEmptyString Then
+                        f.Path = DownloadContentDefault_GetRootDir()
+                        f.Separator = "\"
+                        If f.Extension.IsEmptyString Then f.Extension = "jpg"
+                        If Not f.Exists Then GetWebFile(ImageAddress, f, EDP.ReturnValue)
+                        If f.Exists Then IconBannerDownloaded = True : Return f
+                    End If
+                End If
+                Return Nothing
+            Catch ex As Exception
+                If Not e.Exists Then e = New ErrorsDescriber(EDP.ReturnValue)
+                Return ErrorsDescriber.Execute(e, ex, $"SimpleDownloadAvatar({ImageAddress})", New SFile)
+            End Try
+        End Function
         Protected Overridable Function ChangeFileNameByProvider(ByVal f As SFile, ByVal m As UserMedia) As SFile
             Dim ff As SFile = Nothing
             Try

SCrawler/API/BaseObjects/InternalSettingsForm.vb

@@ -134,6 +134,7 @@ Namespace API.Base
                                                                                          m.GetMemberCustomAttributes(Of Provider).ListExists
                     Dim m1 As MemberInfo, m2 As MemberInfo
                     Dim tmpObj As Object
+                    Dim maxOffset%
 
                     members = GetObjectMembers(MyObject, Function(m) (m.MemberType = MemberTypes.Field Or m.MemberType = MemberTypes.Property) AndAlso
                                                                      Not m.GetCustomAttribute(Of PSettingAttribute) Is Nothing,, True,
@@ -175,6 +176,9 @@ Namespace API.Base
 
                     If MyMembers.Count > 0 Then
 
+                        maxOffset = MyMembers.Max(Function(mm) mm.LeftOffset)
+                        If maxOffset > 0 Then MyMembers.ForEach(Sub(mm) mm.LeftOffset = maxOffset)
+
                         Dim prov As IEnumerable(Of Provider)
                         Dim _prov As Provider
                         Dim si% = -1

SCrawler/API/Bluesky/Declarations.vb

@@ -0,0 +1,18 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.Bluesky
+    Friend Module Declarations
+        Friend Const BlueskySiteKey As String = "AndyProgram_Bluesky"
+        Friend ReadOnly DateProvider As New ADateTime("yyyy-MM-ddTHH:mm:ss.FFF%K")
+        Friend ReadOnly RegEx_PlayLists As RParams = RParams.DM("RESOLUTION=\d+x(\d+)\s*(\S+)", 0, RegexReturn.List, EDP.ReturnValue)
+        Friend ReadOnly RegEx_FilePattern As RParams = RParams.DM("(.+?)(\.|@)(gif|m3u8|[^/\?\&]+)", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
+        Friend ReadOnly RegEx_SinglePostPattern As RParams = RParams.DM("profile/([^/]+)/post/([^/\?\&]+)", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
+    End Module
+End Namespace

SCrawler/API/Bluesky/M3U8.vb

@@ -0,0 +1,51 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Threading
+Imports SCrawler.API.Base
+Imports PersonalUtilities.Forms.Toolbars
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.Bluesky
+    Friend NotInheritable Class M3U8
+        Private Sub New()
+        End Sub
+        Private Shared Function GetUrlsList(ByVal URL As String) As List(Of String)
+            Using resp As New Responser With {.AllowAutoRedirect = False}
+                Dim r$ = resp.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    Dim file$ = String.Empty, appender$
+                    Dim files As List(Of Sizes) = RegexFields(Of Sizes)(r, {RegEx_PlayLists}, {1, 2})
+                    If files.ListExists Then files.RemoveAll(Function(ff) ff.Value = 0 Or ff.Data.IsEmptyString)
+                    If files.ListExists Then
+                        files.Sort()
+                        file = files(0).Data
+                        appender = URL.Replace(URL.Split("/").Last, String.Empty)
+                        file = M3U8Base.CreateUrl(appender, file)
+                        If Not file.IsEmptyString Then
+                            r = resp.GetResponse(file)
+                            If Not r.IsEmptyString Then
+                                Dim l As List(Of String) = RegexReplace(r, M3U8Declarations.TsFilesRegEx)
+                                If l.ListExists Then
+                                    appender = file.Replace(file.Split("/").Last, String.Empty)
+                                    For i% = 0 To l.Count - 1 : l(i) = M3U8Base.CreateUrl(appender, l(i)) : Next
+                                    Return l
+                                End If
+                            End If
+                        End If
+                    End If
+                End If
+            End Using
+            Return Nothing
+        End Function
+        Friend Shared Function Download(ByVal URL As String, ByVal Destination As SFile, ByVal Token As CancellationToken,
+                                        ByVal Progress As MyProgress, ByVal UsePreProgress As Boolean) As SFile
+            Return M3U8Base.Download(GetUrlsList(URL), Destination,, Token, Progress, UsePreProgress)
+        End Function
+    End Class
+End Namespace

SCrawler/API/Bluesky/SiteSettings.vb

@@ -0,0 +1,100 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
+Imports SCrawler.Plugin
+Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Namespace API.Bluesky
+    <Manifest(BlueskySiteKey), SpecialForm(False)>
+    Friend Class SiteSettings : Inherits SiteSettingsBase
+        <PropertyOption(ControlText:="Cookies enabled", ControlToolTip:="If checked, cookies will be used in requests", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property CookiesEnabled As PropertyValue
+        <PropertyOption(ControlText:="User name", IsAuth:=True, AllowNull:=False), PXML>
+        Friend ReadOnly Property UserHandle As PropertyValue
+        <PropertyOption(ControlText:="Password", IsAuth:=True, AllowNull:=False), PXML>
+        Friend ReadOnly Property UserPassword As PropertyValue
+        <PXML> Friend ReadOnly Property Token As PropertyValue
+        <PXML> Friend ReadOnly Property TokenUpdateTime As PropertyValue
+        <PropertyOption(ControlText:="Token update", ControlToolTip:="Token refresh interval (in minutes)." & vbCr & "Default: 120.", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property TokenRefreshInterval As PropertyValue
+        Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
+            MyBase.New("Bluesky", "bsky.app", AccName, Temp, My.Resources.SiteResources.BlueskyIcon_32, My.Resources.SiteResources.BlueskyPic_32)
+
+            Responser.ContentType = "application/json"
+
+            CookiesEnabled = New PropertyValue(False)
+            UserHandle = New PropertyValue(String.Empty, GetType(String))
+            UserPassword = New PropertyValue(String.Empty, GetType(String))
+            Token = New PropertyValue(String.Empty, GetType(String))
+            TokenUpdateTime = New PropertyValue(Now.AddYears(-1))
+            TokenRefreshInterval = New PropertyValue(120)
+
+            _AllowUserAgentUpdate = False
+            UrlPatternUser = "https://bsky.app/profile/{0}"
+            ImageVideoContains = "bsky.app"
+            UserRegex = RParams.DMS("bsky.app/profile/([^/\?]+)", 1, EDP.ReturnValue)
+            UserOptionsType = GetType(EditorExchangeOptionsBase)
+        End Sub
+        Protected Overrides Function UserOptionsValid(ByVal Options As Object) As Boolean
+            Return DirectCast(Options, EditorExchangeOptionsBase).SiteKey = BlueskySiteKey
+        End Function
+        Protected Overrides Sub UserOptionsSetParameters(ByRef Options As Object)
+            DirectCast(Options, EditorExchangeOptionsBase).SiteKey = BlueskySiteKey
+        End Sub
+        Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
+            Return New UserData
+        End Function
+        Friend Overrides Function BaseAuthExists() As Boolean
+            Return Not CStr(UserHandle.Value).IsEmptyString And Not CStr(UserPassword.Value).IsEmptyString
+        End Function
+        Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean
+            Return MyBase.Available(What, Silent) AndAlso UpdateToken()
+        End Function
+        Private _TokenUpdating As Boolean = False
+        Friend Function UpdateToken(Optional ByVal Force As Boolean = False) As Boolean
+            Try
+                While _TokenUpdating : Threading.Thread.Sleep(100) : End While
+                _TokenUpdating = True
+                If BaseAuthExists() Then
+                    If CDate(TokenUpdateTime.Value).AddMinutes(TokenRefreshInterval.Value) < Now Or Force Then
+                        Using resp As Responser = Responser.Copy
+                            With resp
+                                .Mode = Responser.Modes.Curl
+                                .Method = "POST"
+                                .CurlSslNoRevoke = True
+                                .CurlInsecure = True
+                                .CurlArgumentsLeft = "-d ""{\" & $"""identifier\"": \""{UserHandle.Value}\"", \""password\"": \""{UserPassword.Value}\""" & "}"""
+
+                                Dim r$ = .GetResponse("https://bsky.social/xrpc/com.atproto.server.createSession")
+                                If Not r.IsEmptyString Then
+                                    Using j As EContainer = JsonDocument.Parse(r, EDP.ReturnValue)
+                                        If j.ListExists Then
+                                            Dim t$ = j.Value("accessJwt")
+                                            If Not t.IsEmptyString Then Token.Value = $"Bearer {t}" : TokenUpdateTime.Value = Now : Return True
+                                        End If
+                                    End Using
+                                End If
+                            End With
+                        End Using
+                    Else
+                        Return True
+                    End If
+                End If
+                Return False
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, "Bluesky.SiteSettings.UpdateToken", False)
+            Finally
+                _TokenUpdating = False
+            End Try
+        End Function
+    End Class
+End Namespace

SCrawler/API/Bluesky/UserData.vb

@@ -0,0 +1,330 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Threading
+Imports SCrawler.API.Base
+Imports SCrawler.API.YouTube.Objects
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports UTypes = SCrawler.API.Base.UserMedia.Types
+Imports UStates = SCrawler.API.Base.UserMedia.States
+Namespace API.Bluesky
+    Friend Class UserData : Inherits UserDataBase
+#Region "Declarations"
+        Private ReadOnly Property MySettings As SiteSettings
+            Get
+                Return HOST.Source
+            End Get
+        End Property
+        Private ReadOnly Property ID_Encoded As String
+            Get
+                Return If(ID.IsEmptyString, String.Empty, SymbolsConverter.ASCII.EncodeSymbolsOnly(ID))
+            End Get
+        End Property
+#End Region
+#Region "Loader"
+        Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+        End Sub
+        Friend Overrides Function ExchangeOptionsGet() As Object
+            Return New EditorExchangeOptionsBase(Me) With {.SiteKey = BlueskySiteKey}
+        End Function
+        Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptionsBase AndAlso
+               DirectCast(Obj, EditorExchangeOptionsBase).SiteKey = BlueskySiteKey Then NameTrue = DirectCast(Obj, EditorExchangeOptionsBase).UserName
+        End Sub
+#End Region
+#Region "Initializer"
+        Friend Sub New()
+            UseInternalM3U8Function = True
+        End Sub
+#End Region
+#Region "Token"
+        Private Function UpdateToken(Optional ByVal Force As Boolean = False, Optional ByVal OnlyAddHeader As Boolean = False) As Boolean
+            Dim process As Boolean = True
+            If CDate(MySettings.TokenUpdateTime.Value).AddHours(2) <= Now Or Force Then
+                process = MySettings.UpdateToken(Force)
+                If process Then _TokenUpdateCount += 1
+            End If
+            If process Or OnlyAddHeader Then Responser.Headers.Add("authorization", MySettings.Token.Value)
+            Return Not Responser.Headers.Value("authorization").IsEmptyString
+        End Function
+        Private _TokenUpdateCount As Integer = 0
+        Private Sub TokenUpdateCountReset()
+            _TokenUpdateCount = 0
+        End Sub
+#End Region
+#Region "Download"
+        Private _PostCount As Integer = 0
+        Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
+            If Not CBool(MySettings.CookiesEnabled.Value) Then Responser.Cookies.Clear()
+            UpdateToken(, True)
+            _TokenUpdateCount = 0
+            _PostCount = 0
+            DownloadData(String.Empty, Token)
+        End Sub
+        Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Try
+                If ID.IsEmptyString Then GetProfileInfo(Token)
+                If ID.IsEmptyString Then Throw New ArgumentNullException("ID", "ID is null")
+                If UpdateToken() Then
+                    Dim nextCursor$ = String.Empty
+                    Dim c%
+                    URL = $"https://bsky.social/xrpc/app.bsky.feed.getAuthorFeed?actor={ID_Encoded}&filter=posts_and_author_threads&includePins=false&limit=99"
+                    If Not Cursor.IsEmptyString Then URL &= $"&cursor={SymbolsConverter.ASCII.EncodeSymbolsOnly(Cursor)}"
+                    Dim r$ = Responser.GetResponse(URL)
+                    TokenUpdateCountReset()
+                    If Not r.IsEmptyString Then
+                        Using j As EContainer = JsonDocument.Parse(r)
+                            If j.ListExists Then
+                                With j("feed")
+                                    If .ListExists Then
+                                        For Each post As EContainer In .Self
+                                            With post({"post"})
+                                                c = DefaultParser(.Self,, nextCursor)
+                                                Select Case c
+                                                    Case CInt(DateResult.Skip) * -1 : Continue For
+                                                    Case CInt(DateResult.Exit) * -1 : Exit Sub
+                                                    Case Is > 0 : _PostCount += c
+                                                End Select
+                                                If DownloadTopCount.HasValue AndAlso DownloadTopCount.Value <= _PostCount Then Exit Sub
+                                            End With
+                                        Next
+                                    End If
+                                End With
+                            End If
+                        End Using
+
+                        If Not nextCursor.IsEmptyString Then DownloadData(nextCursor, Token)
+                    End If
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, $"DownloadData({URL})")
+            End Try
+        End Sub
+#End Region
+#Region "DefaultParser"
+        Private Const Down_ImageAddress As String = "https://cdn.bsky.app/img/feed_fullsize/plain/{0}/{1}"
+        Private Function GetPostID(ByVal PostUri As String) As String
+            Return If(PostUri.IsEmptyString, String.Empty, PostUri.Split("/").LastOrDefault)
+        End Function
+        Private Function DefaultParser(ByVal e As EContainer, Optional ByVal CheckDateLimits As Boolean = True, Optional ByRef NextCursor As String = Nothing,
+                                       Optional ByVal CheckTempPosts As Boolean = True, Optional ByVal State As UStates = UStates.Unknown) As Integer
+            Const exitReturn% = CInt(DateResult.Exit) * -1
+            Dim postID$, postDate$, __url$, __urlBase$
+            Dim updateUrl As Boolean
+            Dim c% = 0
+            Dim m As UserMedia
+            Dim d As EContainer
+            With e
+                If .ListExists Then
+                    postID = GetPostID(.Value("uri"))
+                    postDate = String.Empty
+                    __urlBase = String.Empty
+                    With .Item({"record"})
+                        If .ListExists Then
+                            '2025-01-28T02:42:12.415Z
+                            postDate = .Value("createdAt")
+                            NextCursor = postDate
+                            If CheckDateLimits Then
+                                Select Case CheckDatesLimit(postDate, DateProvider)
+                                    Case DateResult.Skip : Return CInt(DateResult.Skip) * -1 'Continue For
+                                    Case DateResult.Exit : Return exitReturn 'Exit Sub
+                                End Select
+                            End If
+
+                            If CheckTempPosts Then
+                                If _TempPostsList.Contains(postID) Then Return exitReturn Else _TempPostsList.Add(postID)
+                            End If
+                            __urlBase = $"https://bsky.app/profile/{NameTrue}/post/{postID}"
+                        End If
+                    End With
+
+                    Dim createMedia As Func(Of String, UTypes, UserMedia) =
+                        Function(ByVal url As String, ByVal type As UTypes) As UserMedia
+                            m = New UserMedia(url, type) With {
+                                .URL_BASE = __urlBase,
+                                .File = CreateFileFromUrl(url, type),
+                                .Post = New UserPost(postID, If(AConvert(Of Date)(postDate, DateProvider, Nothing, EDP.ReturnValue), Nothing)),
+                                .State = State
+                            }
+                            _TempMediaList.ListAddValue(m, LNC)
+                            c += 1
+                            Return m
+                        End Function
+
+                    For Each SecondExtraction As Boolean In {False, True}
+                        With If(SecondExtraction, .Item({"record", "embed"}), .Item("embed"))
+                            If .ListExists Then
+
+                                If If(.Item("images")?.Count, 0) > 0 Then
+                                    With .Item("images")
+                                        For Each d In .Self
+                                            updateUrl = False
+                                            __url = d.Value("fullsize")
+                                            If __url.IsEmptyString Then __url = d.Value({"image", "ref"}, "$link") : updateUrl = True
+                                            If __url.IsEmptyString And SecondExtraction Then updateUrl = False : __url = e.Value({"embed"}, "thumb")
+                                            If Not __url.IsEmptyString Then createMedia(__url, UTypes.Picture)
+                                        Next
+                                    End With
+                                End If
+
+                                If Not .Value("playlist").IsEmptyString Then createMedia(.Value("playlist"), UTypes.m3u8)
+
+                                If If(.Item("external")?.Count, 0) > 0 Then createMedia(.Value({"external"}, "uri"), UTypes.GIF)
+
+                            End If
+                        End With
+
+                        If c > 0 Then Exit For
+                    Next
+                End If
+            End With
+            Return c
+        End Function
+#End Region
+#Region "GetProfileInfo"
+        Private Sub GetProfileInfo(ByVal Token As CancellationToken)
+            Try
+                If UpdateToken() Then
+                    Dim r$ = Responser.GetResponse($"https://bsky.social/xrpc/app.bsky.actor.getProfile?actor={ID.IfNullOrEmpty(NameTrue)}")
+                    TokenUpdateCountReset()
+                    If Not r.IsEmptyString Then
+                        Using j As EContainer = JsonDocument.Parse(r)
+                            If j.ListExists Then
+                                ID = j.Value("did")
+                                UserSiteNameUpdate(j.Value("displayName"))
+                                UserDescriptionUpdate(j.Value("description"))
+                                NameTrue = j.Value("handle")
+                                SimpleDownloadAvatar(j.Value("avatar"))
+                                SimpleDownloadAvatar(j.Value("banner"))
+                            End If
+                        End Using
+                    End If
+                Else
+                    Throw New ArgumentException("Token is null", "Token")
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, "GetProfileInfo")
+            End Try
+        End Sub
+#End Region
+#Region "ReparseMissing"
+        Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
+            Const uriPattern$ = "at://{0}/app.bsky.feed.post/{1}"
+            Dim rList As New List(Of Integer)
+            Try
+                If ContentMissingExists AndAlso UpdateToken() Then
+                    Dim r$, url$, uri$
+                    Dim tu As Byte
+                    Dim m As UserMedia
+                    Dim j As EContainer
+                    For i% = 0 To _ContentList.Count - 1
+                        m = _ContentList(i)
+                        If m.State = UStates.Missing Then
+                            uri = SymbolsConverter.ASCII.EncodeSymbolsOnly(String.Format(uriPattern, NameTrue, m.Post.ID))
+                            url = $"https://bsky.social/xrpc/app.bsky.feed.getPostThread?uri={uri}&depth=10"
+                            For tu = 0 To 1
+                                Try
+                                    Responser.ResetStatus()
+                                    r = Responser.GetResponse(url)
+                                    TokenUpdateCountReset()
+                                    If Not r.IsEmptyString Then
+                                        j = JsonDocument.Parse(r)
+                                        If j.ListExists Then
+                                            If DefaultParser(j({"thread", "post"}), False,, False, UStates.Missing) > 0 Then rList.Add(i)
+                                            j.Dispose()
+                                        End If
+                                    End If
+                                    Exit For
+                                Catch eex As Exception
+                                    If ProcessException(eex, Token, $"ReparseMissing({url})",,, False) <> 1 Then Throw eex
+                                End Try
+                            Next
+                        End If
+                    Next
+                Else
+                    Throw New ArgumentException("Token is null", "Token")
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, "ReparseMissing error")
+            Finally
+                If rList.Count > 0 Then
+                    For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
+                    rList.Clear()
+                End If
+            End Try
+        End Sub
+#End Region
+#Region "CreateFileFromUrl"
+        Protected Overloads Overrides Function CreateFileFromUrl(ByVal URL As String) As SFile
+            Return CreateFileFromUrl(URL, UTypes.Undefined)
+        End Function
+        Protected Overloads Function CreateFileFromUrl(ByVal URL As String, ByVal Type As UTypes) As SFile
+            Dim f As SFile = MyBase.CreateFileFromUrl(URL)
+            Dim force As Boolean = False
+            f.Separator = "\"
+            With URL.Split("/")
+                If .ListExists Then
+                    With DirectCast(RegexReplace(.Last, RegEx_FilePattern), List(Of String))
+                        If .ListExists(4) Then
+                            f.Name = .Item(1).IfNullOrEmpty(f.Name)
+                            f.Extension = .Item(3)
+                        End If
+                    End With
+                End If
+            End With
+            If Not f.Extension.IsEmptyString AndAlso f.Extension.ToLower = "m3u8" Then force = True : Type = UTypes.m3u8
+            If f.Extension.IsEmptyString Or force Then
+                Select Case Type
+                    Case UTypes.Picture : f.Extension = "jpg"
+                    Case UTypes.GIF : f.Extension = "gif"
+                    Case UTypes.m3u8 : f.Name = "Video" : f.Extension = "mp4"
+                End Select
+            End If
+            Return f
+        End Function
+#End Region
+#Region "DownloadContent"
+        Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
+            DownloadContentDefault(Token)
+        End Sub
+        Protected Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile, ByVal Token As CancellationToken) As SFile
+            Return M3U8.Download(URL, DestinationFile, Token, Progress, Not IsSingleObjectDownload)
+        End Function
+#End Region
+#Region "DownloadSingleObject"
+        Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
+            _TokenUpdateCount = 0
+            UpdateToken()
+            Dim l As List(Of String) = RegexReplace(Data.URL, RegEx_SinglePostPattern)
+            If l.ListExists(3) Then
+                NameTrue = l(1)
+                _ContentList.Add(New UserMedia(Data.URL) With {.State = UStates.Missing, .Post = l(2)})
+                ReparseMissing(Token)
+            End If
+            MyBase.DownloadSingleObject_GetPosts(Data, Token)
+        End Sub
+#End Region
+#Region "Exception"
+        Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
+                                                          Optional ByVal EObj As Object = Nothing) As Integer
+            If Responser.StatusCode = Net.HttpStatusCode.BadRequest Then '400
+                If _TokenUpdateCount = 0 AndAlso UpdateToken(True) Then
+                    Return 1
+                Else
+                    Return 0
+                End If
+            Else
+                Return 0
+            End If
+        End Function
+#End Region
+    End Class
+End Namespace

SCrawler/API/Facebook/Declarations.vb

@@ -18,6 +18,8 @@ Namespace API.Facebook
         Friend ReadOnly Regex_FileName As RParams = RParams.DM("([^/\?]+\..{3,4})(?=(\?|\Z))", 0, EDP.ReturnValue)
         Friend ReadOnly Regex_ProfileUrlID As RParams = RParams.DMS("profile.php\?id=(\d+)", 1, EDP.ReturnValue)
         Friend ReadOnly Regex_VideoPageID As RParams = RParams.DMS("pageid.:.(\d+)", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly Regex_ReelsPageID As RParams = RParams.DMS("\{[^\}]*""tab_key"":""owner_reels"",?[^\}]*""id"":""([^\}""]+)""", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly Regex_ReelsFilePattern As RParams = RParams.DM("[^/]+\.mp4", 0, EDP.ReturnValue)
         Friend ReadOnly Regex_StoryBucket As RParams = RParams.DMS("story_bucket[^\>]*?(\d+)", 1, EDP.ReturnValue)
 
         Friend ReadOnly Regex_VideoIDFromURL As RParams = RParams.DMS("facebook.com/([^/]+/videos/|watch/\D*[\?&]{1}v=)(\d+)", 2, EDP.ReturnValue)

SCrawler/API/Facebook/SiteSettings.vb

@@ -36,6 +36,8 @@ Namespace API.Facebook
         Friend ReadOnly Property ParsePhotoBlock As PropertyValue
         <PropertyOption(ControlText:="Download videos", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property ParseVideoBlock As PropertyValue
+        <PropertyOption(ControlText:="Download reels", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
+        Friend ReadOnly Property ParseReelsBlock As PropertyValue
         <PropertyOption(ControlText:="Download stories", IsAuth:=False, Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property ParseStoriesBlock As PropertyValue
 #End Region
@@ -52,6 +54,7 @@ Namespace API.Facebook
             Header_Accept = New PropertyValue(String.Empty, GetType(String))
             ParsePhotoBlock = New PropertyValue(True)
             ParseVideoBlock = New PropertyValue(True)
+            ParseReelsBlock = New PropertyValue(False)
             ParseStoriesBlock = New PropertyValue(True)
 
             UrlPatternUser = "https://www.facebook.com/{0}"

SCrawler/API/Facebook/UserData.vb

@@ -23,9 +23,11 @@ Namespace API.Facebook
         Private Const Name_IsNoNameProfile As String = "IsNoNameProfile"
         Private Const Name_OptionsParsed As String = "OptionsParsed"
         Private Const Name_VideoPageID As String = "VideoPageID"
+        Private Const Name_ReelsPageID As String = "ReelsPageID"
         Private Const Name_StoryBucket As String = "StoryBucket"
         Private Const Name_ParsePhotoBlock As String = "ParsePhotoBlock"
         Private Const Name_ParseVideoBlock As String = "ParseVideoBlock"
+        Private Const Name_ParseReelsBlock As String = "ParseReelsBlock"
         Private Const Name_ParseStoriesBlock As String = "ParseStoriesBlock"
 #End Region
 #Region "Declarations"
@@ -37,15 +39,18 @@ Namespace API.Facebook
         Private IsNoNameProfile As Boolean = False
         Private OptionsParsed As Boolean = False
         Private Property VideoPageID As String = String.Empty
+        Private Property ReelsPageID As String = String.Empty
         Private Property StoryBucket As String = String.Empty
         Friend Property ParsePhotoBlock As Boolean = True
         Friend Property ParseVideoBlock As Boolean = True
+        Friend Property ParseReelsBlock As Boolean = False
         Friend Property ParseStoriesBlock As Boolean = True
         Private Enum PageBlock As Integer
             Timeline = Sections.Timeline
             Stories = Sections.Stories
             Photos = 100
             Videos = 101
+            Reels = Sections.Reels
             Undefined = -1
         End Enum
 #End Region
@@ -67,6 +72,7 @@ Namespace API.Facebook
                 With DirectCast(Obj, UserExchangeOptions)
                     ParsePhotoBlock = .ParsePhotoBlock
                     ParseVideoBlock = .ParseVideoBlock
+                    ParseReelsBlock = .ParseReelsBlock
                     ParseStoriesBlock = .ParseStoriesBlock
                 End With
             End If
@@ -90,18 +96,22 @@ Namespace API.Facebook
                     End If
                     OptionsParsed = .Value(Name_OptionsParsed).FromXML(Of Boolean)(False)
                     VideoPageID = .Value(Name_VideoPageID)
+                    ReelsPageID = .Value(Name_ReelsPageID)
                     StoryBucket = .Value(Name_StoryBucket)
                     ParsePhotoBlock = .Value(Name_ParsePhotoBlock).FromXML(Of Boolean)(True)
                     ParseVideoBlock = .Value(Name_ParseVideoBlock).FromXML(Of Boolean)(True)
+                    ParseReelsBlock = .Value(Name_ParseReelsBlock).FromXML(Of Boolean)(False)
                     ParseStoriesBlock = .Value(Name_ParseStoriesBlock).FromXML(Of Boolean)(True)
                 Else
                     updateNames.Invoke
                     .Add(Name_IsNoNameProfile, IsNoNameProfile.BoolToInteger)
                     .Add(Name_OptionsParsed, OptionsParsed.BoolToInteger)
                     .Add(Name_VideoPageID, VideoPageID)
+                    .Add(Name_ReelsPageID, ReelsPageID)
                     .Add(Name_StoryBucket, StoryBucket)
                     .Add(Name_ParsePhotoBlock, ParsePhotoBlock.BoolToInteger)
                     .Add(Name_ParseVideoBlock, ParseVideoBlock.BoolToInteger)
+                    .Add(Name_ParseReelsBlock, ParseReelsBlock.BoolToInteger)
                     .Add(Name_ParseStoriesBlock, ParseStoriesBlock.BoolToInteger)
                 End If
             End With
@@ -146,6 +156,7 @@ Namespace API.Facebook
                     Else
                         If DownloadImages And ParsePhotoBlock Then DownloadData_Photo(String.Empty, Token)
                         If DownloadVideos And ParseVideoBlock Then DownloadData_Video(String.Empty, Token)
+                        If DownloadVideos And ParseReelsBlock Then DownloadData_Reels(String.Empty, Token)
                         If (DownloadImages Or DownloadVideos) And ParseStoriesBlock Then DownloadData_Stories(Token)
                     End If
                     LoadSavePostsKV(False)
@@ -158,10 +169,12 @@ Namespace API.Facebook
         Private Const Header_fb_fr_name_Video As String = "PagesCometChannelTabAllVideosCardImplPaginationQuery"
         Private Const Header_fb_fr_name_Stories As String = "StoriesSuspenseContentPaneRootWithEntryPointQuery"
         Private Const Header_fb_fr_name_SavedPosts As String = "CometSaveDashboardAllItemsPaginationQuery"
+        Private Const Header_fb_fr_name_Reels As String = "ProfileCometAppCollectionReelsRendererPaginationQuery"
         Private Const DocID_Photo As String = "6684543058255697"
         Private Const DocID_Video As String = "24545934291687581"
         Private Const DocID_Stories As String = "6771064226315961"
         Private Const DocID_SavedPosts As String = "7112228098805003"
+        Private Const DocID_Reels As String = "28517740954539304"
         Private Const Graphql_UrlPattern As String = "https://www.facebook.com/api/graphql?lsd={0}&doc_id={1}&server_timestamps=true&fb_dtsg={3}&fb_api_req_friendly_name={2}&variables={4}"
         Private Const VideoHtmlUrlPattern As String = "https://www.facebook.com/watch/?v={0}"
         Private Sub DownloadData_Photo(ByVal Cursor As String, ByVal Token As CancellationToken)
@@ -238,7 +251,7 @@ Namespace API.Facebook
                 Dim newPostsDetected As Boolean = False
                 Dim pid As PostKV
 
-                If VideoPageID.IsEmptyString Then GetVideoPageID(Token)
+                If VideoPageID.IsEmptyString Then GetVideoPageID(False, Token)
                 If VideoPageID.IsEmptyString Then Throw New TokensException("Unable to obtain 'VideoPageID'", False)
                 ValidateBaseTokens()
 
@@ -355,6 +368,123 @@ Namespace API.Facebook
                 ProcessException(ex, Token, $"data (stories) downloading error [{URL}]",, Responser)
             End Try
         End Sub
+        Private Sub DownloadData_Reels(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = String.Empty
+            Const VarPattern$ = """count"":10,""cursor"":{0},""feedLocation"":""COMET_MEDIA_VIEWER"",""feedbackSource"":65,""focusCommentID"":null,""renderLocation"":null,""scale"":1,""useDefaultActor"":true,""id"":""{1}"",""__relay_internal__pv__FBReelsMediaFooter_comet_enable_reels_ads_gkrelayprovider"":true,""__relay_internal__pv__IsWorkUserrelayprovider"":false"
+            Try
+                Dim nextCursor$ = String.Empty
+                Dim newPostsDetected As Boolean = False
+                Dim nodeFound As Boolean = False
+                Dim pid As PostKV = Nothing
+                Dim __urlBase$ = String.Empty
+                Dim lines As List(Of String)
+                Dim j As EContainer, rr As EContainer
+                Dim jDataRoot As EContainer = Nothing
+                Dim indx% = -1
+                Dim s As New List(Of Sizes)
+                Dim videoIdNode$() = {"profile_reel_node", "node", "video", "id"}
+
+                Dim obtainBasePostData As Action = Sub()
+                                                       If indx.ValueBetween(0, jDataRoot.Count - 1) Then
+                                                           With jDataRoot(indx)
+                                                               pid = New PostKV(String.Empty, .Item(videoIdNode).XmlIfNothingValue.
+                                                                                               IfNullOrEmpty(.Value({"node"}, "id")), PageBlock.Reels)
+                                                               pid.Code = $"Reels:{pid.ID}"
+                                                               nextCursor = .Value("cursor")
+                                                               If Not .Item(videoIdNode).XmlIfNothing.IsEmptyString Then
+                                                                   __urlBase = $"https://www.facebook.com/reel/{pid.ID}"
+                                                               Else
+                                                                   __urlBase = String.Empty
+                                                               End If
+                                                           End With
+                                                       Else
+                                                           pid = Nothing
+                                                           nextCursor = String.Empty
+                                                           __urlBase = String.Empty
+                                                       End If
+                                                   End Sub
+                Dim createFile As Func(Of String, SFile, SFile) = Function(ByVal __url As String, ByVal cFile As SFile) As SFile
+                                                                      Dim f As New SFile(RegexReplace(__url, Regex_ReelsFilePattern))
+                                                                      If Not f.IsEmptyString Then Return f Else Return cFile
+                                                                  End Function
+
+                If ReelsPageID.IsEmptyString Then GetVideoPageID(True, Token)
+                If ReelsPageID.IsEmptyString Then Throw New TokensException("Unable to obtain 'ReelsPageID'", False)
+                ValidateBaseTokens()
+
+                URL = String.Format(Graphql_UrlPattern, Token_lsd, DocID_Reels, Header_fb_fr_name_Reels, Token_dtsg_Var,
+                                    SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(VarPattern, If(Cursor.IsEmptyString, "null", $"""{Cursor}"""), ReelsPageID) & "}"))
+
+                ResponserApplyDefs(Header_fb_fr_name_Reels)
+                ThrowAny(Token)
+
+                WaitTimer()
+                Dim r$ = Responser.GetResponse(URL)
+                If Not r.IsEmptyString Then
+                    lines = r.StringToList(Of String)(vbCrLf).ListIfNothing
+                    If lines.ListExists Then
+                        For Each line$ In lines
+                            j = JsonDocument.Parse(line, EDP.ReturnValue)
+                            If j.ListExists Then
+                                jDataRoot = j({"data", "node", "aggregated_fb_shorts", "edges"})
+                                If jDataRoot.ListExists Then
+
+                                    With j({"extensions", "all_video_dash_prefetch_representations"})
+                                        If .ListExists Then
+                                            ProgressPre.ChangeMax(.Count)
+                                            For indx = 0 To .Count - 1
+                                                ProgressPre.Perform()
+
+                                                obtainBasePostData()
+                                                If Not pid.ID.IsEmptyString AndAlso Not PostKvExists(pid) Then
+                                                    newPostsDetected = True
+                                                    PostsKVIDs.ListAddValue(pid, LNC)
+                                                    _TempPostsList.Add(pid.Code)
+
+                                                    With .ItemF({indx, "representations"})
+                                                        If .ListExists Then
+                                                            s.Clear()
+                                                            For Each rr In .Self : s.Add(New Sizes(rr.Value("width"), rr.Value("base_url"))) : Next
+                                                            If s.Count > 0 Then s.RemoveAll(Function(ss) ss.Value = 0 Or ss.Data.IsEmptyString)
+                                                            If s.Count > 0 Then
+                                                                s.Sort()
+                                                                _TempMediaList.ListAddValue(New UserMedia(s(0).Data, UTypes.Video) With {
+                                                                                                .URL_BASE = __urlBase.IfNullOrEmpty(.URL_BASE),
+                                                                                                .Post = pid.ID,
+                                                                                                .File = createFile(s(0).Data, .File),
+                                                                                                .SpecialFolder = "Reels*"
+                                                                                            }, LNC)
+                                                                s.Clear()
+                                                            End If
+                                                        End If
+                                                    End With
+
+
+                                                    If Limit > 0 And _TempMediaList.Count >= Limit Then j.Dispose() : Exit Sub
+                                                Else
+                                                    j.Dispose()
+                                                    Exit Sub
+                                                End If
+
+                                            Next
+                                        End If
+                                    End With
+
+                                End If
+
+                                j.Dispose()
+                            End If
+                        Next
+                    End If
+                End If
+
+                If newPostsDetected And Not nextCursor.IsEmptyString Then DownloadData_Reels(nextCursor, Token)
+            Catch tex As TokensException When Not tex.BasicTokens
+                TokensException.SendToLog(Me, tex, "data (reels)")
+            Catch ex As Exception
+                ProcessException(ex, Token, $"data (reels) downloading error [{URL}]",, Responser)
+            End Try
+        End Sub
         Private Sub DownloadData_SavedPosts(ByVal Cursor As String, ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
             Const VarPattern$ = """content_filter"":[],""count"":10,""cursor"":{0},""scale"":1,""use_case"":""SAVE_DEFAULT"""
@@ -507,13 +637,19 @@ Namespace API.Facebook
                 Return True
             End If
         End Function
-        Private Sub GetVideoPageID(ByVal Token As CancellationToken)
+        Private Sub GetVideoPageID(ByVal GetReels As Boolean, ByVal Token As CancellationToken)
-            Dim URL$ = $"{GetProfileUrl()}\videos"
+            Dim URL$ = $"{GetProfileUrl()}\{IIf(GetReels, "reels", "videos")}"
             Dim resp As Responser = HtmlResponserCreate()
             Try
                 WaitTimer()
                 Dim r$ = resp.GetResponse(URL)
-                If Not r.IsEmptyString Then VideoPageID = RegexReplace(r, Regex_VideoPageID)
+                If Not r.IsEmptyString Then
+                    If GetReels Then
+                        ReelsPageID = RegexReplace(r, Regex_ReelsPageID)
+                    Else
+                        VideoPageID = RegexReplace(r, Regex_VideoPageID)
+                    End If
+                End If
             Catch ex As Exception
                 ProcessException(ex, Token, "get video page ID",, resp)
             Finally
@@ -658,17 +794,39 @@ Namespace API.Facebook
                 HtmlResponserDispose(resp)
             End Try
         End Sub
+        Private Structure VideoResolution : Implements IComparable(Of VideoResolution)
+            Friend W As Integer
+            Friend H As Integer
+            Friend B As Integer
+            Friend U As String
+            Friend ReadOnly Property Wrong As Boolean
+                Get
+                    Return W = 0 Or H = 0 Or B = 0 Or U.IsEmptyString
+                End Get
+            End Property
+            Private Function CompareTo(ByVal Other As VideoResolution) As Integer Implements IComparable(Of VideoResolution).CompareTo
+                Return CLng(Math.Max(W, H) * B).CompareTo(CLng(Math.Max(Other.W, Other.H) * Other.B)) * -1
+            End Function
+        End Structure
         Protected Function ReparseSingleVideo(ByVal m As UserMedia, ByVal resp As Responser, ByRef result As Boolean) As UserMedia
             Const nameSD$ = "browser_native_sd_url"
             Const nameHD$ = "browser_native_hd_url"
+            Const nameDPR$ = "all_video_dash_prefetch_representations"
             Const pattern$ = "<script type=""application/json""[^\>]*data-sjs>([^<]+?{0}[^<]+)<"
             Dim URL$ = String.Empty
             Dim j As EContainer = Nothing
             Try
-                Dim r$, script$, __url$
+                Dim r$ = String.Empty, script$ = String.Empty, __url$ = String.Empty
+                Dim isNewNodes As Boolean = False
+                Dim __date As Date? = Nothing
                 Dim jNode As EContainer
                 Dim jf As Predicate(Of EContainer) = Function(ee) Not ee.Name.IsEmptyString AndAlso (ee.Name.ToLower = nameSD Or ee.Name.ToLower = nameHD)
                 Dim re As RParams = RParams.DMS("", 1, RegexOptions.IgnoreCase, EDP.ReturnValue)
+                Dim nf As New XML.Base.NodeParams(nameDPR, True, True, True, True, 20)
+                Dim __extractScript As Action(Of String) = Sub(ByVal inputName As String)
+                                                               re.Pattern = String.Format(pattern, inputName)
+                                                               script = RegexReplace(r, re)
+                                                           End Sub
                 If m.Post.ID.IsEmptyString Then
                     URL = m.URL_BASE
                 Else
@@ -677,30 +835,47 @@ Namespace API.Facebook
                 WaitTimer()
                 r = resp.GetResponse(URL)
                 If Not r.IsEmptyString Then
-                    re.Pattern = String.Format(pattern, nameHD)
+                    __extractScript(nameHD)
-                    script = RegexReplace(r, re)
+                    If script.IsEmptyString Then __extractScript(nameSD)
-                    If script.IsEmptyString Then
+                    If script.IsEmptyString Then __extractScript(nameDPR) : isNewNodes = True
-                        re.Pattern = String.Format(pattern, nameSD)
-                        script = RegexReplace(r, re)
-                    End If
                     If Not script.IsEmptyString Then
                         j = JsonDocument.Parse(script)
                         If j.ListExists Then
                             j.SetSourceReferences()
-                            jNode = j.Find(jf, True)
+                            If isNewNodes Then
-                            If Not jNode Is Nothing Then
+                                jNode = j.GetNode({nf})
-                                With DirectCast(jNode.Source, EContainer)
+                                If Not jNode Is Nothing Then
-                                    __url = .Value(nameHD).IfNullOrEmpty(.Value(nameSD))
+                                    With jNode.ItemF({0, "representations"})
-                                    If Not __url.IsEmptyString Then
+                                        If .ListExists Then
-                                        m.URL = __url
+                                            Dim intE As New ErrorsDescriber(False, False, False, 0)
-                                        m.URL_BASE = URL
+                                            Dim intC As Func(Of String, Integer) = Function(__input) AConvert(Of Integer)(__input, intE)
-                                        m.Type = UTypes.Video
+                                            Dim dataV As List(Of VideoResolution) = .Select(Function(jj) New VideoResolution With {
-                                        m.File = CreateFileFromUrl(__url)
+                                                                                                .W = intC(jj.Value("width")),
-                                        m.Post.Date = AConvert(Of Date)(.Value("publish_time"), UnixDate32Provider, Nothing)
+                                                                                                .H = intC(jj.Value("height")),
-                                        result = True
+                                                                                                .B = intC(jj.Value("bandwidth")),
-                                        Return m
+                                                                                                .U = jj.Value("base_url")}).ListIfNothing
-                                    End If
+                                            If dataV.ListExists Then dataV.RemoveAll(Function(dd) dd.Wrong)
-                                End With
+                                            If dataV.ListExists Then dataV.Sort() : __url = dataV(0).U : dataV.Clear() : __date = m.Post.Date
+                                        End If
+                                    End With
+                                End If
+                            Else
+                                jNode = j.Find(jf, True)
+                                If Not jNode Is Nothing Then
+                                    With DirectCast(jNode.Source, EContainer)
+                                        __url = .Value(nameHD).IfNullOrEmpty(.Value(nameSD))
+                                        If Not __url.IsEmptyString Then __date = AConvert(Of Date)(.Value("publish_time"), UnixDate32Provider, Nothing)
+                                    End With
+                                End If
+                            End If
+                            If Not __url.IsEmptyString Then
+                                m.URL = __url
+                                m.URL_BASE = URL
+                                m.Type = UTypes.Video
+                                m.File = CreateFileFromUrl(__url)
+                                m.Post.Date = __date
+                                result = True
+                                Return m
                             End If
                         End If
                     End If
@@ -738,7 +913,10 @@ Namespace API.Facebook
 #End Region
 #Region "DownloadSingleObject"
         Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
-            _ContentList.Add(New UserMedia(Data.URL, UTypes.VideoPre) With {.Post = CStr(AConvert(Of String)(Data.URL, Regex_VideoIDFromURL, String.Empty))})
+            _ContentList.Add(New UserMedia(Data.URL, UTypes.VideoPre) With {
+                                 .Post = CStr(AConvert(Of String)(Data.URL, Regex_VideoIDFromURL, String.Empty)),
+                                 .State = UStates.Missing
+                             })
             ReparseMissing(Token)
         End Sub
 #End Region

SCrawler/API/Facebook/UserExchangeOptions.vb

@@ -13,6 +13,8 @@ Namespace API.Facebook
         Friend Property ParsePhotoBlock As Boolean = True
         <PSetting(NameOf(SiteSettings.ParseVideoBlock), NameOf(MySettings))>
         Friend Property ParseVideoBlock As Boolean = True
+        <PSetting(NameOf(SiteSettings.ParseReelsBlock), NameOf(MySettings))>
+        Friend Property ParseReelsBlock As Boolean = False
         <PSetting(NameOf(SiteSettings.ParseStoriesBlock), NameOf(MySettings))>
         Friend Property ParseStoriesBlock As Boolean = True
         Private ReadOnly Property MySettings As SiteSettings
@@ -20,12 +22,14 @@ Namespace API.Facebook
             MySettings = u.HostCollection.Default.Source
             ParsePhotoBlock = u.ParsePhotoBlock
             ParseVideoBlock = u.ParseVideoBlock
+            ParseReelsBlock = u.ParseReelsBlock
             ParseStoriesBlock = u.ParseStoriesBlock
         End Sub
         Friend Sub New(ByVal s As SiteSettings)
             MySettings = s
             ParsePhotoBlock = s.ParsePhotoBlock.Value
             ParseVideoBlock = s.ParseVideoBlock.Value
+            ParseReelsBlock = s.ParseReelsBlock.Value
             ParseStoriesBlock = s.ParseStoriesBlock.Value
         End Sub
     End Class

SCrawler/API/Instagram/EditorExchangeOptions.vb

@@ -7,9 +7,8 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports SCrawler.Plugin.Attributes
-Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.Instagram
-    Friend NotInheritable Class EditorExchangeOptions
+    Friend NotInheritable Class EditorExchangeOptions : Inherits Base.EditorExchangeOptionsBase
 #Region "Download"
         <PSetting(Caption:="Get timeline", ToolTip:="Download user timeline")>
         Friend Property GetTimeline As Boolean
@@ -36,13 +35,13 @@ Namespace API.Instagram
 #End Region
         <PSetting(Caption:="Place the extracted image into the video folder")>
         Friend Property PutImageVideoFolder As Boolean
-        <PSetting(Address:=SettingAddress.User, Caption:=DN.UserNameChangeCaption, ToolTip:=DN.UserNameChangeToolTip)>
+        Friend Overrides Property UserName As String
-        Friend Property UserName As String = String.Empty
         <PSetting(Address:=SettingAddress.User, Caption:="Force update UserName", ToolTip:="Try to force update UserName if it is not found on the site")>
         Friend Property ForceUpdateUserName As Boolean = False
         <PSetting(Address:=SettingAddress.User, Caption:="Force update user information")>
         Friend Property ForceUpdateUserInfo As Boolean = False
         Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
             With u
                 GetTimeline = .GetTimeline
                 GetReels = .GetReels
@@ -58,7 +57,6 @@ Namespace API.Instagram
 
                 PutImageVideoFolder = .PutImageVideoFolder
 
-                UserName = .NameTrue(True)
                 ForceUpdateUserName = .ForceUpdateUserName
                 ForceUpdateUserInfo = .ForceUpdateUserInfo
             End With

SCrawler/API/Instagram/SiteSettings.vb

@@ -732,9 +732,6 @@ Namespace API.Instagram
                 Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
             End If
         End Sub
-        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
-            Return String.Format(UrlPatternUser, DirectCast(User, UserData).NameTrue)
-        End Function
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
             Try
                 Dim code$ = DirectCast(User, UserData).GetPostCodeById(Media.Post.ID)

SCrawler/API/Instagram/UserData.vb

@@ -37,7 +37,6 @@ Namespace API.Instagram
         Private Const Name_GetTagged_VideoPic As String = "GetTaggedData_VideoPic"
         Private Const Name_PutImageVideoFolder As String = "PutImageVideoFolder"
         Private Const Name_TaggedChecked As String = "TaggedChecked"
-        Private Const Name_NameTrue As String = "NameTrue"
         Private Const Name_ForceUpdateUserName As String = "ForceUpdateUserName"
         Private Const Name_ForceUpdateUserInfo As String = "ForceUpdateUserInfo"
 #End Region
@@ -113,12 +112,6 @@ Namespace API.Instagram
                 Case Else : Return True
             End Select
         End Function
-        Protected _NameTrue As String = String.Empty
-        Friend ReadOnly Property NameTrue(Optional ByVal Exact As Boolean = False) As String
-            Get
-                Return If(Exact, _NameTrue, _NameTrue.IfNullOrEmpty(Name))
-            End Get
-        End Property
         Private UserNameRequested As Boolean = False
         Friend Property ForceUpdateUserName As Boolean = False
         Friend Property ForceUpdateUserInfo As Boolean = False
@@ -141,7 +134,6 @@ Namespace API.Instagram
                     GetTaggedData = .Value(Name_GetTagged).FromXML(Of Boolean)(CBool(MySiteSettings.GetTagged.Value))
                     GetTaggedData_VideoPic = .Value(Name_GetTagged_VideoPic).FromXML(Of Boolean)(CBool(MySiteSettings.GetTagged_VideoPic.Value))
                     TaggedChecked = .Value(Name_TaggedChecked).FromXML(Of Boolean)(False)
-                    _NameTrue = .Value(Name_NameTrue)
                     ForceUpdateUserName = .Value(Name_ForceUpdateUserName).FromXML(Of Boolean)(False)
                     ForceUpdateUserInfo = .Value(Name_ForceUpdateUserInfo).FromXML(Of Boolean)(False)
                 Else
@@ -159,7 +151,6 @@ Namespace API.Instagram
                     .Add(Name_GetTagged_VideoPic, GetTaggedData_VideoPic.BoolToInteger)
                     .Add(Name_PutImageVideoFolder, PutImageVideoFolder.BoolToInteger)
                     .Add(Name_TaggedChecked, TaggedChecked.BoolToInteger)
-                    .Add(Name_NameTrue, _NameTrue)
                     .Add(Name_ForceUpdateUserName, ForceUpdateUserName.BoolToInteger)
                     .Add(Name_ForceUpdateUserInfo, ForceUpdateUserInfo.BoolToInteger)
                 End If
@@ -187,7 +178,7 @@ Namespace API.Instagram
 
                     PutImageVideoFolder = .PutImageVideoFolder
 
-                    _NameTrue = .UserName
+                    NameTrue = .UserName
                     ForceUpdateUserName = .ForceUpdateUserName
                     ForceUpdateUserInfo = .ForceUpdateUserInfo
                 End With
@@ -480,17 +471,24 @@ Namespace API.Instagram
             End Try
         End Sub
         Protected Sub ValidateExtension()
+            Dim tmpList As List(Of UserMedia) = Nothing
             Try
                 Const heic$ = "heic"
                 If _TempMediaList.Count > 0 AndAlso _TempMediaList.Exists(Function(mm) mm.File.Extension = heic) Then
                     Dim m As UserMedia
-                    For i% = 0 To _TempMediaList.Count - 1
+                    tmpList = New List(Of UserMedia)
-                        m = _TempMediaList(i)
+                    tmpList.ListAddList(_TempMediaList)
+                    _TempMediaList.Clear()
+                    For i% = 0 To tmpList.Count - 1
+                        m = tmpList(i)
+                        _TempMediaList.Add(m)
                         If m.Type = UTypes.Picture AndAlso Not m.File.Extension.IsEmptyString AndAlso m.File.Extension = heic Then _
-                           m.File.Extension = "jpg" : _TempMediaList(i) = m
+                           m.File.Extension = "jpg" : _TempMediaList.Add(m)
                     Next
+                    tmpList.Clear()
                 End If
             Catch ex As Exception
+                If tmpList.ListExists Then _TempMediaList.Clear() : _TempMediaList.ListAddList(tmpList) : tmpList.Clear()
             End Try
         End Sub
         Protected Overridable Sub UpdateResponser()
@@ -1212,11 +1210,8 @@ NextPageBlock:
                                 Dim f As New SFile With {.Path = DownloadContentDefault_GetRootDir(), .Name = "ProfilePicture", .Extension = "jpg"}
                                 f = SFile.IndexReindex(f)
                                 If Not f.Exists Then
-                                    Dim profilePicture$ = .Value("profile_pic_url_hd")
+                                    If SimpleDownloadAvatar(.Value("profile_pic_url_hd"), Function(ff) f).IsEmptyString Then _
-                                    If profilePicture.IsEmptyString OrElse Not GetWebFile(profilePicture, f, EDP.ReturnValue) Then
+                                       SimpleDownloadAvatar(.Value("profile_pic_url"), Function(ff) f)
-                                        profilePicture = .Value("profile_pic_url")
-                                        If Not profilePicture.IsEmptyString Then GetWebFile(profilePicture, f, EDP.ReturnValue)
-                                    End If
                                 End If
                             End With
                         End If
@@ -1249,11 +1244,10 @@ NextPageBlock:
                                 If Not newName.IsEmptyString Then
                                     Dim oldName$ = NameTrue
                                     If Not newName = oldName Then
-                                        MyMainLOG = $"{ToStringForLog()}: username changed from '{oldName}' to '{newName}'"
+                                        Dim uStr$ = $"username changed from '{oldName}' to '{newName}'"
-                                        _NameTrue = newName
+                                        LogError(Nothing, uStr)
-                                        Dim descr$ = $"Username changed from '{oldName}' to '{newName}' ({Now.ToStringDate(ADateTime.Formats.BaseDateTime)})!"
+                                        NameTrue = newName
-                                        descr.StringAppendLine(UserDescription)
+                                        UserDescriptionUpdate(uStr, True, True, True)
-                                        UserDescription = descr
                                         _ForceSaveUserInfo = True
                                     End If
                                     Return True

SCrawler/API/LPSG/UserData.vb

@@ -106,7 +106,8 @@ Namespace API.LPSG
         End Sub
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer
-            If Responser.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then '503
+            If Responser.StatusCode = Net.HttpStatusCode.ServiceUnavailable Or
+               Responser.StatusCode = Net.HttpStatusCode.Forbidden Then '503, 403
                 MyMainLOG = $"{ToStringForLog()}: LPSG not available"
                 Return 1
             ElseIf Responser.StatusCode = Net.HttpStatusCode.NotFound Then '404

SCrawler/API/Mastodon/SiteSettings.vb

@@ -171,12 +171,12 @@ Namespace API.Mastodon
             With DirectCast(User, UserData)
                 If UserRelatedToMyDomain.Value Then
                     If MyDomain.Value = .UserDomain Then
-                        Return $"https://{ .UserDomain}/@{ .TrueName}"
+                        Return $"https://{ .UserDomain}/@{ .NameTrue}"
                     Else
-                        Return $"https://{MyDomain.Value}/@{ .TrueName}@{ .UserDomain}"
+                        Return $"https://{MyDomain.Value}/@{ .NameTrue}@{ .UserDomain}"
                     End If
                 Else
-                    Return $"https://{ .UserDomain}/@{ .TrueName}"
+                    Return $"https://{ .UserDomain}/@{ .NameTrue}"
                 End If
             End With
         End Function

SCrawler/API/Mastodon/UserData.vb

@@ -29,7 +29,6 @@ Namespace API.Mastodon
                 _UserDomain = d
             End Set
         End Property
-        Friend Property TrueName As String = String.Empty
         Private ReadOnly Property MySettings As SiteSettings
             Get
                 Return HOST.Source
@@ -52,22 +51,21 @@ Namespace API.Mastodon
                                                 Dim l$() = Name.Split("@")
                                                 If l.ListExists(2) Then
                                                     _UserDomain = l(0)
-                                                    TrueName = l(1)
+                                                    NameTrue = l(1)
                                                 Else
                                                     _UserDomain = MySettings.MyDomain.Value
-                                                    TrueName = Name
+                                                    NameTrue = Name
                                                 End If
-                                                If FriendlyName.IsEmptyString Then FriendlyName = TrueName
+                                                If FriendlyName.IsEmptyString Then FriendlyName = NameTrue
                                             End If
                                         End Sub
             If Loading Then
                 _UserDomain = Container.Value(Name_UserDomain)
-                TrueName = Container.Value(Name_TrueName)
                 obtainNames.Invoke
             Else
                 obtainNames.Invoke
                 Container.Add(Name_UserDomain, _UserDomain)
-                Container.Add(Name_TrueName, TrueName)
+                Container.Add(Name_TrueName, NameTrue(True))
                 Container.Value(Name_FriendlyName) = FriendlyName
             End If
         End Sub
@@ -208,12 +206,12 @@ Namespace API.Mastodon
                     Dim url$ = $"https://{MyCredentials.Domain}/api/v1/accounts/lookup?acct="
                     If Not UserDomain.IsEmptyString Then
                         If UserDomain = MyCredentials.Domain Then
-                            url &= $"@{TrueName}"
+                            url &= $"@{NameTrue}"
                         Else
-                            url &= $"@{TrueName}@{UserDomain}"
+                            url &= $"@{NameTrue}@{UserDomain}"
                         End If
                     Else
-                        url &= $"@{TrueName}"
+                        url &= $"@{NameTrue}"
                     End If
                     Dim r$ = Responser.GetResponse(url)
                     If Not r.IsEmptyString Then

SCrawler/API/OnlyFans/SiteSettings.vb

@@ -36,12 +36,18 @@ Namespace API.OnlyFans
         Private Const HeaderUserID As String = "User-Id"
         Friend Const HeaderXBC As String = "X-Bc"
         Friend Const HeaderAppToken As String = "App-Token"
+        Private Const AppTokenDefault As String = "33d57ade8c02dbc5a333db99ff9ae26a"
         <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_USER_ID As PropertyValue
         <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Private ReadOnly Property HH_X_BC As PropertyValue
         <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False, IsAuth:=True), PClonable(Clone:=False)>
         Private ReadOnly Property HH_APP_TOKEN As PropertyValue
+        <PropertyUpdater(NameOf(HH_APP_TOKEN))>
+        Private Function UpdateAppToken() As Boolean
+            HH_APP_TOKEN.Value = AppTokenDefault
+            Return True
+        End Function
         <PropertyOption(ControlText:=HeaderBrowser, ControlToolTip:="Can be null", AllowNull:=True,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua, IsAuth:=True), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_BROWSER As PropertyValue
@@ -225,13 +231,13 @@ Namespace API.OnlyFans
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.AcceptEncoding))
                     HH_USER_ID = New PropertyValue(.Value(HeaderUserID), GetType(String), Sub(v) UpdateHeader(NameOf(HH_USER_ID), v))
                     HH_X_BC = New PropertyValue(.Value(HeaderXBC), GetType(String), Sub(v) UpdateHeader(NameOf(HH_X_BC), v))
-                    HH_APP_TOKEN = New PropertyValue(.Value(HeaderAppToken), GetType(String), Sub(v) UpdateHeader(NameOf(HH_APP_TOKEN), v))
+                    HH_APP_TOKEN = New PropertyValue(.Value(HeaderAppToken).IfNullOrEmpty(AppTokenDefault), GetType(String), Sub(v) UpdateHeader(NameOf(HH_APP_TOKEN), v))
                     HH_BROWSER = New PropertyValue(.Value(HeaderBrowser), GetType(String), Sub(v) UpdateHeader(NameOf(HH_BROWSER), v))
                 End With
                 UserAgent = New PropertyValue(IIf(.UserAgentExists, .UserAgent, String.Empty), GetType(String), Sub(v) UpdateHeader(NameOf(UserAgent), v))
             End With
 
-            EnableCookiesUpdate = New PropertyValue(True)
+            EnableCookiesUpdate = New PropertyValue(False)
 
             DownloadTimeline = New PropertyValue(True)
             DownloadStories = New PropertyValue(True)
@@ -262,6 +268,15 @@ Namespace API.OnlyFans
             UrlPatternUser = "https://onlyfans.com/{0}"
             ImageVideoContains = "onlyfans.com"
         End Sub
+        Private Const SettingsVersionCurrent As Integer = 1
+        Friend Overrides Sub EndInit()
+            If CInt(SettingsVersion.Value) < SettingsVersionCurrent Then
+                If CStr(HH_APP_TOKEN.Value).IsEmptyString Then HH_APP_TOKEN.Value = AppTokenDefault
+                EnableCookiesUpdate.Value = False
+                SettingsVersion.Value = SettingsVersionCurrent
+            End If
+            MyBase.EndInit()
+        End Sub
 #End Region
 #Region "GetInstance"
         Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
@@ -312,7 +327,7 @@ Namespace API.OnlyFans
 #Region "GetUserUrl, GetUserPostUrl, UserOptions"
         Friend Const UserPostPattern As String = "https://onlyfans.com/{0}/{1}"
         Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
-            Return String.Format(UrlPatternUser, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
+            Return String.Format(UrlPatternUser, If(User.ID.IsEmptyString, User.NameTrue.IfNullOrEmpty(User.Name), $"u{User.ID}"))
         End Function
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
             If Not Media.Post.ID.IsEmptyString Then

SCrawler/API/OnlyFans/UserData.vb

@@ -467,16 +467,8 @@ Namespace API.OnlyFans
                                 Dim descr$ = j.Value("about")
                                 If Not descr.IsEmptyString Then descr = descr.Replace(brTag, String.Empty)
                                 UserDescriptionUpdate(descr)
-                                Dim a As Action(Of String) = Sub(ByVal address As String)
+                                SimpleDownloadAvatar(j.Value("avatar"))
-                                                                 If Not address.IsEmptyString Then
+                                SimpleDownloadAvatar(j.Value("header"))
-                                                                     Dim f As SFile = address
-                                                                     f.Separator = "\"
-                                                                     f.Path = DownloadContentDefault_GetRootDir()
-                                                                     If Not f.Exists Then GetWebFile(address, f, EDP.None)
-                                                                 End If
-                                                             End Sub
-                                a.Invoke(j.Value("avatar"))
-                                a.Invoke(j.Value("header"))
                             End If
                         End Using
                     End If

SCrawler/API/Pinterest/Declarations.vb

@@ -7,11 +7,18 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports System.Globalization
+Imports System.Text.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Pinterest
     Friend Module Declarations
         Friend ReadOnly DateProvider As ADateTime = GetDateProvider()
         Friend ReadOnly PwsHeader As New HttpHeader("x-pinterest-pws-handler", "www/[username]/pins.js")
+        Friend ReadOnly GdlUrlPattern As RParams = RParams.DM(Base.GDL.GDLBatch.UrlLibStart.Replace("[", "\[").Replace("]", "\]") &
+                                                              "([^""]+?)""(GET [^""]+)""", 0, EDP.ReturnValue)
+        Friend ReadOnly SubBoardRegEx As RParams = RParams.DMS("\[pinterest\]\[debug\] Using PinterestSectionExtractor for '[^']+id:(\d+)'", 1,
+                                                               RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly BoardInfoRootNode As String() = {"resource_response", "data"}
         Private Function GetDateProvider() As ADateTime
             Dim n As DateTimeFormatInfo = CultureInfo.GetCultureInfo("en-us").DateTimeFormat.Clone
             n.FullDateTimePattern = "ddd dd MMM yyyy HH:mm:ss"

SCrawler/API/Pinterest/EditorExchangeOptions.vb

@@ -0,0 +1,20 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.Plugin.Attributes
+Namespace API.Pinterest
+    Friend Class EditorExchangeOptions
+        <PSetting(Caption:="Get Sub-Boards", ToolTip:="Extract the Sub-Boards from the boards and download them")>
+        Friend Property ExtractSubBoards As Boolean = True
+        Friend Sub New(ByVal u As UserData)
+            ExtractSubBoards = u.ExtractSubBoards
+        End Sub
+        Friend Sub New()
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Pinterest/SiteSettings.vb

@@ -11,7 +11,7 @@ Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Pinterest
-    <Manifest("AndyProgram_Pinterest"), SavedPosts, SeparatedTasks>
+    <Manifest("AndyProgram_Pinterest"), SavedPosts, SeparatedTasks, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
         <PropertyOption(ControlText:=DeclaredNames.ConcurrentDownloadsCaption,
@@ -30,7 +30,7 @@ Namespace API.Pinterest
             MyConcurrentDownloadsProvider = New ConcurrentDownloadsProvider
             CheckNetscapeCookiesOnEndInit = True
             UseNetscapeCookies = True
-            UserRegex = RParams.DMS("https?://w{0,3}.?[^/]*?.?pinterest.com/([^/]+)/?(?(_)|([^/]*))", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
+            UserRegex = RParams.DMS("https?://w{0,3}.?[^/]*?.?pinterest.com/([^/]+)/?(?(_)|([^/]*))/?([^/\?]*)", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
         End Sub
 #End Region
 #Region "GetInstance, Available"
@@ -41,13 +41,14 @@ Namespace API.Pinterest
             Return Settings.GalleryDLFile.Exists And (Not What = ISiteSettings.Download.SavedPosts OrElse ACheck(SavedPostsUserName.Value))
         End Function
 #End Region
-#Region "IsMyUser, IsMyImageVideo, GetUserUrl, GetUserPostUrl"
+#Region "IsMyUser, IsMyImageVideo, GetUserUrl, GetUserPostUrl, UserOptions"
         Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
             If Not UserURL.IsEmptyString Then
                 Dim l As List(Of String) = RegexReplace(UserURL, UserRegex)
                 If l.ListExists(3) Then
                     Dim n$ = l(1)
                     If Not l(2).IsEmptyString Then n &= $"@{l(2)}"
+                    If l.Count > 3 AndAlso Not l(3).IsEmptyString Then n &= $"@{l(3)}"
                     Return New ExchangeOptions(Site, n) With {.Exists = True}
                 End If
             End If
@@ -71,6 +72,12 @@ Namespace API.Pinterest
                 Return String.Empty
             End If
         End Function
+        Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
+            If Options Is Nothing Then Options = New EditorExchangeOptions
+            If OpenForm Then
+                Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
+            End If
+        End Sub
 #End Region
     End Class
 End Namespace

SCrawler/API/Pinterest/UserData.vb

@@ -7,10 +7,12 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports System.Threading
+Imports System.Text.RegularExpressions
 Imports SCrawler.API.Base
 Imports SCrawler.API.Base.GDL
 Imports SCrawler.API.YouTube.Objects
 Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Documents.JSON
 Namespace API.Pinterest
     Friend Class UserData : Inherits UserDataBase
@@ -18,6 +20,8 @@ Namespace API.Pinterest
         Private Const Name_IsUser As String = "IsUser"
         Private Const Name_TrueUserName As String = "TrueUserName"
         Private Const Name_TrueBoardName As String = "TrueBoardName"
+        Private Const Name_ExtractSubBoards As String = "ExtractSubBoards"
+        Private Const Name_IsSubBoard As String = "IsSubBoard"
 #End Region
 #Region "Structures"
         Private Structure BoardInfo
@@ -38,6 +42,8 @@ Namespace API.Pinterest
         Friend Property TrueUserName As String
         Friend Property TrueBoardName As String
         Friend Property IsUser_NB As Boolean
+        Private Property IsSubBoard As Boolean = False
+        Friend Property ExtractSubBoards As Boolean = True
         Private Const BoardLabelName As String = "Board"
         Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
             Get
@@ -45,14 +51,18 @@ Namespace API.Pinterest
             End Get
         End Property
 #End Region
-#Region "Load"
+#Region "Load, Exchange"
         Private Function ReconfUserName() As Boolean
             If TrueUserName.IsEmptyString Then
                 Dim n$() = Name.Split("@")
                 If n.ListExists Then
                     TrueUserName = n(0)
                     IsUser_NB = True
-                    If n.Length > 1 Then TrueBoardName = n(1) : IsUser_NB = False
+                    If n.Length > 1 Then
+                        TrueBoardName = n(1)
+                        If n.Length > 2 AndAlso Not n(2).IsEmptyString Then TrueBoardName &= $"/{n(2)}" : IsSubBoard = True
+                        IsUser_NB = False
+                    End If
                     If Not IsSavedPosts And Not IsSingleObjectDownload Then
                         Dim l$ = IIf(IsUser_NB, UserLabelName, BoardLabelName)
                         Settings.Labels.Add(l)
@@ -70,15 +80,25 @@ Namespace API.Pinterest
                     TrueUserName = .Value(Name_TrueUserName)
                     TrueBoardName = .Value(Name_TrueBoardName)
                     IsUser_NB = .Value(Name_IsUser).FromXML(Of Boolean)(False)
+                    ExtractSubBoards = .Value(Name_ExtractSubBoards).FromXML(Of Boolean)(True)
+                    IsSubBoard = .Value(Name_IsSubBoard).FromXML(Of Boolean)(False)
                     ReconfUserName()
                 Else
                     If ReconfUserName() Then .Value(Name_LabelsName) = LabelsString
                     .Add(Name_TrueUserName, TrueUserName)
                     .Add(Name_TrueBoardName, TrueBoardName)
                     .Add(Name_IsUser, IsUser_NB.BoolToInteger)
+                    .Add(Name_ExtractSubBoards, ExtractSubBoards.BoolToInteger)
+                    .Add(Name_IsSubBoard, IsSubBoard.BoolToInteger)
                 End If
             End With
         End Sub
+        Friend Overrides Function ExchangeOptionsGet() As Object
+            Return New EditorExchangeOptions(Me)
+        End Function
+        Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then ExtractSubBoards = DirectCast(Obj, EditorExchangeOptions).ExtractSubBoards
+        End Sub
 #End Region
 #Region "Download overrides"
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
@@ -125,6 +145,19 @@ Namespace API.Pinterest
         End Sub
 #End Region
 #Region "Get boards, images"
+        Private Function GetBoardInfo(ByVal e As EContainer) As BoardInfo
+            If Not e Is Nothing Then
+                Dim b As New BoardInfo With {
+                    .URL = e.Value("url"),
+                    .Title = TitleHtmlConverter(e.Value("name")).IfNullOrEmpty(TitleHtmlConverter(e.Value("title"))),
+                    .ID = e.Value("id")
+                }
+                If Not b.URL.IsEmptyString Then b.URL = $"https://www.pinterest.com/{b.URL.StringTrimStart("/").StringTrimEnd("/")}/"
+                Return b
+            Else
+                Return Nothing
+            End If
+        End Function
         Private Function GetBoards(ByVal Token As CancellationToken) As List(Of BoardInfo)
             Dim URL$ = $"https://www.pinterest.com/{TrueUserName}/"
             Try
@@ -132,9 +165,9 @@ Namespace API.Pinterest
                 Dim b As BoardInfo
                 Dim r$
                 Dim j As EContainer, jj As EContainer
-                Dim rootNode$() = {"resource_response", "data"}
                 Dim jErr As New ErrorsDescriber(EDP.SendToLog + EDP.ReturnValue)
-                Dim urls As List(Of String) = GetDataFromGalleryDL(URL, True, Token)
+                Dim urls As New List(Of String)
+                urls.ListAddList(GetDataFromGalleryDL(URL, True, Token), LNC)
                 If urls.ListExists Then urls.RemoveAll(Function(__url) Not __url.Contains("BoardsResource/get/"))
                 If urls.ListExists Then
                     ProgressPre.ChangeMax(urls.Count)
@@ -145,17 +178,10 @@ Namespace API.Pinterest
                         If Not r.IsEmptyString Then
                             j = JsonDocument.Parse(r, jErr)
                             If Not j Is Nothing Then
-                                If If(j(rootNode)?.Count, 0) > 0 Then
+                                If If(j(BoardInfoRootNode)?.Count, 0) > 0 Then
-                                    For Each jj In j(rootNode)
+                                    For Each jj In j(BoardInfoRootNode)
-                                        b = New BoardInfo With {
+                                        b = GetBoardInfo(jj)
-                                            .URL = jj.Value("url"),
+                                        If Not b.URL.IsEmptyString Then boards.Add(b)
-                                            .Title = TitleHtmlConverter(jj.Value("name")),
-                                            .ID = jj.Value("id")
-                                        }
-                                        If Not b.URL.IsEmptyString Then
-                                            b.URL = $"https://www.pinterest.com/{b.URL.StringTrimStart("/").StringTrimEnd("/")}/"
-                                            boards.Add(b)
-                                        End If
                                     Next
                                 End If
                                 j.Dispose()
@@ -170,95 +196,156 @@ Namespace API.Pinterest
             End Try
         End Function
         Private Sub DownloadBoardImages(ByRef Board As BoardInfo, ByVal Token As CancellationToken)
-            Dim bUrl$ = String.Empty
+            Dim bUrl As GDLURL = Nothing
             Try
                 Dim r$
                 Dim j As EContainer, jj As EContainer
                 Dim u As UserMedia
-                Dim folder$ = If(IsUser_NB, Board.Title.IfNullOrEmpty(Board.ID), String.Empty)
+                Dim __getBoardTitle As Func(Of BoardInfo, String) = Function(__board) __board.Title.IfNullOrEmpty(__board.ID)
+                Dim folderDef$ = If(IsUser_NB, __getBoardTitle(Board), String.Empty)
                 Dim titleExists As Boolean = Not Board.Title.IsEmptyString
                 Dim i% = -1
                 Dim jErr As New ErrorsDescriber(EDP.SendToLog + EDP.ReturnValue)
-                Dim rootNode$() = {"resource_response", "data"}
+                Dim rErr As New ErrorsDescriber(EDP.ReturnValue)
                 Dim images As List(Of Sizes)
                 Dim imgSelector As Func(Of EContainer, Sizes) = Function(img) New Sizes(img.Value("width"), img.Value("url"))
                 Dim fullData As Predicate(Of EContainer) = Function(e) e.Count > 5
-                Dim l As List(Of String) = GetDataFromGalleryDL(Board.URL, False, Token)
+                Dim subBoard As BoardInfo = Nothing
-                If l.ListExists Then l.RemoveAll(Function(ll) Not ll.Contains("BoardFeedResource/get/"))
+                Dim subBoardAppender As Func(Of String) = Function() _
+                    If(Not __getBoardTitle(subBoard).IsEmptyString,
+                       $"{IIf(folderDef.IsEmptyString, String.Empty, "\")}{__getBoardTitle(subBoard)}",
+                       String.Empty)
+                Dim __getSubBoard As Func(Of Boolean) = Function() ExtractSubBoards Or (IsSubBoard And i = -1)
+                Dim sbCount% = 0
+                Dim __getBoardInfo As Action(Of GDLURL) = Sub(ByVal sb As GDLURL)
+                                                              sbCount += 1
+                                                              r = Responser.GetResponse(sb.URL,, rErr)
+                                                              If Not r.IsEmptyString Then
+                                                                  Using jsb As EContainer = JsonDocument.Parse(r, jErr)
+                                                                      If jsb.ListExists Then subBoard = GetBoardInfo(jsb(BoardInfoRootNode)) : Exit Sub
+                                                                  End Using
+                                                              End If
+                                                              subBoard = Nothing
+                                                          End Sub
+                Dim l As List(Of GDLURL) = GetDataFromGalleryDL(Board.URL, False, Token)
+                If l.ListExists Then l.RemoveAll(Function(ll) ll.URL.IsEmptyString)
                 If l.ListExists Then
                     Responser.Headers.Add(PwsHeader)
                     ProgressPre.ChangeMax(l.Count)
                     For Each bUrl In l
                         ProgressPre.Perform()
                         ThrowAny(Token)
-                        r = Responser.GetResponse(bUrl,, EDP.ReturnValue)
-                        If Not r.IsEmptyString Then
-                            j = JsonDocument.Parse(r, jErr)
-                            If Not j Is Nothing Then
-                                If If(j(rootNode)?.Count, 0) > 0 Then
-                                    ProgressPre.ChangeMax(j(rootNode).Count)
-                                    For Each jj In j(rootNode)
-                                        ProgressPre.Perform()
-                                        With jj
-                                            If .Contains("images") Then
-                                                images = .Item("images").Select(imgSelector).ToList
-                                                If images.Count > 0 Then
-                                                    images.Sort()
-                                                    i += 1
-                                                    u = New UserMedia(images(0).Data) With {
-                                                        .Post = New UserPost(jj.Value("id"), AConvert(Of Date)(jj.Value("created_at"), DateProvider, Nothing)),
-                                                        .Type = UserMedia.Types.Picture,
-                                                        .SpecialFolder = folder
-                                                    }
-                                                    If i = 0 Then
-                                                        If Board.Title.IsEmptyString Or Board.ID.IsEmptyString Then
-                                                            Board.Title = TitleHtmlConverter(.Value({"board"}, "name"))
-                                                            Board.ID = .Value({"board"}, "id")
-                                                        End If
-                                                        Board.UserID = .Value({"board", "owner"}, "id")
-                                                        Board.UserTitle = TitleHtmlConverter(.Value({"board", "owner"}, "full_name"))
-                                                        If Not titleExists And IsUser_NB Then
-                                                            If Not Board.Title.IsEmptyString Then
-                                                                folder = Board.Title
-                                                            ElseIf Not Board.ID.IsEmptyString Then
-                                                                folder = Board.ID
-                                                            End If
-                                                            u.SpecialFolder = folder
-                                                        End If
-                                                    End If
 
-                                                    If Not u.URL.IsEmptyString Then
+                        If bUrl.URL.Contains("BoardFeedResource/get/") Or (bUrl.URL.Contains("BoardSectionPinsResource/get/") And (ExtractSubBoards Or (IsSubBoard And sbCount = 1))) Then
-                                                        If u.Post.Date.HasValue Then
+                            r = Responser.GetResponse(bUrl.URL,, rErr)
-                                                            Select Case CheckDatesLimit(u.Post.Date.Value, Nothing)
+                            If Not r.IsEmptyString Then
-                                                                Case DateResult.Skip : _TempPostsList.ListAddValue(u.Post.ID, LNC) : Continue For
+                                j = JsonDocument.Parse(r, jErr)
-                                                                Case DateResult.Exit : Exit Sub
+                                If Not j Is Nothing Then
-                                                            End Select
+                                    If If(j(BoardInfoRootNode)?.Count, 0) > 0 Then
+                                        ProgressPre.ChangeMax(j(BoardInfoRootNode).Count)
+                                        For Each jj In j(BoardInfoRootNode)
+                                            ProgressPre.Perform()
+                                            With jj
+                                                If .Contains("images") Then
+                                                    images = .Item("images").Select(imgSelector).ToList
+                                                    If images.Count > 0 Then
+                                                        images.Sort()
+                                                        i += 1
+                                                        u = New UserMedia(images(0).Data) With {
+                                                            .Post = New UserPost(jj.Value("id"), AConvert(Of Date)(jj.Value("created_at"), DateProvider, Nothing)),
+                                                            .Type = UserMedia.Types.Picture,
+                                                            .SpecialFolder = folderDef & subBoardAppender.Invoke
+                                                        }
+                                                        If i = 0 Then
+                                                            If Board.Title.IsEmptyString Or Board.ID.IsEmptyString Then
+                                                                Board.Title = TitleHtmlConverter(.Value({"board"}, "name"))
+                                                                Board.ID = .Value({"board"}, "id")
+                                                            End If
+                                                            Board.UserID = .Value({"board", "owner"}, "id")
+                                                            Board.UserTitle = TitleHtmlConverter(.Value({"board", "owner"}, "full_name"))
+                                                            If Not titleExists And IsUser_NB Then
+                                                                folderDef = Board.Title.IfNullOrEmpty(Board.ID).IfNullOrEmpty(folderDef)
+                                                                u.SpecialFolder = folderDef & subBoardAppender.Invoke
+                                                            End If
                                                         End If
-                                                        If Not _TempPostsList.Contains(u.Post.ID) Then
+
-                                                            _TempPostsList.ListAddValue(u.Post.ID, LNC)
+                                                        If Not u.URL.IsEmptyString Then
-                                                            _TempMediaList.ListAddValue(u, LNC)
+                                                            If u.Post.Date.HasValue Then
-                                                        Else
+                                                                Select Case CheckDatesLimit(u.Post.Date.Value, Nothing)
-                                                            Exit For
+                                                                    Case DateResult.Skip : _TempPostsList.ListAddValue(u.Post.ID, LNC) : Continue For
+                                                                    Case DateResult.Exit : Exit Sub
+                                                                End Select
+                                                            End If
+                                                            If Not _TempPostsList.Contains(u.Post.ID) Then
+                                                                _TempPostsList.ListAddValue(u.Post.ID, LNC)
+                                                                _TempMediaList.ListAddValue(u, LNC)
+                                                            Else
+                                                                Exit For
+                                                            End If
                                                         End If
                                                     End If
                                                 End If
-                                            End If
+                                            End With
-                                        End With
+                                        Next
-                                    Next
+                                    End If
+                                    j.Dispose()
                                 End If
-                                j.Dispose()
+                            End If
+                        ElseIf bUrl.URL.Contains("BoardSectionResource/get/") And (ExtractSubBoards Or (IsSubBoard And i = -1)) Then
+                            __getBoardInfo(bUrl)
+                            If IsSubBoard And i = -1 And Board.Title.IsEmptyString Then
+                                Board.Title = subBoard.Title
+                                If Board.ID.IsEmptyString Then Board.ID = subBoard.ID
+                                subBoard = Nothing
+                                folderDef = String.Empty
                             End If
                         End If
+
                     Next
                 End If
             Catch ex As Exception
-                ProcessException(ex, Token, $"data (gallery-dl images) downloading error [{bUrl}]")
+                ProcessException(ex, Token, $"data (gallery-dl images) downloading error [{bUrl.URL}]")
             Finally
                 Responser.Headers.Remove(PwsHeader)
             End Try
         End Sub
 #End Region
 #Region "Gallery-DL Support"
+        Private Structure GDLURL : Implements IRegExCreator
+            Friend URL As String
+            Friend BoardId As String
+            Public Shared Widening Operator CType(ByVal u As String) As GDLURL
+                Return New GDLURL With {.URL = u}
+            End Operator
+            Public Shared Widening Operator CType(ByVal u As GDLURL) As String
+                Return u.URL
+            End Operator
+            Private Function CreateFromArray(ByVal ParamsArray() As String) As Object Implements IRegExCreator.CreateFromArray
+                If ParamsArray.ListExists(2) Then
+                    Dim u$ = ParamsArray(0).StringTrim.StringTrimEnd("/"), u2$
+                    If Not u.IsEmptyString Then
+                        u2 = ParamsArray(1).StringTrim
+                        If Not u2.IsEmptyString AndAlso u2.StartsWith("GET", StringComparison.OrdinalIgnoreCase) Then
+                            u2 = u2.Remove(0, 3).StringTrim.StringTrimStart("/")
+                            If Not u2.IsEmptyString Then URL = $"{u}/{u2}"
+                        End If
+                    End If
+                End If
+                Return Me
+            End Function
+            Public Shared Operator =(ByVal x As GDLURL, ByVal y As GDLURL) As Boolean
+                Return x.URL = y.URL
+            End Operator
+            Public Shared Operator <>(ByVal x As GDLURL, ByVal y As GDLURL) As Boolean
+                Return Not x.URL = y.URL
+            End Operator
+            Public Overrides Function ToString() As String
+                Return URL
+            End Function
+            Public Overrides Function Equals(ByVal Obj As Object) As Boolean
+                Return URL = CType(Obj, String)
+            End Function
+        End Structure
         Private Class GDLBatch : Inherits GDL.GDLBatch
             Private ReadOnly Property Source As UserData
             Private ReadOnly IsBoardsRequested As Boolean
@@ -289,14 +376,30 @@ Namespace API.Pinterest
                 End If
             End Function
         End Class
-        Private Function GetDataFromGalleryDL(ByVal URL As String, ByVal IsBoardsRequested As Boolean, ByVal Token As CancellationToken) As List(Of String)
+        Private Function GetDataFromGalleryDL(ByVal URL As String, ByVal IsBoardsRequested As Boolean, ByVal Token As CancellationToken) As List(Of GDLURL)
             Dim command$ = $"""{Settings.GalleryDLFile.File}"" --verbose --simulate "
             Try
                 If Not URL.IsEmptyString Then
+                    Dim urls As New List(Of GDLURL)
+                    Dim u As GDLURL
+                    Dim s$ = String.Empty
                     If MySettings.CookiesNetscapeFile.Exists Then command &= $"--cookies ""{MySettings.CookiesNetscapeFile}"" "
                     command &= URL
                     Using batch As New GDLBatch(Me, IsBoardsRequested, Token)
-                        Return GetUrlsFromGalleryDl(batch, command)
+                        With batch
+                            .Execute(command)
+                            If .ErrorOutputData.Count > 0 Then
+                                For Each eValue$ In .ErrorOutputData
+                                    s = CStr(RegexReplace(eValue, SubBoardRegEx)).IfNullOrEmpty(s)
+                                    u = RegexFields(Of GDLURL)(eValue, {GdlUrlPattern}, {1, 2}, EDP.ReturnValue).ListIfNothing.FirstOrDefault
+                                    If Not u.URL.IsEmptyString Then
+                                        If Not s.IsEmptyString Then u.BoardId = s
+                                        urls.Add(u)
+                                    End If
+                                Next
+                                Return urls
+                            End If
+                        End With
                     End Using
                 End If
                 Return Nothing

SCrawler/API/PornHub/Declarations.vb

@@ -36,11 +36,12 @@ Namespace API.PornHub
         Friend ReadOnly Regex_Gif_UrlName As RParams = RParams.DMS("""name"":.*?""([^""]*)""[^\}]+?""contentUrl"":.*?""([^""]+)""", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
 #End Region
 #Region "Declarations photo"
-        Friend ReadOnly Regex_Photo_ModelHub_PhotoBlocks As RParams = RParams.DM("var PHOTOS_ARRAY_(\d+) = \{[\r\n\s]*?(urls:.*?\[[^]]*\])", 0, RegexReturn.List, EDP.ReturnValue)
         Friend ReadOnly Regex_Photo_PornHub_PhotoBlocks As RParams = RParams.DM("photoAlbumListContainer[\r\n\s\S]+?title=""([^""]+)""[\r\n\s\S]+?a href=""(/album/\d+)""", 0, RegexReturn.List)
-        Friend ReadOnly Regex_Photo_PornHub_AlbumPhotoArr As RParams = RParams.DMS("\<a href=""(/photo/\d+)""", 1, RegexReturn.List, EDP.ReturnValue,
+        Friend ReadOnly Regex_Photo_PornHub_PhotoBlocks2 As RParams = RParams.DM("albumInfoTitle"" href=""([^""]+)""\>([^\<]+)", 0, RegexReturn.List)
+        Friend ReadOnly Regex_Photo_PornHub_AlbumPhotoArr As RParams = RParams.DMS("href=""(/photo/\d+)""", 1, RegexReturn.List, EDP.ReturnValue,
                                                                                    CType(Function(Input$) If(Input.IsEmptyString, String.Empty, $"https://www.pornhub.com{Input.Trim}"), Func(Of String, String)))
-        Friend ReadOnly Regex_Photo_PornHub_SinglePhoto As RParams = RParams.DMS("(?<!thumbImage.+?)<img src=""(https://[^""]+\d+[^""]+)""", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_Photo_PornHub_SinglePhoto As RParams = RParams.DM("data-image=""([^""]+)""\s*src=""([^""]+)""", 0, RegexReturn.ListByMatch, EDP.ReturnValue)
+        Friend ReadOnly Regex_Photo_PornHub_SinglePhoto2 As RParams = RParams.DMS("image:src"" content=""([^""]+)""", 1, EDP.ReturnValue)
         Friend ReadOnly Regex_Photo_File As RParams = RParams.DM("\d+\.[\w]{3,4}", 0, EDP.ReturnValue)
 #End Region
     End Module

SCrawler/API/PornHub/SiteSettings.vb

@@ -29,10 +29,6 @@ Namespace API.PornHub
         Friend ReadOnly Property DownloadGifs As PropertyValue
         <PropertyOption(ControlText:="Download GIFs as mp4", ControlToolTip:="Download gifs in 'mp4' format instead of native 'webm'"), PXML, PClonable>
         Friend ReadOnly Property DownloadGifsAsMp4 As PropertyValue
-        <PropertyOption(ControlText:="Photo ModelHub only",
-                        ControlToolTip:="Download photo only from ModelHub. Prornstar photos hosted on PornHub itself will not be downloaded." & vbCr &
-                                        "Attention! Downloading photos hosted on PornHub is a very heavy job."), PXML, PClonable>
-        Friend ReadOnly Property DownloadPhotoOnlyFromModelHub As PropertyValue
         <PropertyOption(ControlText:=DeclaredNames.SavedPostsUserNameCaption, ControlToolTip:=DeclaredNames.SavedPostsUserNameToolTip), PXML, PClonable(Clone:=False)>
         Friend ReadOnly Property SavedPostsUserName As PropertyValue
 #End Region
@@ -48,7 +44,6 @@ Namespace API.PornHub
             DownloadFavorite = New PropertyValue(False)
             DownloadGifsAsMp4 = New PropertyValue(True)
             DownloadGifs = New PropertyValue(CInt(CheckState.Indeterminate), GetType(Integer))
-            DownloadPhotoOnlyFromModelHub = New PropertyValue(True)
             SavedPostsUserName = New PropertyValue(String.Empty, GetType(String))
 
             _SubscriptionsAllowed = True

SCrawler/API/PornHub/UserData.vb

@@ -20,15 +20,12 @@ Namespace API.PornHub
 #Region "Declarations"
 #Region "XML names"
         Private Const Name_PersonType As String = "PersonType"
-        Private Const Name_NameTrue As String = "NameTrue"
-        Private Const Name_PhotoPageModel As String = "PhotoPageModel"
         Private Const Name_DownloadUHD As String = "DownloadUHD"
         Private Const Name_DownloadUploaded As String = "DownloadUploaded"
         Private Const Name_DownloadTagged As String = "DownloadTagged"
         Private Const Name_DownloadPrivate As String = "DownloadPrivate"
         Private Const Name_DownloadFavorite As String = "DownloadFavorite"
         Private Const Name_DownloadGifs As String = "DownloadGifs"
-        Private Const Name_DownloadPhotoOnlyFromModelHub As String = "DownloadPhotoOnlyFromModelHub"
 #End Region
 #Region "Structures"
         Private Structure FlashVar : Implements IRegExCreator
@@ -98,11 +95,6 @@ Namespace API.PornHub
         End Structure
 #End Region
 #Region "Enums"
-        Private Enum PhotoPageModels As Integer
-            Undefined = 0
-            PornHubPage = 1
-            ModelHubPage = 2
-        End Enum
         Private Enum VideoTypes
             Undefined
             Uploaded
@@ -121,7 +113,6 @@ Namespace API.PornHub
 #End Region
 #Region "Person"
         Friend Property PersonType As String
-        Friend Property NameTrue As String
         Friend Overrides Property FriendlyName As String
             Get
                 If _FriendlyName.IsEmptyString Then Return NameTrue Else Return _FriendlyName
@@ -137,14 +128,12 @@ Namespace API.PornHub
                 Return IsUser Or SiteMode = SiteModes.Playlists
             End Get
         End Property
-        Private Property PhotoPageModel As PhotoPageModels = PhotoPageModels.Undefined
         Friend Property DownloadUHD As Boolean = False
         Friend Property DownloadUploaded As Boolean = True
         Friend Property DownloadTagged As Boolean = False
         Friend Property DownloadPrivate As Boolean = False
         Friend Property DownloadFavorite As Boolean = False
         Friend Property DownloadGifs As Boolean
-        Friend Property DownloadPhotoOnlyFromModelHub As Boolean = True
         Friend Overrides ReadOnly Property IsUser As Boolean
             Get
                 Return SiteMode = SiteModes.User
@@ -182,7 +171,6 @@ Namespace API.PornHub
                     DownloadPrivate = .DownloadPrivate
                     DownloadFavorite = .DownloadFavorite
                     DownloadGifs = .DownloadGifs
-                    DownloadPhotoOnlyFromModelHub = .DownloadPhotoOnlyFromModelHub
                     QueryString = .QueryString
                 End With
             End If
@@ -244,29 +232,23 @@ Namespace API.PornHub
             With Container
                 If Loading Then
                     PersonType = .Value(Name_PersonType)
-                    NameTrue = .Value(Name_NameTrue)
-                    PhotoPageModel = .Value(Name_PhotoPageModel).FromXML(Of Integer)(PhotoPageModels.Undefined)
                     DownloadUHD = .Value(Name_DownloadUHD).FromXML(Of Boolean)(False)
                     DownloadUploaded = .Value(Name_DownloadUploaded).FromXML(Of Boolean)(True)
                     DownloadTagged = .Value(Name_DownloadTagged).FromXML(Of Boolean)(False)
                     DownloadPrivate = .Value(Name_DownloadPrivate).FromXML(Of Boolean)(False)
                     DownloadFavorite = .Value(Name_DownloadFavorite).FromXML(Of Boolean)(False)
                     DownloadGifs = .Value(Name_DownloadGifs).FromXML(Of Integer)(False)
-                    DownloadPhotoOnlyFromModelHub = .Value(Name_DownloadPhotoOnlyFromModelHub).FromXML(Of Boolean)(True)
                     SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
                     UpdateUserOptions()
                 Else
                     If UpdateUserOptions() Then .Value(Name_LabelsName) = LabelsString
                     .Add(Name_PersonType, PersonType)
-                    .Add(Name_NameTrue, NameTrue)
-                    .Add(Name_PhotoPageModel, CInt(PhotoPageModel))
                     .Add(Name_DownloadUHD, DownloadUHD.BoolToInteger)
                     .Add(Name_DownloadUploaded, DownloadUploaded.BoolToInteger)
                     .Add(Name_DownloadTagged, DownloadTagged.BoolToInteger)
                     .Add(Name_DownloadPrivate, DownloadPrivate.BoolToInteger)
                     .Add(Name_DownloadFavorite, DownloadFavorite.BoolToInteger)
                     .Add(Name_DownloadGifs, DownloadGifs.BoolToInteger)
-                    .Add(Name_DownloadPhotoOnlyFromModelHub, DownloadPhotoOnlyFromModelHub.BoolToInteger)
                     .Add(Name_SiteMode, CInt(SiteMode))
 
                     'Debug.WriteLine(GetNonUserUrl(0))
@@ -283,6 +265,7 @@ Namespace API.PornHub
         Private _PageVideosRepeat As Integer = 0
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
+                UpdateM3U8URLS = False
                 PlaylistToken = String.Empty
                 Responser.ResetStatus()
                 _PageVideosRepeat = 0
@@ -295,7 +278,6 @@ Namespace API.PornHub
 
                 Dim limit% = If(DownloadTopCount, -1)
                 If DownloadVideos Then
-
                     If SiteMode = SiteModes.Playlists Then
                         Responser.Mode = Responser.Modes.Default
                         GetPlaylistToken(Token)
@@ -519,25 +501,12 @@ Namespace API.PornHub
             Dim pFile$ = RegexReplace(URL, Regex_Photo_File)
             If Not pFile.IsEmptyString Then Return New SFile(pFile) Else Return File
         End Function
-        Private Const PhotoUrlPattern_ModelHub As String = "https://www.modelhub.com/{0}/photos"
         Private Const PhotoUrlPattern_PornHub As String = "https://www.pornhub.com/{0}/{1}/photos"
         Private Sub DownloadUserPhotos(ByVal Token As CancellationToken)
             Try
                 If IsSavedPosts Then
                     DownloadUserPhotos_SavedPosts(Token)
-                ElseIf PersonType = PersonTypeModel Then
+                Else
-                    If PhotoPageModel = PhotoPageModels.Undefined Then
-                        If DownloadUserPhotos_ModelHub(Token) Then PhotoPageModel = PhotoPageModels.ModelHubPage
-                        ThrowAny(Token)
-                        If PhotoPageModel = PhotoPageModels.Undefined AndAlso Not DownloadPhotoOnlyFromModelHub AndAlso
-                           DownloadUserPhotos_PornHub(Token) Then PhotoPageModel = PhotoPageModels.PornHubPage
-                    Else
-                        Select Case PhotoPageModel
-                            Case PhotoPageModels.ModelHubPage : DownloadUserPhotos_ModelHub(Token)
-                            Case PhotoPageModels.PornHubPage : If Not DownloadPhotoOnlyFromModelHub Then DownloadUserPhotos_PornHub(Token)
-                        End Select
-                    End If
-                ElseIf Not DownloadPhotoOnlyFromModelHub Then
                     DownloadUserPhotos_PornHub(Token)
                 End If
                 ThrowAny(Token)
@@ -545,48 +514,6 @@ Namespace API.PornHub
                 ProcessException(ex, Token, "photos downloading error")
             End Try
         End Sub
-        Private Function DownloadUserPhotos_ModelHub(ByVal Token As CancellationToken) As Boolean
-            Dim URL$ = String.Empty
-            Try
-                Dim j As EContainer
-                Dim jErr As New ErrorsDescriber(EDP.SendToLog + EDP.ReturnValue)
-                Dim albumName$
-                If PersonType = PersonTypeModel Then
-                    URL = String.Format(PhotoUrlPattern_ModelHub, NameTrue)
-                    Dim r$ = Responser.GetResponse(URL)
-                    If Not r.IsEmptyString Then
-                        Dim l As List(Of PhotoBlock) = RegexFields(Of PhotoBlock)(r, {Regex_Photo_ModelHub_PhotoBlocks}, {1, 2}, EDP.ReturnValue)
-                        If l.ListExists Then l.RemoveAll(Function(ll) ll.Data.IsEmptyString)
-                        If l.ListExists Then
-                            ProgressPre.ChangeMax(l.Count)
-                            Dim albumRegex As RParams = RParams.DMS("", 1, EDP.ReturnValue)
-                            For Each block As PhotoBlock In l
-                                ProgressPre.Perform()
-                                If Not _TempPostsList.Contains(block.AlbumID) Then _TempPostsList.Add(block.AlbumID) Else Continue For
-                                albumRegex.Pattern = "<li id=""" & block.AlbumID & """ class=""modelBox"">[\r\n\s]*?<div class=""modelPhoto"">[\r\n\s]*?\<[^\>]*?alt=""([^""]*)"""
-                                albumName = StringTrim(RegexReplace(r, albumRegex))
-                                If albumName.IsEmptyString Then albumName = block.AlbumID
-                                j = JsonDocument.Parse("{" & block.Data & "}", jErr)
-                                If Not j Is Nothing Then
-                                    If If(j("urls")?.Count, 0) > 0 Then
-                                        _TempMediaList.ListAddList(j("urls").Select(Function(jj) _
-                                                                   New UserMedia(jj.ItemF({0}).XmlIfNothingValue, UTypes.Picture) With {
-                                                                                 .SpecialFolder = $"Albums\{albumName}\",
-                                                                                 .File = CreatePhotoFile(.URL, .File)}), LNC)
-                                    End If
-                                    j.Dispose()
-                                End If
-                            Next
-                            l.Clear()
-                        End If
-                    End If
-                End If
-                Return True
-            Catch ex As Exception
-                ThrowAny(Token)
-                Return False
-            End Try
-        End Function
         Private Overloads Function DownloadUserPhotos_PornHub(ByVal Token As CancellationToken) As Boolean
             Try
                 Dim albumName$
@@ -594,6 +521,7 @@ Namespace API.PornHub
                 Dim r$ = Responser.GetResponse(String.Format(PhotoUrlPattern_PornHub, PersonType, NameTrue))
                 If Not r.IsEmptyString Then
                     Dim l As List(Of PhotoBlock) = RegexFields(Of PhotoBlock)(r, {Regex_Photo_PornHub_PhotoBlocks}, {2, 1}, EDP.ReturnValue)
+                    l.ListAddList(RegexFields(Of PhotoBlock)(r, {Regex_Photo_PornHub_PhotoBlocks2}, {1, 2}, EDP.ReturnValue))
                     If l.ListExists Then l.RemoveAll(Function(ll) ll.AlbumID.IsEmptyString)
                     If l.ListExists Then
                         ProgressPre.ChangeMax(l.Count)
@@ -618,6 +546,14 @@ Namespace API.PornHub
                 Return False
             End Try
         End Function
+        Private Function DownloadUserPhotos_PornHub_ParseSinglePhoto(ByVal r As String) As String
+            Dim url$ = String.Empty
+            With DirectCast(RegexReplace(r, Regex_Photo_PornHub_SinglePhoto), List(Of String))
+                If .ListExists(3) Then url = .Item(2).IfNullOrEmpty(.Item(1)).StringTrim
+            End With
+            If url.IsEmptyString Then url = RegexReplace(r, Regex_Photo_PornHub_SinglePhoto2)
+            Return url
+        End Function
         Private Overloads Function DownloadUserPhotos_PornHub(ByVal Page As Integer, ByVal AlbumID As String, ByVal AlbumName As String,
                                                               ByVal Token As CancellationToken) As Boolean
             Try
@@ -633,7 +569,7 @@ Namespace API.PornHub
                             Try
                                 r = Responser.GetResponse(url)
                                 If Not r.IsEmptyString Then
-                                    url = RegexReplace(r, Regex_Photo_PornHub_SinglePhoto)
+                                    url = DownloadUserPhotos_PornHub_ParseSinglePhoto(r)
                                     If Not url.IsEmptyString Then _
                                        _TempMediaList.ListAddValue(New UserMedia(url, UTypes.Picture) With {
                                                                    .SpecialFolder = $"Albums\{AlbumName}\",
@@ -679,7 +615,7 @@ Namespace API.PornHub
                                     Try
                                         r = Responser.GetResponse(m.URL)
                                         If Not r.IsEmptyString Then
-                                            NewUrl = RegexReplace(r, Regex_Photo_PornHub_SinglePhoto)
+                                            NewUrl = DownloadUserPhotos_PornHub_ParseSinglePhoto(r)
                                             If Not NewUrl.IsEmptyString Then
                                                 m.URL = NewUrl
                                                 pFile = RegexReplace(NewUrl, Regex_Photo_File)
@@ -852,11 +788,34 @@ Namespace API.PornHub
         End Sub
 #End Region
 #Region "Download content"
+        Private UpdateM3U8URLS As Boolean = False
+        Private UpdateM3U8URLS_Error As Boolean = False
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
-            DownloadContentDefault(Token)
+            Try : DownloadContentDefault(Token) : Finally : UpdateM3U8URLS = False : End Try
         End Sub
-        Protected Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile, ByVal Token As CancellationToken) As SFile
+        Protected Overloads Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile,
-            Return M3U8.Download(URL, Responser, DestinationFile, DownloadUHD, Token, Progress, Not IsSingleObjectDownload)
+                                                            ByVal Token As CancellationToken) As SFile
+            UpdateM3U8URLS_Error = False
+            Return DownloadM3U8(URL, Media, DestinationFile, Token, UpdateM3U8URLS)
+        End Function
+        Private Overloads Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile,
+                                                ByVal Token As CancellationToken, ByVal Second As Boolean) As SFile
+            Try
+                If Second Then
+                    Dim r$ = Responser.Curl(Media.URL_BASE,, EDP.ReturnValue)
+                    If Not r.IsEmptyString Then Media.URL = CreateVideoURL(r).IfNullOrEmpty(URL) : URL = Media.URL
+                End If
+                Dim f As SFile = M3U8.Download(URL, Responser, DestinationFile, DownloadUHD, Token, Progress, Not IsSingleObjectDownload)
+                If Not f.Exists And Not Second Then UpdateM3U8URLS = True : f = DownloadM3U8(URL, Media, DestinationFile, Token, True)
+                Return f
+            Catch ex As Exception
+                If Not UpdateM3U8URLS_Error Then
+                    UpdateM3U8URLS_Error = True
+                    Thread.Sleep(1000)
+                    Return DownloadM3U8(URL, Media, DestinationFile, Token, True)
+                End If
+                Return Nothing
+            End Try
         End Function
 #End Region
 #Region "CreateVideoURL"
@@ -953,6 +912,7 @@ Namespace API.PornHub
 #End Region
 #Region "DownloadSingleObject"
         Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
+            UpdateM3U8URLS = False
             _TempMediaList.Add(New UserMedia(Data.URL, UTypes.VideoPre))
             ReparseVideo(Token, True, Data)
         End Sub

SCrawler/API/PornHub/UserExchangeOptions.vb

@@ -21,8 +21,6 @@ Namespace API.PornHub
         Friend Property DownloadFavorite As Boolean
         <PSetting(Caption:="Download gifs")>
         Friend Property DownloadGifs As Boolean
-        <PSetting(NameOf(SiteSettings.DownloadPhotoOnlyFromModelHub), NameOf(MySettings), Caption:="Download photo only from ModelHub")>
-        Friend Property DownloadPhotoOnlyFromModelHub As Boolean
         Private ReadOnly Property MySettings As SiteSettings
         Friend Sub New(ByVal u As UserData)
             DownloadUHD = u.DownloadUHD
@@ -31,7 +29,6 @@ Namespace API.PornHub
             DownloadPrivate = u.DownloadPrivate
             DownloadFavorite = u.DownloadFavorite
             DownloadGifs = u.DownloadGifs
-            DownloadPhotoOnlyFromModelHub = u.DownloadPhotoOnlyFromModelHub
             QueryString = u.QueryString
             MySettings = u.HOST.Source
         End Sub
@@ -43,7 +40,6 @@ Namespace API.PornHub
             DownloadPrivate = s.DownloadPrivate.Value
             DownloadFavorite = s.DownloadFavorite.Value
             DownloadGifs = Not v = CheckState.Unchecked
-            DownloadPhotoOnlyFromModelHub = s.DownloadPhotoOnlyFromModelHub.Value
             MySettings = s
         End Sub
     End Class

SCrawler/API/Reddit/SiteSettings.vb

@@ -172,7 +172,7 @@ Namespace API.Reddit
             End If
         End Function
         Friend Overrides Function Available(ByVal What As Download, ByVal Silent As Boolean) As Boolean
-            Return AvailableTrueValue(What)
+            Return AvailableTrueValue(What) AndAlso UpdateTokenIfRequired()
         End Function
         Private Function AvailableTrueValue(ByVal What As Download) As Boolean
             Return Not What = Download.SavedPosts OrElse (Responser.CookiesExists And ACheck(SavedPostsUserName.Value))
@@ -199,7 +199,7 @@ Namespace API.Reddit
             End If
         End Function
         Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
-            With DirectCast(User, UserData) : Return String.Format(UrlPatternUser, IIf(.IsChannel, ChannelOption, "user"), .TrueName) : End With
+            With DirectCast(User, UserData) : Return String.Format(UrlPatternUser, IIf(.IsChannel, ChannelOption, "user"), .NameTrue) : End With
         End Function
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
             If Not Media.Post.ID.IsEmptyString Then
@@ -250,6 +250,7 @@ Namespace API.Reddit
             Return False
         End Function
         Private Function UpdateTokenIfRequired() As Boolean
+            UpdateRedGifsToken()
             If (CBool(UseTokenForTimelines.Value) Or CBool(UseTokenForSavedPosts.Value)) AndAlso CredentialsExists Then
                 If CDate(BearerTokenDateUpdate.Value).AddMinutes(TokenUpdateInterval.Value) <= Now Then Return UpdateToken()
             End If

SCrawler/API/Reddit/UserData.vb

@@ -42,7 +42,6 @@ Namespace API.Reddit
             End Get
         End Property
         Friend Property IsChannel As Boolean = False
-        Friend Property TrueName As String = String.Empty
         Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
             Get
                 Return {CannelsLabelName, CannelsLabelName_ChannelsForm, UserLabelName}
@@ -173,16 +172,16 @@ Namespace API.Reddit
 #End Region
 #Region "Load and Update user info"
         Private Function UpdateNames() As Boolean
-            If TrueName.IsEmptyString Then
+            If NameTrue(True).IsEmptyString Then
                 Dim n$() = Name.Split("@")
                 If n.ListExists Then
                     If n.Length = 2 Then
-                        TrueName = n(0)
+                        NameTrue = n(0)
                         IsChannel = True
                     ElseIf IsChannel Then
-                        TrueName = Name
+                        NameTrue = Name
                     Else
-                        TrueName = n(0)
+                        NameTrue = n(0)
                     End If
                 End If
                 If Not IsSavedPosts Then
@@ -201,7 +200,6 @@ Namespace API.Reddit
                     ViewMode = .Value(Name_ViewMode).FromXML(Of Integer)(CInt(CView.New))
                     ViewPeriod = .Value(Name_ViewPeriod).FromXML(Of Integer)(CInt(CPeriod.All))
                     IsChannel = .Value(Name_IsChannel).FromXML(Of Boolean)(False)
-                    TrueName = .Value(Name_TrueName)
                     RedGifsAccount = .Value(Name_RedGifsAccount)
                     RedditAccount = .Value(Name_RedditAccount)
                     UpdateNames()
@@ -210,7 +208,7 @@ Namespace API.Reddit
                     .Add(Name_ViewMode, CInt(ViewMode))
                     .Add(Name_ViewPeriod, CInt(ViewPeriod))
                     .Add(Name_IsChannel, IsChannel.BoolToInteger)
-                    .Add(Name_TrueName, TrueName)
+                    .Add(Name_TrueName, NameTrue(True))
                     .Add(Name_RedGifsAccount, RedGifsAccount)
                     .Add(Name_RedditAccount, RedditAccount)
                 End If
@@ -230,7 +228,7 @@ Namespace API.Reddit
             If CreatedByChannel And Settings.FromChannelDownloadTopUse And Settings.FromChannelDownloadTop > 0 Then _
                DownloadTopCount = Settings.FromChannelDownloadTop.Value
             If IsChannel Or IsSavedPosts Then UseMD5Comparison = False
-            If IsSavedPosts Then TrueName = MySiteSettings.SavedPostsUserName.Value
+            If IsSavedPosts Then NameTrue = MySiteSettings.SavedPostsUserName.Value
             UpdateNames()
             If IsChannelForm Then
                 UseMD5Comparison = False
@@ -310,10 +308,10 @@ Namespace API.Reddit
                     Dim NewPostDetected As Boolean = False
                     Dim ExistsDetected As Boolean = False
                     Dim IsCrossPost As Predicate(Of EContainer) = Function(e) Not e.Value(Node_CrosspostRootId).IsEmptyString Or Not e.Value(Node_CrosspostParentId).IsEmptyString Or Not e.Value(Node_CrosspostParent).IsEmptyString
-                    Dim CheckNode As Predicate(Of EContainer) = Function(e) Not ParseUserMediaOnly OrElse If(e("author")?.Value, "/").ToLower.Equals(TrueName.StringToLower)
+                    Dim CheckNode As Predicate(Of EContainer) = Function(e) Not ParseUserMediaOnly OrElse If(e("author")?.Value, "/").ToLower.Equals(NameTrue.StringToLower)
                     Dim _PostID As Func(Of String) = Function() PostTmp.IfNullOrEmpty(PostID)
 
-                    URL = $"https://gateway.reddit.com/desktopapi/v1/user/{TrueName}/posts?rtj=only&allow_quarantined=true&allow_over18=1&include=identity&after={POST}&dist=25&sort={View}&t={Period}&layout=classic"
+                    URL = $"https://gateway.reddit.com/desktopapi/v1/user/{NameTrue}/posts?rtj=only&allow_quarantined=true&allow_over18=1&include=identity&after={POST}&dist=25&sort={View}&t={Period}&layout=classic"
                     ThrowAny(Token)
                     Dim r$ = Responser.GetResponse(URL)
                     If Not r.IsEmptyString Then
@@ -394,10 +392,10 @@ Namespace API.Reddit
                     Dim lDate As Date?
 
                     If IsSavedPosts Then
-                        URL = $"https://www.reddit.com/user/{TrueName}/saved.json?after={POST}"
+                        URL = $"https://www.reddit.com/user/{NameTrue}/saved.json?after={POST}"
                         If Not POST.IsEmptyString Then Thread.Sleep(savedPostsSleepTimer)
                     Else
-                        URL = $"https://reddit.com/r/{TrueName}/{View}.json?allow_quarantined=true&allow_over18=1&include=identity&after={POST}&dist=25&sort={View}&t={Period}&layout=classic"
+                        URL = $"https://reddit.com/r/{NameTrue}/{View}.json?allow_quarantined=true&allow_over18=1&include=identity&after={POST}&dist=25&sort={View}&t={Period}&layout=classic"
                     End If
 
                     ThrowAny(Token)
@@ -480,7 +478,7 @@ Namespace API.Reddit
         Private Sub GetUserInfo()
             Try
                 If Not IsSavedPosts And ChannelInfo Is Nothing Then
-                    Dim r$ = Responser.GetResponse($"https://reddit.com/{IIf(IsChannel, "r", "user")}/{TrueName}/about.json",, EDP.ReturnValue)
+                    Dim r$ = Responser.GetResponse($"https://reddit.com/{IIf(IsChannel, "r", "user")}/{NameTrue}/about.json",, EDP.ReturnValue)
                     If Not r.IsEmptyString Then
                         Using j As EContainer = JsonDocument.Parse(r)
                             If Not j Is Nothing AndAlso j.Contains({"data", "subreddit"}) Then
@@ -489,20 +487,10 @@ Namespace API.Reddit
                                     UserSiteNameUpdate(.Value("title"))
                                     UserDescriptionUpdate(.Value("public_description"))
                                     Dim dir As SFile = MyFile.CutPath
-                                    Dim __getFile As Action(Of String) = Sub(ByVal img As String)
+                                    Dim fileCrFunc As Func(Of String, SFile) = Function(img) CreateFileFromUrl(img)
-                                                                             If Not img.IsEmptyString Then
-                                                                                 Dim f As SFile = CreateFileFromUrl(img)
-                                                                                 If Not f.Name.IsEmptyString Then
-                                                                                     If f.Extension.IsEmptyString Then f.Extension = "jpg"
-                                                                                     f.Path = dir.Path
-                                                                                     If Not f.Exists Then GetWebFile(img, f, EDP.ReturnValue)
-                                                                                     If f.Exists Then IconBannerDownloaded = True
-                                                                                 End If
-                                                                             End If
-                                                                         End Sub
                                     If DownloadIconBanner Then
-                                        __getFile.Invoke(.Value("icon_img"))
+                                        SimpleDownloadAvatar(.Value("icon_img"), fileCrFunc)
-                                        __getFile.Invoke(.Value("banner_img"))
+                                        SimpleDownloadAvatar(.Value("banner_img"), fileCrFunc)
                                     End If
                                 End With
                             End If

SCrawler/API/ThisVid/UserData.vb

@@ -44,7 +44,6 @@ Namespace API.ThisVid
         Friend Property DownloadPrivate As Boolean = True
         Friend Property DownloadFavourite As Boolean = False
         Friend Property DifferentFolders As Boolean = True
-        Friend Property TrueName As String = String.Empty
         Friend Property SiteMode As SiteModes = SiteModes.User
         Private Property Arguments As String = String.Empty
         Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
@@ -80,7 +79,7 @@ Namespace API.ThisVid
             If Not Force OrElse (Not SiteMode = SiteModes.User AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
                 Dim eObj As Plugin.ExchangeOptions = Nothing
                 If Force Then eObj = MySettings.IsMyUser(NewUrl)
-                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And TrueName.IsEmptyString) Then
+                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And NameTrue(True).IsEmptyString) Then
                     Dim n$() = If(Force, eObj.UserName, Name).Split("@")
                     If n.ListExists(2) Then
 
@@ -98,8 +97,8 @@ Namespace API.ThisVid
                         End If
                         __TrueName = n(1)
 
-                        If Force AndAlso (Not TrueName = __TrueName Or Not SiteMode = __Mode) Then
+                        If Force AndAlso (Not NameTrue(True) = __TrueName Or Not SiteMode = __Mode) Then
-                            If ValidateChangeSearchOptions(ToStringForLog, $"{__Mode}: {__TrueName}", $"{SiteMode}: {TrueName}") Then
+                            If ValidateChangeSearchOptions(ToStringForLog, $"{__Mode}: {__TrueName}", $"{SiteMode}: {NameTrue(True)}") Then
                                 __ForceApply = True
                             Else
                                 Return False
@@ -109,21 +108,21 @@ Namespace API.ThisVid
                         Arguments = __Arguments
                         Options = If(Force, eObj.Options, Options)
                         If Not Force Then
-                            TrueName = __TrueName
+                            NameTrue = __TrueName
                             SiteMode = __Mode
                             Settings.Labels.Add(SearchRequestLabelName)
                             Labels.ListAddValue(SearchRequestLabelName, LNC)
                             Labels.Sort()
-                            UserSiteName = $"{SiteMode}: {TrueName}"
+                            UserSiteName = $"{SiteMode}: {NameTrue}"
                             If FriendlyName.IsEmptyString Then FriendlyName = UserSiteName
                         ElseIf Force And __ForceApply Then
-                            TrueName = __TrueName
+                            NameTrue = __TrueName
                             SiteMode = __Mode
                         End If
                         Return True
                     Else
                         SiteMode = SiteModes.User
-                        TrueName = Name
+                        NameTrue = Name
                     End If
                 End If
             End If
@@ -136,7 +135,6 @@ Namespace API.ThisVid
                     DownloadPrivate = .Value(Name_DownloadPrivate).FromXML(Of Boolean)(True)
                     DownloadFavourite = .Value(Name_DownloadFavourite).FromXML(Of Boolean)(False)
                     DifferentFolders = .Value(Name_DifferentFolders).FromXML(Of Boolean)(True)
-                    TrueName = .Value(Name_TrueName)
                     SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
                     Arguments = .Value(Name_Arguments)
                     UpdateUserOptions()
@@ -150,7 +148,7 @@ Namespace API.ThisVid
                     .Add(Name_DownloadPrivate, DownloadPrivate.BoolToInteger)
                     .Add(Name_DownloadFavourite, DownloadFavourite.BoolToInteger)
                     .Add(Name_DifferentFolders, DifferentFolders.BoolToInteger)
-                    .Add(Name_TrueName, TrueName)
+                    .Add(Name_TrueName, NameTrue(True))
                     .Add(Name_SiteMode, CInt(SiteMode))
                     .Add(Name_Arguments, Arguments)
 
@@ -259,18 +257,18 @@ Namespace API.ThisVid
             Dim url$ = String.Empty
             Select Case SiteMode
                 Case SiteModes.Tags
-                    url = $"https://thisvid.com/{SiteSettings.P_Tags}/{TrueName}/"
+                    url = $"https://thisvid.com/{SiteSettings.P_Tags}/{NameTrue}/"
                     If Not Arguments.IsEmptyString Then url &= $"{Arguments}/"
                     If Page > 1 Then url &= $"{Page}/"
                 Case SiteModes.Categories
-                    url = $"https://thisvid.com/{SiteSettings.P_Categories}/{TrueName}/"
+                    url = $"https://thisvid.com/{SiteSettings.P_Categories}/{NameTrue}/"
                     If Not Arguments.IsEmptyString Then url &= $"{Arguments}/"
                     If Page > 1 Then url &= $"{Page}/"
                 Case SiteModes.Search
                     If Not Arguments.IsEmptyString Then
                         url = $"https://thisvid.com/{Arguments}/"
                         If Page > 1 Then url &= $"{Page}/"
-                        url &= $"?q={TrueName}/"
+                        url &= $"?q={NameTrue}/"
                     End If
             End Select
             Return url

SCrawler/API/ThreadsNet/SiteSettings.vb

@@ -15,7 +15,7 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Imports IG = SCrawler.API.Instagram.SiteSettings
 Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.ThreadsNet
-    <Manifest("AndyProgram_ThreadsNet"), SavedPosts, SeparatedTasks(1)>
+    <Manifest("AndyProgram_ThreadsNet"), SavedPosts, SeparatedTasks(1), SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
 #Region "Authorization"
@@ -155,6 +155,7 @@ Namespace API.ThreadsNet
             UrlPatternUser = "https://www.threads.net/@{0}"
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "threads.net/@"), 1)
             ImageVideoContains = "threads.net"
+            UserOptionsType = GetType(EditorExchangeOptionsBase)
         End Sub
 #End Region
 #Region "UpdateResponserData"
@@ -180,9 +181,6 @@ Namespace API.ThreadsNet
         Friend Overrides Function BaseAuthExists() As Boolean
             Return Responser.CookiesExists And {HH_CSRF_TOKEN, HH_IG_APP_ID}.All(Function(v) ACheck(Of String)(v.Value)) And CBool(DownloadData_Impl.Value)
         End Function
-        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
-            Return String.Format(UrlPatternUser, DirectCast(User, UserData).NameTrue)
-        End Function
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
             Try
                 Dim code$ = DirectCast(User, UserData).GetPostCodeById(Media.Post.ID)

SCrawler/API/ThreadsNet/UserData.vb

@@ -7,6 +7,7 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports System.Threading
+Imports SCrawler.Plugin
 Imports SCrawler.API.Base
 Imports SCrawler.API.YouTube.Objects
 Imports PersonalUtilities.Functions.XML
@@ -51,9 +52,10 @@ Namespace API.ThreadsNet
 #End Region
 #Region "Exchange"
         Friend Overrides Function ExchangeOptionsGet() As Object
-            Return Nothing
+            Return New EditorExchangeOptionsBase(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptionsBase Then NameTrue = DirectCast(Obj, EditorExchangeOptionsBase).UserName
         End Sub
 #End Region
 #Region "Initializer"
@@ -73,12 +75,13 @@ Namespace API.ThreadsNet
         End Sub
         Private Sub DisableDownload()
             MySettings.DownloadData_Impl.Value = False
-            MyMainLOG = $"{Site} downloading is disabled until you update your credentials"
+            LogError(Nothing, $"{Site} downloading is disabled until you update your credentials")
         End Sub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             If CBool(MySettings.DownloadData_Impl.Value) Then
                 Dim errorFound As Boolean = False
                 Try
+                    _IdChanged = False
                     Responser.Method = "POST"
                     LoadSavePostsKV(True)
                     ResetBaseTokens()
@@ -103,9 +106,9 @@ Namespace API.ThreadsNet
                     End If
                     If IsSavedPosts Then
                         DefaultParser_ElemNode = {"node", "thread_items", 0, "post"}
-                        DownloadSavedPosts(String.Empty, Token)
+                        DownloadSavedPosts(String.Empty, 0, Token)
                     Else
-                        DownloadData(String.Empty, Token)
+                        DownloadData(String.Empty, 0, Token)
                     End If
                     If _TempMediaList.Count > 0 Then FirstLoadingDone = True : setMaxPostDate.Invoke(_TempMediaList)
                 Catch ex As Exception
@@ -151,12 +154,14 @@ Namespace API.ThreadsNet
                    Responser.Headers.Add(IGS.Header_CSRF_TOKEN, csrf)
             End If
         End Sub
-        Private Const GQL_Q As String = "https://www.threads.net/api/graphql?lsd={0}&fb_dtsg={1}&doc_id={2}&fb_api_req_friendly_name={3}&server_timestamps=true&variables={4}"
+        'Private Const GQL_Q As String = "https://www.threads.net/api/graphql?lsd={0}&fb_dtsg={1}&doc_id={2}&fb_api_req_friendly_name={3}&server_timestamps=true&variables={4}"
-        Private Const GQL_P_DOC_ID As String = "6371597506283707"
+        Private Const GQL_Q2 As String = "https://www.threads.net/graphql/query"
-        Private Const GQL_P_NAME As String = "BarcelonaProfileThreadsTabRefetchableQuery"
+        Private Const PayloadData As String = "lsd={0}&fb_dtsg={1}&doc_id={2}&fb_api_req_friendly_name={3}&server_timestamps=true&variables={4}"
-        Private Const GQL_S_DOC_ID_1 As String = "7758166704280174"
+        Private Const GQL_P_DOC_ID As String = "9039187972876777" '"8779269398849532" '"6371597506283707"
-        Private Const GQL_S_NAME_1 As String = "BarcelonaSavedPageViewerQuery"
+        Private Const GQL_P_NAME As String = "BarcelonaProfileThreadsTabRefetchableDirectQuery" '"BarcelonaProfileThreadsTabRefetchableQuery"
-        Private Const GQL_S_DOC_ID_2 As String = "8617275414954442"
+        'Private Const GQL_S_DOC_ID_1 As String = "9227844190587889" '"7758166704280174"
+        'Private Const GQL_S_NAME_1 As String = "BarcelonaSavedPageViewerQuery"
+        Private Const GQL_S_DOC_ID_2 As String = "9116629201788321" '"8617275414954442"
         Private Const GQL_S_NAME_2 As String = "BarcelonaSavedPageRefetchableQuery"
         Private Sub DownloadCheckCredentials()
             If Not Valid Then
@@ -166,106 +171,189 @@ Namespace API.ThreadsNet
             End If
             If Not Valid Then DisableDownload() : Throw New Plugin.ExitException("Some credentials are missing")
         End Sub
-        Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
+        Private Function CheckErrors(ByVal e As EContainer) As Boolean
-            Const var_init$ = """userID"":""{0}"""
+            Return e.ListExists AndAlso Not JsonErrorMessage(e).IsEmptyString
-            Const var_cursor$ = """after"":""{1}"",""before"":null,""first"":25,""last"":null,""userID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false"
+        End Function
-            Dim URL$ = String.Empty
+        Private Function JsonErrorMessage(ByVal e As EContainer) As String
+            Return e.ItemF({"errors", 0, "summary"})?.Value
+        End Function
+        Private Sub ProcessJsonErrorException(ByVal uex As JsonErrorException, Optional ByVal ThrowEx As Boolean = True)
+            If uex.UserNotFound Then
+                UserExists = False
+                _ForceSaveUserInfo = True
+                _ForceSaveUserInfoOnException = True
+            ElseIf ThrowEx Then
+                Throw New ExitException(uex.ErrMessage) With {.SimpleLogLine = True}
+            Else
+                LogError(Nothing, uex.ErrMessage)
+            End If
+        End Sub
+        Private Class JsonErrorException : Inherits Exception
+            Friend Property UserNotFound As Boolean = False
+            Private _ErrMessage As String = String.Empty
+            Public Property ErrMessage As String
+                Get
+                    Return _ErrMessage
+                End Get
+                Set(ByVal m As String)
+                    _ErrMessage = m
+                    UserNotFound = _ErrMessage.StringToLower = "not found"
+                End Set
+            End Property
+            Public Overrides ReadOnly Property Message As String
+                Get
+                    Return _ErrMessage
+                End Get
+            End Property
+            Friend Sub New()
+            End Sub
+            Friend Sub New(ByVal Message As String)
+                ErrMessage = Message
+            End Sub
+        End Class
+        Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Round As Integer, ByVal Token As CancellationToken)
+            'Const var_init$ = """userID"":""{0}"""
+            'Const var_cursor$ = """after"":""{1}"",""before"":null,""first"":25,""last"":null,""userID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false"
+            Const var_cursor2$ = """after"":{1},""before"":null,""first"":10,""last"":null,""userID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaHasSelfReplyContextrelayprovider"":false,""__relay_internal__pv__BarcelonaShareableListsrelayprovider"":true,""__relay_internal__pv__BarcelonaIsSearchDiscoveryEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaQuotedPostUFIEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaIsCrawlerrelayprovider"":false,""__relay_internal__pv__BarcelonaHasDisplayNamesrelayprovider"":false,""__relay_internal__pv__BarcelonaCanSeeSponsoredContentrelayprovider"":false,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowTagRedesignrelayprovider"":false,""__relay_internal__pv__BarcelonaIsInternalUserrelayprovider"":false"
             Try
                 DownloadCheckCredentials()
 
-                Responser.Method = "POST"
+                With Responser
-                Responser.Referer = $"https://www.threads.net/@{NameTrue}"
+                    .Method = "POST"
-                Responser.Headers.Add(GQL_HEADER_FB_LSD, Token_lsd)
+                    .Referer = $"https://www.threads.net/@{NameTrue}"
-                Responser.Headers.Add(GQL_HEADER_FB_FRINDLY_NAME, GQL_P_NAME)
+                    .ContentType = "application/x-www-form-urlencoded"
+                    With .Headers
+                        .Add(GQL_HEADER_FB_LSD, Token_lsd)
+                        .Add(GQL_HEADER_FB_FRINDLY_NAME, GQL_P_NAME)
+                    End With
+                End With
 
                 Dim nextCursor$ = String.Empty
                 Dim dataFound As Boolean = False
 
-                Dim vars$
+                Dim vars$ = String.Format(var_cursor2, ID, IIf(Cursor.IsEmptyString, "null", $"""{Cursor}"""))
-                If Cursor.IsEmptyString Then
+                'If Cursor.IsEmptyString Then
-                    vars = String.Format(var_init, ID)
+                '    vars = String.Format(var_init, ID)
-                Else
+                'Else
-                    vars = String.Format(var_cursor, ID, Cursor)
+                '    vars = String.Format(var_cursor, ID, Cursor)
-                End If
+                'End If
-                vars = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & vars & "}")
+                vars = String.Format(PayloadData, Token_lsd, Token_dtsg_Var, GQL_P_DOC_ID, GQL_P_NAME,
+                                     SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & vars & "}"))
 
-                URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_P_DOC_ID, GQL_P_NAME, vars)
+                'URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_P_DOC_ID, GQL_P_NAME, vars)
 
-                Using j As EContainer = GetDocument(URL, Token)
+                Using j As EContainer = GetDocument(GQL_Q2, vars, Token)
-                    If j.ListExists Then
+                    If Not CheckErrors(j) Then
-                        With j({"data", "mediaData"})
+                        If j.ListExists Then
-                            If .ListExists Then
+                            With j({"data", "mediaData"})
-                                nextCursor = .Value({"page_info"}, "end_cursor")
+                                If .ListExists Then
-                                With .Item({"edges"})
+                                    nextCursor = .Value({"page_info"}, "end_cursor")
-                                    If .ListExists Then dataFound = DefaultParser(.Self, Sections.Timeline, Token)
+                                    With .Item({"edges"})
-                                End With
+                                        If .ListExists Then dataFound = DefaultParser(.Self, Sections.Timeline, Token)
-                            End If
+                                    End With
-                        End With
+                                End If
+                            End With
+                        End If
+                    Else
+                        Throw New JsonErrorException(JsonErrorMessage(j))
                     End If
                 End Using
 
-                If dataFound And Not nextCursor.IsEmptyString Then DownloadData(nextCursor, Token)
+                If dataFound And Not nextCursor.IsEmptyString Then DownloadData(nextCursor, 0, Token)
+            Catch uex As JsonErrorException
+                If Round > 0 Then
+                    ProcessJsonErrorException(uex)
+                ElseIf Not _IdChanged AndAlso UpdateCredentials() Then
+                    DownloadData(Cursor, Round + 1, Token)
+                Else
+                    ProcessJsonErrorException(uex)
+                End If
+            Catch eex As ExitException
+                Throw eex
             Catch ex As Exception
-                ProcessException(ex, Token, $"data downloading error [{URL}]")
+                ProcessException(ex, Token, "data downloading error")
             End Try
         End Sub
-        Private Sub DownloadSavedPosts(ByVal Cursor As String, ByVal Token As CancellationToken)
+        Private Sub DownloadSavedPosts(ByVal Cursor As String, ByVal Round As Integer, ByVal Token As CancellationToken)
-            Const var_init$ = """__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
+            'Const var_init$ = """__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
-            Const var_cursor$ = """after"":""{0}"",""first"":25,""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
+            'Const var_cursor$ = """after"":""{0}"",""first"":25,""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsInlineReelsEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaUseCometVideoPlaybackEnginerelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsTextFragmentsEnabledForPostCaptionsrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true"
-            Dim URL$ = String.Empty
+            Const var_cursor2$ = """after"":{0},""first"":25,""__relay_internal__pv__BarcelonaQuotedPostUFIEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaHasSelfReplyContextrelayprovider"":false,""__relay_internal__pv__BarcelonaShareableListsrelayprovider"":true,""__relay_internal__pv__BarcelonaIsSearchDiscoveryEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaIsCrawlerrelayprovider"":false,""__relay_internal__pv__BarcelonaHasDisplayNamesrelayprovider"":false,""__relay_internal__pv__BarcelonaCanSeeSponsoredContentrelayprovider"":false,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowTagRedesignrelayprovider"":false,""__relay_internal__pv__BarcelonaIsInternalUserrelayprovider"":false"
             Try
                 DownloadCheckCredentials()
 
-                Responser.Method = "POST"
+                With Responser
-                Responser.Referer = "https://www.threads.net/"
+                    .Method = "POST"
-                Responser.Headers.Add(GQL_HEADER_FB_LSD, Token_lsd)
+                    .Referer = "https://www.threads.net/"
-                Responser.Headers.Add(GQL_HEADER_FB_FRINDLY_NAME, If(Cursor.IsEmptyString, GQL_S_NAME_1, GQL_S_NAME_2))
+                    .ContentType = "application/x-www-form-urlencoded"
+                    With .Headers
+                        .Add(GQL_HEADER_FB_LSD, Token_lsd)
+                        '.Add(GQL_HEADER_FB_FRINDLY_NAME, If(Cursor.IsEmptyString, GQL_S_NAME_1, GQL_S_NAME_2))
+                        .Add(GQL_HEADER_FB_FRINDLY_NAME, GQL_S_NAME_2)
+                    End With
+                End With
 
                 Dim nextCursor$ = String.Empty
                 Dim dataFound As Boolean = False
 
-                Dim vars$ = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & If(Cursor.IsEmptyString, var_init, String.Format(var_cursor, Cursor)) & "}")
+                'Dim vars$ = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & If(Cursor.IsEmptyString, var_init, String.Format(var_cursor, Cursor)) & "}")
+                Dim vars$ = String.Format(PayloadData, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_2, GQL_S_NAME_2,
+                                          SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(var_cursor2, IIf(Cursor.IsEmptyString, "null", $"""{Cursor}""")) & "}"))
 
-                If Cursor.IsEmptyString Then
+                'If Cursor.IsEmptyString Then
-                    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_1, GQL_S_NAME_1, vars)
+                '    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_1, GQL_S_NAME_1, vars)
-                Else
+                'Else
-                    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_2, GQL_S_NAME_2, vars)
+                '    URL = String.Format(GQL_Q, Token_lsd, Token_dtsg_Var, GQL_S_DOC_ID_2, GQL_S_NAME_2, vars)
-                End If
+                'End If
 
-                Using j As EContainer = GetDocument(URL, Token)
+                Using j As EContainer = GetDocument(GQL_Q2, vars, Token)
-                    If j.ListExists Then
+                    If Not CheckErrors(j) Then
-                        With j({"data", "xdt_viewer", "text_app_saved_media"})
+                        If j.ListExists Then
-                            If .ListExists Then
+                            With j({"data", "xdt_text_app_viewer", "saved_media"})
-                                nextCursor = .Value({"page_info"}, "end_cursor")
+                                If .ListExists Then
-                                With .Item({"edges"})
+                                    nextCursor = .Value({"page_info"}, "end_cursor")
-                                    If .ListExists Then dataFound = DefaultParser(.Self, Sections.Timeline, Token)
+                                    With .Item({"edges"})
-                                End With
+                                        If .ListExists Then dataFound = DefaultParser(.Self, Sections.Timeline, Token)
-                            End If
+                                    End With
-                        End With
+                                End If
+                            End With
+                        End If
+                    Else
+                        Throw New JsonErrorException(JsonErrorMessage(j))
                     End If
                 End Using
 
-                If dataFound And Not nextCursor.IsEmptyString Then DownloadSavedPosts(nextCursor, Token)
+                If dataFound And Not nextCursor.IsEmptyString Then DownloadSavedPosts(nextCursor, 0, Token)
+            Catch uex As JsonErrorException
+                If Round > 0 Then
+                    ProcessJsonErrorException(uex)
+                ElseIf Not _IdChanged AndAlso UpdateCredentials() Then
+                    DownloadSavedPosts(Cursor, Round + 1, Token)
+                Else
+                    ProcessJsonErrorException(uex)
+                End If
+            Catch eex As ExitException
+                Throw eex
             Catch ex As Exception
-                ProcessException(ex, Token, $"saved posts downloading error [{URL}]")
+                ProcessException(ex, Token, "saved posts downloading error")
             End Try
         End Sub
-        Private Function GetDocument(ByVal URL As String, ByVal Token As CancellationToken, Optional ByVal Round As Integer = 0) As EContainer
+        Private Function GetDocument(ByVal URL As String, ByVal PayLoad As String, ByVal Token As CancellationToken, Optional ByVal Round As Integer = 0) As EContainer
             Try
                 ThrowAny(Token)
                 If Round > 0 AndAlso Not UpdateCredentials() Then DisableDownload() : Throw New Exception("Failed to update credentials")
                 ThrowAny(Token)
                 WaitTimer()
-                Dim r$ = Responser.GetResponse(URL)
+                Dim r$ = Responser.GetResponse(URL, PayLoad)
                 If Not r.IsEmptyString Then Return JsonDocument.Parse(r) Else Throw New Exception("Failed to get a response")
             Catch ex As Exception
                 If Round = 0 Then
-                    Return GetDocument(URL, Token, Round + 1)
+                    Return GetDocument(URL, PayLoad, Token, Round + 1)
                 Else
                     Throw ex
                 End If
             End Try
         End Function
+        Private _IdChanged As Boolean = False
         Private Function UpdateCredentials(Optional ByVal e As ErrorsDescriber = Nothing) As Boolean
             Dim URL$ = If(IsSavedPosts, "https://www.threads.net/", $"https://www.threads.net/@{NameTrue}")
             ResetBaseTokens()
@@ -294,9 +382,25 @@ Namespace API.ThreadsNet
                 End With
                 WaitTimer()
                 Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException)
+                Dim newID$
+                Dim idStr$ = String.Empty
                 If Not r.IsEmptyString Then
                     ParseTokens(r, 0)
-                    If ID.IsEmptyString Then ID = RegexReplace(r, RParams.DMS("""props"":\{""user_id"":""(\d+)""", 1, EDP.ReturnValue))
+                    newID = RegexReplace(r, RParams.DMS("""props"":\{[^\{\}]*?""user_id"":""(\d+)""", 1, EDP.ReturnValue))
+                    If ID.IsEmptyString OrElse ID = newID Then
+                        _IdChanged = ID.IsEmptyString
+                        ID = newID
+                    Else
+                        _IdChanged = True
+                        idStr = $"user ID changed from {ID} to {newID}"
+                        LogError(Nothing, idStr)
+                        ID = newID
+                    End If
+                    If _IdChanged Then
+                        If Not idStr.IsEmptyString Then UserDescriptionUpdate(idStr, True, True, True)
+                        _ForceSaveUserInfo = True
+                        _ForceSaveUserInfoOnException = True
+                    End If
                 End If
                 Return Valid
             Catch ex As Exception
@@ -322,20 +426,22 @@ Namespace API.ThreadsNet
 #End Region
 #Region "ReparseMissing"
         Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
-            Const varsPattern$ = """postID"":""{0}"",""userID"":""{1}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false"
+            'Const varsPattern$ = """postID"":""{0}"",""userID"":""{1}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false"
+            Const varsPattern$ = """postID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowFediverseM1Featuresrelayprovider"":true,""__relay_internal__pv__BarcelonaShareableListsrelayprovider"":true,""__relay_internal__pv__BarcelonaIsSearchDiscoveryEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaOptionalCookiesEnabledrelayprovider"":true,""__relay_internal__pv__BarcelonaQuotedPostUFIEnabledrelayprovider"":false,""__relay_internal__pv__BarcelonaIsCrawlerrelayprovider"":false,""__relay_internal__pv__BarcelonaHasDisplayNamesrelayprovider"":false,""__relay_internal__pv__BarcelonaCanSeeSponsoredContentrelayprovider"":false,""__relay_internal__pv__BarcelonaShouldShowFediverseM075Featuresrelayprovider"":true,""__relay_internal__pv__BarcelonaShouldShowTagRedesignrelayprovider"":false,""__relay_internal__pv__BarcelonaIsInternalUserrelayprovider"":false,""__relay_internal__pv__BarcelonaInlineComposerEnabledrelayprovider"":false"
             'Const varsPattern$ = "{""postID"":""{0}"",""__relay_internal__pv__BarcelonaIsLoggedInrelayprovider"":true,""__relay_internal__pv__BarcelonaIsFeedbackHubEnabledrelayprovider"":false}"
-            Const urlPattern$ = "https://www.threads.net/api/graphql?lsd={0}&variables={1}&fb_api_req_friendly_name=BarcelonaPostPageQuery&server_timestamps=true&fb_dtsg={2}&doc_id=25460088156920903"
+            'Const urlPattern$ = "https://www.threads.net/api/graphql?lsd={0}&variables={1}&fb_api_req_friendly_name=BarcelonaPostPageQuery&server_timestamps=true&fb_dtsg={2}&doc_id=25460088156920903"
             Dim rList As New List(Of Integer)
-            Dim URL$ = String.Empty
             DefaultParser_ElemNode = Nothing
             DefaultParser_IgnorePass = True
             Try
                 If ContentMissingExists Then
                     Responser.Method = "POST"
+                    Responser.ContentType = "application/x-www-form-urlencoded"
                     Responser.Referer = $"https://www.threads.net/@{NameTrue}"
                     If Not IsSingleObjectDownload AndAlso Not UpdateCredentials() Then Throw New Exception("Failed to update credentials")
                     Dim m As UserMedia
                     Dim vars$
+                    Dim r As Byte
                     Dim j As EContainer
                     ProgressPre.ChangeMax(_ContentList.Count)
                     For i% = 0 To _ContentList.Count - 1
@@ -343,21 +449,39 @@ Namespace API.ThreadsNet
                         m = _ContentList(i)
                         If m.State = UserMedia.States.Missing And Not m.Post.ID.IsEmptyString Then
                             ThrowAny(Token)
-                            vars = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(varsPattern, m.Post.ID.Split("_").FirstOrDefault, ID) & "}")
+                            'vars = SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(varsPattern, m.Post.ID.Split("_").FirstOrDefault, ID) & "}")
-                            URL = String.Format(urlPattern, Token_lsd, vars, Token_dtsg_Var)
+                            'URL = String.Format(urlPattern, Token_lsd, vars, Token_dtsg_Var)
 
-                            j = GetDocument(URL, Token)
+                            vars = String.Format(PayloadData, Token_lsd, Token_dtsg_Var, "9094233770675261", "BarcelonaPostPageDirectQuery",
-                            If j.ListExists Then
+                                                 SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(varsPattern, m.Post.ID) & "}"))
-                                With j.ItemF({"data", "data", "edges", 0, "node", "thread_items", 0, "post"})
+
-                                    If .ListExists AndAlso DefaultParser({ .Self}, Sections.Timeline, Token) Then rList.Add(i)
+                            For r = 0 To 1
-                                End With
+                                j = GetDocument(GQL_Q2, vars, Token)
-                                j.Dispose()
+                                If Not CheckErrors(j) Then
-                            End If
+                                    If j.ListExists Then
+                                        With j.ItemF({"data", "data", "edges", 0, "node", "thread_items", 0, "post"})
+                                            If .ListExists AndAlso DefaultParser({ .Self}, Sections.Timeline, Token) Then rList.Add(i)
+                                        End With
+                                        j.Dispose()
+                                    End If
+                                Else
+                                    j.DisposeIfReady(False)
+                                    If r > 0 Then
+                                        ProcessJsonErrorException(New JsonErrorException(JsonErrorMessage(j)))
+                                    ElseIf Not _IdChanged AndAlso UpdateCredentials() Then
+                                        Continue For
+                                    Else
+                                        ProcessJsonErrorException(New JsonErrorException(JsonErrorMessage(j)))
+                                    End If
+                                End If
+                            Next
                         End If
                     Next
                 End If
+            Catch eex As ExitException
+                Throw eex
             Catch ex As Exception
-                ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
+                ProcessException(ex, Token, "reparseMissing error")
             Finally
                 DefaultParser_ElemNode = DefaultParser_ElemNode_Default
                 DefaultParser_IgnorePass = False
@@ -375,9 +499,9 @@ Namespace API.ThreadsNet
             If Not postCode.IsEmptyString Then
                 Dim postId$ = CodeToID(postCode)
                 If Not postId.IsEmptyString Then
-                    _NameTrue = MySettings.IsMyUser(url).UserName
+                    NameTrue = MySettings.IsMyUser(url).UserName
                     DefaultParser_PostUrlCreator = Function(post) url
-                    If Not _NameTrue.IsEmptyString AndAlso UpdateCredentials(EDP.ReturnValue) Then
+                    If Not NameTrue(True).IsEmptyString AndAlso UpdateCredentials(EDP.ReturnValue) Then
                         _ContentList.Add(New UserMedia(url) With {.State = UserMedia.States.Missing, .Post = postId})
                         ReparseMissing(Token)
                     End If

SCrawler/API/TikTok/SiteSettings.vb

@@ -62,8 +62,5 @@ Namespace API.TikTok
                 Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
             End If
         End Sub
-        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
-            Return String.Format(UrlPatternUser, DirectCast(User, UserData).TrueName)
-        End Function
     End Class
 End Namespace

SCrawler/API/TikTok/UserData.vb

@@ -62,15 +62,6 @@ Namespace API.TikTok
         Friend Property TitleUseRegexForTitle_Value As String = String.Empty
         Friend Property TitleUseGlobalRegexOptions As Boolean = True
         Private Property LastDownloadDate As Date? = Nothing
-        Private _TrueName As String = String.Empty
-        Friend Property TrueName As String
-            Get
-                Return _TrueName.IfNullOrEmpty(Name)
-            End Get
-            Set(ByVal NewName As String)
-                _TrueName = NewName
-            End Set
-        End Property
 #End Region
 #Region "Exchange"
         Friend Overrides Function ExchangeOptionsGet() As Object
@@ -98,7 +89,6 @@ Namespace API.TikTok
                     TitleAddVideoID = .Value(Name_TitleAddVideoID).FromXML(Of Boolean)(True)
                     LastDownloadDate = AConvert(Of Date)(.Value(Name_LastDownloadDate), ADateTime.Formats.BaseDateTime, Nothing)
                     If Not LastDownloadDate.HasValue Then LastDownloadDate = LastUpdated
-                    _TrueName = .Value(Name_TrueName)
                     TitleUseRegexForTitle = .Value(Name_TitleUseRegexForTitle).FromXML(Of Boolean)(False)
                     TitleUseRegexForTitle_Value = .Value(Name_TitleUseRegexForTitle_Value)
                     TitleUseGlobalRegexOptions = .Value(Name_TitleUseGlobalRegexOptions).FromXML(Of Boolean)(True)
@@ -107,7 +97,6 @@ Namespace API.TikTok
                     .Add(Name_TitleUseNative, TitleUseNative.BoolToInteger)
                     .Add(Name_TitleAddVideoID, TitleAddVideoID.BoolToInteger)
                     .Add(Name_LastDownloadDate, AConvert(Of String)(LastDownloadDate, AModes.XML, ADateTime.Formats.BaseDateTime, String.Empty))
-                    .Add(Name_TrueName, _TrueName)
                     .Add(Name_TitleUseRegexForTitle, TitleUseRegexForTitle.BoolToInteger)
                     .Add(Name_TitleUseRegexForTitle_Value, TitleUseRegexForTitle_Value)
                     .Add(Name_TitleUseGlobalRegexOptions, TitleUseGlobalRegexOptions.BoolToInteger)
@@ -174,7 +163,7 @@ Namespace API.TikTok
             UserCache = Nothing
         End Sub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
-            Dim URL$ = $"https://www.tiktok.com/@{TrueName}"
+            Dim URL$ = $"https://www.tiktok.com/@{NameTrue}"
             UserCache = CreateCache()
             Try
                 Dim postID$, title$, postUrl$, newName$
@@ -232,10 +221,7 @@ Namespace API.TikTok
                                         If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
                                     End If
                                     newName = j.Value("uploader")
-                                    If Not newName.IsEmptyString Then
+                                    If Not newName.IsEmptyString Then NameTrue = newName
-                                        If Not _TrueName = newName Then _ForceSaveUserInfo = True
-                                        _TrueName = newName
-                                    End If
                                     newName = j.Value("creator")
                                     If Not newName.IsEmptyString Then UserSiteName = newName
                                 End If

SCrawler/API/Twitter/Declarations.vb

@@ -7,7 +7,7 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports System.Globalization
-Imports PersonalUtilities.Functions.XML.Base
+Imports System.Text.RegularExpressions
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Twitter
     Friend Module Declarations
@@ -17,6 +17,8 @@ Namespace API.Twitter
         Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue)
         Friend ReadOnly StatusRegEx As RParams = RParams.DM(".*?(twitter|x)\.com/\S+/status/\d+", 0, EDP.ReturnValue)
         Friend ReadOnly BroadcastsUrls As Object() = {"entities", "urls", 0, "expanded_url"}
+        Friend ReadOnly GdlLimitRegEx As RParams = RParams.DM("Waiting until[\s\W\d\:]+\(rate limit\)", 0, RegexOptions.IgnoreCase, EDP.ReturnValue)
+        Friend ReadOnly GdlPostCoutNumberNodes As String() = {"data", "user", "result", "legacy", "statuses_count"}
         Private Function GetDateProvider() As ADateTime
             Dim n As DateTimeFormatInfo = CultureInfo.GetCultureInfo("en-us").DateTimeFormat.Clone
             n.FullDateTimePattern = "ddd MMM dd HH:mm:ss +ffff yyyy"

SCrawler/API/Twitter/EditorExchangeOptions.vb

@@ -8,11 +8,10 @@
 ' but WITHOUT ANY WARRANTY
 Imports SCrawler.Plugin.Attributes
 Imports DModels = SCrawler.API.Twitter.UserData.DownloadModels
-Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.Twitter
-    Friend Class EditorExchangeOptions
+    Friend Class EditorExchangeOptions : Inherits Base.EditorExchangeOptionsBase
         Private Const DefaultOffset As Integer = 100
-        Friend Property SiteKey As String = TwitterSiteKey
+        Friend Overrides Property SiteKey As String = TwitterSiteKey
         <PSetting(NameOf(SiteSettings.GifsDownload), NameOf(MySettings), LeftOffset:=DefaultOffset)>
         Friend Property GifsDownload As Boolean
         <PSetting(NameOf(SiteSettings.GifsSpecialFolder), NameOf(MySettings), LeftOffset:=DefaultOffset)>
@@ -51,8 +50,6 @@ Namespace API.Twitter
                   Caption:="Force apply",
                   ToolTip:="Force overrides the default parameters for the first download." & vbCr & "Applies to first download only.", LeftOffset:=DefaultOffset)>
         Friend Overridable Property DownloadModelForceApply As Boolean = False
-        <PSetting(Address:=SettingAddress.User, Caption:=DN.UserNameChangeCaption, ToolTip:=DN.UserNameChangeToolTip, LeftOffset:=DefaultOffset)>
-        Friend Overridable Property UserName As String = String.Empty
         Private ReadOnly Property MySettings As Object
         Friend Sub New(ByVal s As SiteSettings)
             GifsDownload = s.GifsDownload.Value
@@ -71,6 +68,7 @@ Namespace API.Twitter
             MySettings = s
         End Sub
         Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
             GifsDownload = u.GifsDownload
             GifsSpecialFolder = u.GifsSpecialFolder
             GifsPrefix = u.GifsPrefix
@@ -88,7 +86,6 @@ Namespace API.Twitter
                     DownloadModelLikes = dm.Contains(DModels.Likes)
                 End If
             End If
-            UserName = u.NameTrue(True)
             MySettings = u.HOST.Source
         End Sub
     End Class

SCrawler/API/Twitter/SiteSettings.vb

@@ -11,14 +11,50 @@ Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Cookies
 Imports DN = SCrawler.API.Base.DeclaredNames
+Imports IG = SCrawler.API.Instagram.SiteSettings
 Namespace API.Twitter
     <Manifest(TwitterSiteKey), SavedPosts, SeparatedTasks, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
+#Region "Icon"
+        <PXML("UseNewIcon")> Private ReadOnly Property UseNewIconXML As PropertyValue
+        <PropertyOption(ControlText:="Use the new Twitter icon (X)", ControlToolTip:="Restart SCrawler to take effect")>
+        Private ReadOnly Property UseNewIcon As PropertyValue
+            Get
+                If Not DefaultInstance Is Nothing Then
+                    Return DirectCast(DefaultInstance, SiteSettings).UseNewIconXML
+                Else
+                    Return UseNewIconXML
+                End If
+            End Get
+        End Property
+        Private Sub UpdateIcon()
+            If CBool(UseNewIcon.Value) Then _Icon = My.Resources.SiteResources.TwitterIconNew_32 : _Image = _Icon.ToBitmap
+        End Sub
+#End Region
 #Region "Categories"
         Private Const CAT_DOWN As String = "Downloading"
 #End Region
+#Region "Auth"
+        <PropertyOption(ControlText:="Update cookies", ControlToolTip:="Update cookies during requests", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property CookiesUpdate As PropertyValue
+        <PropertyOption(ControlText:="Use UserAgent", ControlToolTip:="Use UserAgent in requests", IsAuth:=True), PXML, PClonable>
+        Friend ReadOnly Property UserAgentUse As PropertyValue
+        <PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=True, InheritanceName:=SettingsCLS.HEADER_DEF_UserAgent),
+         PXML("UserAgent", OnlyForChecked:=True), PClonable>
+        Private ReadOnly Property UserAgentXML As PropertyValue
+        Friend ReadOnly Property UserAgent As String
+            Get
+                If CBool(UserAgentUse.Value) AndAlso Not CStr(UserAgentXML.Value).IsEmptyString Then
+                    Return UserAgentXML.Value
+                Else
+                    Return String.Empty
+                End If
+            End Get
+        End Property
+#End Region
 #Region "Other properties"
         <PropertyOption(ControlText:="Use the appropriate model",
                         ControlToolTip:="Use the appropriate model for new users." & vbCr &
@@ -35,10 +71,30 @@ Namespace API.Twitter
         Friend Property UseNewEndPointProfiles As PropertyValue
 #End Region
 #Region "Limits"
+        Friend Const TimerDisabled As Integer = -1
+        Friend Const TimerFirstUseTheSame As Integer = -2
         <PropertyOption(ControlText:="Abort on limit", ControlToolTip:="Abort twitter downloading when limit is reached", Category:=CAT_DOWN), PXML, PClonable>
         Friend Property AbortOnLimit As PropertyValue
         <PropertyOption(ControlText:="Download already parsed", ControlToolTip:="Download already parsed content on abort", Category:=CAT_DOWN), PXML, PClonable>
         Friend Property DownloadAlreadyParsed As PropertyValue
+#End Region
+#Region "Timers"
+        <PropertyOption(ControlText:="Sleep timer",
+                        ControlToolTip:="Use sleep timer in requests." & vbCr &
+                                        "You can set a timer value (in seconds) to wait before each subsequent request." & vbCr &
+                                        "-1 to disable and use the default algorithm." & vbCr &
+                                        "Default: 20", Category:=CAT_DOWN), PXML, PClonable>
+        Friend ReadOnly Property SleepTimer As PropertyValue
+        <Provider(NameOf(SleepTimer), FieldsChecker:=True)>
+        Private ReadOnly Property SleepTimerProvider As IFormatProvider
+        <PropertyOption(ControlText:="Sleep timer at start",
+                        ControlToolTip:="Set a sleep timer (in seconds) before the first request." & vbCr &
+                                        "-1 to disable" & vbCr &
+                                        "-2 to use the 'Sleep timer' value" & vbCr &
+                                        "Default: -2", Category:=CAT_DOWN), PXML, PClonable>
+        Friend ReadOnly Property SleepTimerBeforeFirst As PropertyValue
+        <Provider(NameOf(SleepTimerBeforeFirst), FieldsChecker:=True)>
+        Private ReadOnly Property SleepTimerBeforeFirstProvider As IFormatProvider
 #End Region
         <PropertyOption(ControlText:="Media Model: allow non-user tweets", ControlToolTip:="Allow downloading non-user tweets in the media-model.", Category:=DN.CAT_UserDefs), PXML, PClonable>
         Friend ReadOnly Property MediaModelAllowNonUserTweets As PropertyValue
@@ -76,7 +132,17 @@ Namespace API.Twitter
         <Provider(NameOf(ConcurrentDownloads), FieldsChecker:=True)>
         Private ReadOnly Property MyConcurrentDownloadsProvider As IFormatProvider
 #End Region
+        Friend Overrides Property DefaultInstance As ISiteSettings
+            Get
+                Return MyBase.DefaultInstance
+            End Get
+            Set(ByVal Instance As ISiteSettings)
+                MyBase.DefaultInstance = Instance
+                If Not Instance Is Nothing Then UpdateIcon()
+            End Set
+        End Property
 #End Region
+#Region "Initializer"
         Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
             MyBase.New(TwitterSite, "x.com", AccName, Temp, My.Resources.SiteResources.TwitterIcon_32, My.Resources.SiteResources.TwitterIcon_32.ToBitmap)
 
@@ -87,11 +153,28 @@ Namespace API.Twitter
                 .Cookies.Changed = False
             End With
 
+            UseNewIconXML = New PropertyValue(False)
+
+            CookiesUpdate = New PropertyValue(False)
+            UserAgentUse = New PropertyValue(True)
+            UserAgentXML = New PropertyValue(If(Responser.UserAgentExists, Responser.UserAgent, String.Empty), GetType(String),
+                                             Sub(ByVal Value As Object)
+                                                 Responser.UserAgent = CStr(Value)
+                                                 Responser.SaveSettings(, EDP.ReturnValue)
+                                             End Sub)
+
             UseAppropriateModel = New PropertyValue(True)
+
             UseNewEndPointSearch = New PropertyValue(True)
             UseNewEndPointProfiles = New PropertyValue(True)
+
             AbortOnLimit = New PropertyValue(True)
             DownloadAlreadyParsed = New PropertyValue(True)
+            SleepTimer = New PropertyValue(TimerDisabled)
+            SleepTimerProvider = New IG.TimersChecker(TimerDisabled)
+            SleepTimerBeforeFirst = New PropertyValue(TimerFirstUseTheSame)
+            SleepTimerBeforeFirstProvider = New IG.TimersChecker(TimerFirstUseTheSame)
+
             MediaModelAllowNonUserTweets = New PropertyValue(False)
             GifsDownload = New PropertyValue(True)
             GifsSpecialFolder = New PropertyValue(String.Empty, GetType(String))
@@ -101,28 +184,60 @@ Namespace API.Twitter
             ConcurrentDownloads = New PropertyValue(1)
             MyConcurrentDownloadsProvider = New ConcurrentDownloadsProvider
 
+            _AllowUserAgentUpdate = False
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, $"/(twitter|x).com({CommunitiesUser}|)/"), 3)
             UrlPatternUser = "https://x.com/{0}"
             ImageVideoContains = "twitter"
             CheckNetscapeCookiesOnEndInit = True
             UseNetscapeCookies = True
         End Sub
+        Friend Overrides Sub EndInit()
+            UpdateIcon()
+            MyBase.EndInit()
+        End Sub
+#End Region
+#Region "GetInstance"
         Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
             Return New UserData
         End Function
-        Friend Const SinglePostPattern As String = "https://x.com/i/web/status/{0}"
+#End Region
-        Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
+#Region "BaseAuthExists, Available"
-            Return String.Format(SinglePostPattern, Media.Post.ID)
-        End Function
         Friend Overrides Function BaseAuthExists() As Boolean
             Return Responser.CookiesExists
         End Function
         Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean
             Return Settings.GalleryDLFile.Exists And BaseAuthExists()
         End Function
+#End Region
+#Region "Download"
         Friend Property LIMIT_ABORT As Boolean = False
         Friend ReadOnly Property LimitSkippedUsers As List(Of UserDataBase)
+        Friend Property UserNumber As Integer = -1
+        Friend Overrides Sub DownloadStarted(ByVal What As ISiteSettings.Download)
+            UserNumber = 0
+            MyBase.DownloadStarted(What)
+        End Sub
+        Friend Overrides Sub AfterDownload(ByVal User As Object, ByVal What As ISiteSettings.Download)
+            If Not User Is Nothing AndAlso DirectCast(User, UserData).GDL_REQUESTS_COUNT > 0 Then UserNumber += 1
+            MyBase.AfterDownload(User, What)
+        End Sub
         Friend Overrides Sub DownloadDone(ByVal What As ISiteSettings.Download)
+            If UserNumber > 0 Then
+                If CBool(CookiesUpdate.Value) Then
+                    With CookieKeeper.ParseNetscapeText(CookiesNetscapeFile.GetText(EDP.ReturnValue), EDP.ReturnValue)
+                        If .ListExists Then
+                            Responser.Cookies.Clear()
+                            Responser.Cookies.AddRange(.Self,, EDP.ReturnValue)
+                            Responser.SaveCookies(EDP.ReturnValue)
+                        Else
+                            Update_SaveCookiesNetscape(True)
+                        End If
+                    End With
+                Else
+                    Update_SaveCookiesNetscape(True)
+                End If
+            End If
+            UserNumber = -1
             If LimitSkippedUsers.Count > 0 Then
                 With LimitSkippedUsers
                     If .Count = 1 Then
@@ -137,6 +252,8 @@ Namespace API.Twitter
             LIMIT_ABORT = False
             MyBase.DownloadDone(What)
         End Sub
+#End Region
+#Region "UserOptions, Update"
         Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
             If Options Is Nothing OrElse (Not TypeOf Options Is EditorExchangeOptions OrElse
                                           Not DirectCast(Options, EditorExchangeOptions).SiteKey = TwitterSiteKey) Then _
@@ -152,6 +269,8 @@ Namespace API.Twitter
             End If
             MyBase.Update()
         End Sub
+#End Region
+#Region "IsMyUser, IsMyImageVideo, GetUserPostUrl, GetUserUrl"
         Friend Const CommunitiesUser As String = "/i/communities"
         Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
             Dim e As ExchangeOptions = MyBase.IsMyUser(UserURL)
@@ -169,8 +288,13 @@ Namespace API.Twitter
                 Return Nothing
             End If
         End Function
+        Friend Const SinglePostPattern As String = "https://x.com/i/web/status/{0}"
+        Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
+            Return String.Format(SinglePostPattern, Media.Post.ID)
+        End Function
         Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
             Return DirectCast(User, UserData).GetUserUrl
         End Function
+#End Region
     End Class
 End Namespace

SCrawler/API/Twitter/UserData.vb

@@ -35,15 +35,6 @@ Namespace API.Twitter
 #Region "Declarations"
         Private Const BroadCastPartUrl As String = "i/broadcasts"
         Private Const Label_Community As String = "Community"
-        Private _NameTrue As String = String.Empty
-        Friend Property NameTrue(Optional ByVal Exact As Boolean = False) As String
-            Get
-                Return If(Exact, _NameTrue, _NameTrue.IfNullOrEmpty(Name))
-            End Get
-            Set(ByVal NewName As String)
-                _NameTrue = NewName
-            End Set
-        End Property
         Friend Overrides ReadOnly Property SpecialLabels As IEnumerable(Of String)
             Get
                 Return {Label_Community}
@@ -114,7 +105,7 @@ Namespace API.Twitter
                     If .DownloadModelSearch Then DownloadModel += DownloadModels.Search
                     If .DownloadModelLikes Then DownloadModel += DownloadModels.Likes
                     If Not dModel = DownloadModel Then DownloadModelChanged = True
-                    _NameTrue = .UserName
+                    NameTrue = .UserName
                 End With
             End If
         End Sub
@@ -164,12 +155,11 @@ Namespace API.Twitter
                     StartMD5Checked = .Value(Name_StartMD5Checked).FromXML(Of Boolean)(False)
                     MediaModelAllowNonUserTweets = .Value(Name_MediaModelAllowNonUserTweets).FromXML(Of Boolean)(False)
                     IsCommunity = .Value(Name_IsCommunity).FromXML(Of Boolean)(False)
-                    _NameTrue = .Value(Name_TrueName)
                 Else
                     If Name.Contains("@") And Not IsCommunity Then
                         IsCommunity = True
-                        _NameTrue = Name.Split("@")(0)
+                        NameTrue = Name.Split("@")(0)
-                        ID = _NameTrue
+                        ID = NameTrue
                         ParseUserMediaOnly = False
                         Labels.ListAddValue(Label_Community, LNC)
                         Labels.Sort()
@@ -190,7 +180,7 @@ Namespace API.Twitter
                     .Add(Name_StartMD5Checked, StartMD5Checked.BoolToInteger)
                     .Add(Name_MediaModelAllowNonUserTweets, MediaModelAllowNonUserTweets.BoolToInteger)
                     .Add(Name_IsCommunity, IsCommunity.BoolToInteger)
-                    .Add(Name_TrueName, _NameTrue)
+                    .Add(Name_TrueName, NameTrue(True))
                 End If
             End With
         End Sub
@@ -266,8 +256,10 @@ Namespace API.Twitter
             End If
             Return result
         End Function
+        Friend Property GDL_REQUESTS_COUNT As Integer = 0
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
+                GDL_REQUESTS_COUNT = 0
                 If MySettings.LIMIT_ABORT Then
                     Throw New TwitterLimitException(Me)
                 Else
@@ -426,17 +418,7 @@ Namespace API.Twitter
                                                 userInfoParsed = True
                                                 Dim resValue$ = j.Value({"data", IIf(IsCommunity, "communityResults", "user"), "result"}, "__typename").StringTrim.StringToLower
                                                 Dim icon$
-                                                Dim __getImage As Action(Of String) = Sub(ByVal img As String)
+                                                Dim fileCrFunc As Func(Of String, SFile) = Function(img) UrlFile(img, True)
-                                                                                          If Not img.IsEmptyString Then
-                                                                                              Dim __imgFile As SFile = UrlFile(img, True)
-                                                                                              If Not __imgFile.Name.IsEmptyString Then
-                                                                                                  If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
-                                                                                                  __imgFile.Path = MyFile.CutPath.Path
-                                                                                                  If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
-                                                                                                  If __imgFile.Exists Then IconBannerDownloaded = True
-                                                                                              End If
-                                                                                          End If
-                                                                                      End Sub
                                                 If resValue.IsEmptyString Then
                                                     UserExists = False
                                                     j.Dispose()
@@ -458,7 +440,7 @@ Namespace API.Twitter
 
                                                                                 icon = .Value({"custom_banner_media", "media_info"}, "original_img_url").
                                                                                        IfNullOrEmpty(.Value({"default_banner_media", "media_info"}, "original_img_url"))
-                                                                                If Not icon.IsEmptyString And DownloadIconBanner Then __getImage.Invoke(icon)
+                                                                                If Not icon.IsEmptyString And DownloadIconBanner Then SimpleDownloadAvatar(icon, fileCrFunc)
                                                                             End If
                                                                         End If
                                                                     End With
@@ -484,8 +466,8 @@ Namespace API.Twitter
                                                                         icon = .Value("profile_image_url_https")
                                                                         If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty)
                                                                         If DownloadIconBanner Then
-                                                                            __getImage.Invoke(.Value("profile_banner_url"))
+                                                                            SimpleDownloadAvatar(.Value("profile_banner_url"), fileCrFunc)
-                                                                            __getImage.Invoke(icon)
+                                                                            SimpleDownloadAvatar(icon, fileCrFunc)
                                                                         End If
                                                                     End If
                                                                 End If
@@ -799,19 +781,87 @@ nextpIndx:
             End Sub
             Private Sub CheckForLimit(ByVal Value As String)
                 If Token.IsCancellationRequested Or (KillOnLimit AndAlso Not ProcessKilled AndAlso
-                   Not Value.IsEmptyString AndAlso Value.ToLower.Contains("for rate limit reset")) Then
+                   Not Value.IsEmptyString AndAlso (Value.ToLower.Contains("for rate limit reset") OrElse
+                                                    Not CStr(RegexReplace(Value, GdlLimitRegEx)).IsEmptyString)) Then
                     LimitReached = True
                     Kill()
                 End If
             End Sub
         End Class
+        Private ReadOnly Property SleepTimerValue(ByVal First As Boolean) As Integer
+            Get
+                Dim fTimer% = If(First, MySettings.SleepTimerBeforeFirst, MySettings.SleepTimer).Value
+                If First And fTimer = SiteSettings.TimerFirstUseTheSame Then fTimer = MySettings.SleepTimer.Value
+                Return fTimer
+            End Get
+        End Property
+        Private ReadOnly Property SleepRequest As String
+            Get
+                Dim s% = SleepTimerValue(False)
+                Return If(s = SiteSettings.TimerDisabled, String.Empty, $" --sleep-request {s}")
+            End Get
+        End Property
+        Private Sub GdlWaitFirstTimer(ByVal fTimer As Integer)
+            If GDL_REQUESTS_COUNT = 0 And Not fTimer = SiteSettings.TimerDisabled And MySettings.UserNumber > 0 Then Thread.Sleep(fTimer * 1000)
+            GDL_REQUESTS_COUNT += 1
+        End Sub
+        Private _GdlPreProgressPerformEnabled As Boolean = False
+        Private Sub GdlPreProgressPerform(ByVal Path As SFile, ByVal UpDir As SFile)
+            Dim f As SFile = Nothing
+            Try
+                Dim c% = -1, lb% = -1, cc%
+                Dim e As New ErrorsDescriber(EDP.ReturnValue)
+                While _GdlPreProgressPerformEnabled
+                    Thread.Sleep(100)
+                    If c > 0 Then
+                        cc = If(SFile.GetFiles(Path,,, e)?.Count, 0)
+                        If cc > c Then
+                            Exit Sub
+                        ElseIf cc > 0 And (lb = -1 Or cc > lb) Then
+                            ProgressPre.Perform(cc - IIf(lb = -1, 0, lb))
+                            lb = cc
+                        End If
+                    ElseIf Path.Exists(SFO.Path, False) Then
+                        Dim files As List(Of SFile) = SFile.GetFiles(Path,,, e)
+                        If files.ListExists Then
+                            f = files.FirstOrDefault(Function(ff) Not ff.Name.IsEmptyString AndAlso ff.Name.StartsWith("01_"))
+                            If UpDir.Exists(SFO.Path, False) Then
+                                Dim fNew As SFile = $"{UpDir.PathWithSeparator}00.txt"
+                                If f.Copy(fNew,, True, SFODelete.DeletePermanently,, e) Then
+                                    f = fNew
+                                    Using j As EContainer = JsonDocument.Parse(f.GetText(e), e)
+                                        If j.ListExists Then
+                                            c = j(GdlPostCoutNumberNodes).XmlIfNothingValue.FromXML(Of Integer)(-1)
+                                            If c > 0 Then c /= 30 : ProgressPre.ChangeMax(c) : Continue While
+                                        End If
+                                    End Using
+                                End If
+                            End If
+                            Exit Sub
+                        End If
+                    End If
+                End While
+            Catch ex As Exception
+                ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{ToStringForLog()}:{vbCr}Path: '{Path.Path}'{vbCr}Path 2: '{UpDir.Path}'")
+            Finally
+                If f.Exists Then f.Delete(SFO.File, SFODelete.DeletePermanently, EDP.ReturnValue)
+            End Try
+        End Sub
+        Private Sub GdlPreProgressPerformWait(ByVal t As Task)
+            _GdlPreProgressPerformEnabled = False
+            Try
+                While t.Status = TaskStatus.Running Or t.Status = TaskStatus.WaitingToRun : Thread.Sleep(100) : End While
+            Catch
+            End Try
+        End Sub
         Private Function GetDataFromGalleryDL(ByVal URL As String, ByVal Cache As CacheKeeper, ByVal UseTempPostList As Boolean,
                                               Optional ByVal Token As CancellationToken = Nothing) As SFile
             Dim command$ = String.Empty
             Try
                 Dim conf As SFile = GdlCreateConf(Cache.NewPath)
-                command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
+                Dim fTimer% = SleepTimerValue(True)
-                command &= GdlGetIdFilterString()
+                command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip{SleepRequest} --config ""{conf}"" --write-pages "
+                'command &= GdlGetIdFilterString()
                 Dim dir As SFile = Cache.NewPath
                 If dir.Exists(SFO.Path,, EDP.ThrowException) Then
                     Using batch As New TwitterGDL(dir, Token, MySettings.AbortOnLimit.Value)
@@ -823,6 +873,7 @@ nextpIndx:
                         '#If DEBUG Then
                         'Debug.WriteLine(command)
                         '#End If
+                        GdlWaitFirstTimer(fTimer)
                         batch.Execute(command)
                         If batch.LimitReached Then
                             If CBool(MySettings.DownloadAlreadyParsed.Value) And
@@ -867,6 +918,8 @@ nextpIndx:
                 Dim process As Boolean, multiMode As Boolean
                 Dim currentModel As DownloadModels
                 Dim urlPrePattern$ = $"https://x.com{IIf(IsCommunity, SiteSettings.CommunitiesUser, String.Empty)}/"
+                Dim fTimer% = SleepTimerValue(True)
+                Dim t As Task
 
                 If DownloadBroadcasts AndAlso Not dm.Contains(DownloadModels.Profile) Then dm.Add(DownloadModels.Profile)
 
@@ -885,7 +938,7 @@ nextpIndx:
                         dir.Exists(SFO.Path, True, EDP.ThrowException)
                         outList.Add(dir)
                         tgdl.ChangeDirectory(dir)
-                        command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
+                        command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip{SleepRequest} --config ""{conf}"" --write-pages "
                         If multiMode Then
                             command &= "{0}"
                         Else
@@ -913,7 +966,18 @@ nextpIndx:
                             Else
                                 tgdl.TempPostsList = _TempPostsList
                             End If
+
+                            GdlWaitFirstTimer(fTimer)
+
+                            _GdlPreProgressPerformEnabled = True
+                            t = New Task(Sub() GdlPreProgressPerform(dir, conf))
+                            t.Start()
+
                             tgdl.Execute(command)
+
+                            _GdlPreProgressPerformEnabled = False
+                            GdlPreProgressPerformWait(t)
+
                             If tgdl.LimitReached Then
                                 If CBool(MySettings.DownloadAlreadyParsed.Value) And
                                    SFile.GetFiles(rootDir, "*.txt", IO.SearchOption.AllDirectories, EDP.ReturnValue).Count > 0 Then
@@ -936,6 +1000,8 @@ nextpIndx:
             Catch ex As Exception
                 ProcessException(ex, Token, $"{ToStringForLog()}: GetTimelineFromGalleryDL({command})")
                 Return Nothing
+            Finally
+                _GdlPreProgressPerformEnabled = False
             End Try
         End Function
         Private Function GdlGetIdFilterString(Optional ByVal TL As List(Of String) = Nothing) As String
@@ -945,8 +1011,11 @@ nextpIndx:
         Private Function GdlCreateConf(ByVal Path As SFile) As SFile
             Try
                 Dim conf As SFile = $"{Path.PathWithSeparator}TwitterGdlConfig.conf"
+                Dim __userAgent$ = MySettings.UserAgent
+                If Not __userAgent.IsEmptyString Then __userAgent = $"""user-agent"": ""{__userAgent}"","
                 Dim confText$ = "{""extractor"":{""cookies"": """ & MySettings.CookiesNetscapeFile.ToString.Replace("\", "/") &
-                                """,""cookies-update"": false,""twitter"":{""tweet-endpoint"": ""detail"",""cards"": false,""conversations"": true,""pinned"": false,""quoted"": false,""replies"": true,""retweets"": true,""strategy"": null,""text-tweets"": false,""twitpic"": false,""unique"": true,""users"": ""timeline"",""videos"": true}}}"
+                                 $""",""cookies-update"": {IIf(CBool(MySettings.CookiesUpdate.Value), "true", "false")}," & __userAgent &
+                                 """twitter"":{""tweet-endpoint"": ""detail"",""cards"": false,""conversations"": true,""pinned"": false,""quoted"": false,""replies"": true,""retweets"": true,""strategy"": null,""text-tweets"": false,""twitpic"": false,""unique"": true,""users"": ""timeline"",""videos"": true}}}"
                 If conf.Exists(SFO.Path, True, EDP.ThrowException) Then TextSaver.SaveTextToFile(confText, conf)
                 If Not conf.Exists Then Throw New IO.FileNotFoundException("Can't find Twitter GDL config file", conf)
                 Return conf
@@ -1050,6 +1119,7 @@ nextpIndx:
 #End Region
 #Region "DownloadSingleObject"
         Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
+            GDL_REQUESTS_COUNT = 0
             Dim um As New UserMedia(Data.URL) With {.State = UStates.Missing}
             If Not Data.URL.IsEmptyString AndAlso Data.URL.Contains(BroadCastPartUrl) Then um.Type = UTypes.m3u8
             _ContentList.Add(um)

SCrawler/API/XVIDEOS/UserData.vb

@@ -45,7 +45,6 @@ Namespace API.XVIDEOS
             End Get
         End Property
         Private Property SiteMode As SiteModes = SiteModes.User
-        Private Property TrueName As String = String.Empty
         Private Property Arguments As String = String.Empty
         Private Property PersonType As String = String.Empty
         Friend Overrides ReadOnly Property IsUser As Boolean
@@ -92,14 +91,14 @@ Namespace API.XVIDEOS
             If Not Force OrElse (Not SiteMode = SiteModes.User AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
                 Dim eObj As Plugin.ExchangeOptions = Nothing
                 If Force Then eObj = MySettings.IsMyUser(NewUrl)
-                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And TrueName.IsEmptyString) Then
+                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And NameTrue(True).IsEmptyString) Then
                     Dim n$() = If(Force, eObj.UserName, Name).Split("@")
                     If n.ListExists(2) Then
                         Dim opt$ = If(Force, eObj.Options, Options)
                         If opt.IsEmptyString AndAlso Not IsNumeric(n(0)) Then
                             If Not Force Then
                                 PersonType = n(0)
-                                TrueName = If(Force, eObj.UserName, Name).Replace($"{PersonType}@", String.Empty)
+                                NameTrue = If(Force, eObj.UserName, Name).Replace($"{PersonType}@", String.Empty)
                             End If
                         ElseIf Not opt.IsEmptyString Then
                             Dim n2$() = opt.Split("@")
@@ -108,8 +107,8 @@ Namespace API.XVIDEOS
                             Dim __Arguments$ = opt.Replace($"{__TrueName}@", String.Empty)
                             Dim __ForceApply As Boolean = False
 
-                            If Force AndAlso (Not TrueName = __TrueName Or Not SiteMode = __SiteMode) Then
+                            If Force AndAlso (Not NameTrue(True) = __TrueName Or Not SiteMode = __SiteMode) Then
-                                If ValidateChangeSearchOptions(ToStringForLog, $"{__SiteMode}: {__TrueName}", $"{SiteMode}: {TrueName}") Then
+                                If ValidateChangeSearchOptions(ToStringForLog, $"{__SiteMode}: {__TrueName}", $"{SiteMode}: {NameTrue(True)}") Then
                                     __ForceApply = True
                                 Else
                                     Return False
@@ -120,15 +119,15 @@ Namespace API.XVIDEOS
                             Options = opt
                             If Not Force Then
                                 SiteMode = __SiteMode
-                                TrueName = __TrueName
+                                NameTrue = __TrueName
-                                UserSiteName = $"{SiteMode}: {TrueName}"
+                                UserSiteName = $"{SiteMode}: {NameTrue}"
                                 If FriendlyName.IsEmptyString Then FriendlyName = UserSiteName
                                 Settings.Labels.Add(SearchRequestLabelName)
                                 Labels.ListAddValue(SearchRequestLabelName, LNC)
                                 Labels.Sort()
                             ElseIf Force And __ForceApply Then
                                 SiteMode = __SiteMode
-                                TrueName = __TrueName
+                                NameTrue = __TrueName
                             End If
 
                             Return True
@@ -142,14 +141,13 @@ Namespace API.XVIDEOS
             With Container
                 If Loading Then
                     SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
-                    TrueName = .Value(Name_TrueName)
                     Arguments = .Value(Name_Arguments)
                     PersonType = .Value(Name_PersonType)
-                    If PersonType.IsEmptyString And TrueName.IsEmptyString And Not Name.IsEmptyString Then
+                    If PersonType.IsEmptyString And NameTrue(True).IsEmptyString And Not Name.IsEmptyString Then
                         If Not Name.Contains("@") Then
                             Dim n$() = Name.Split("_")
                             PersonType = n(0)
-                            TrueName = Name.Replace($"{PersonType}_", String.Empty)
+                            NameTrue = Name.Replace($"{PersonType}_", String.Empty)
                         End If
                     End If
                     UpdateUserOptions()
@@ -160,7 +158,7 @@ Namespace API.XVIDEOS
                         .Value(Name_FriendlyName) = FriendlyName
                     End If
                     .Add(Name_SiteMode, CInt(SiteMode))
-                    .Add(Name_TrueName, TrueName)
+                    .Add(Name_TrueName, NameTrue(True))
                     .Add(Name_Arguments, Arguments)
                     .Add(Name_PersonType, PersonType)
 
@@ -181,19 +179,19 @@ Namespace API.XVIDEOS
         Friend Function GetUserUrl(ByVal Page As Integer) As String
             Dim url$ = String.Empty
             If SiteMode = SiteModes.User Then
-                url = $"https://xvideos.com/{PersonType}/{TrueName}"
+                url = $"https://xvideos.com/{PersonType}/{NameTrue}"
             ElseIf SiteMode = SiteModes.Categories Then
                 url = "https://xvideos.com/c/"
                 If Not Arguments.IsEmptyString Then url &= $"{Arguments}/"
-                url &= TrueName
+                url &= NameTrue
                 If Page > 1 Then url &= $"/{Page - 1}"
             ElseIf SiteMode = SiteModes.Tags Then
                 url = "https://www.xvideos.com/tags/"
                 If Not Arguments.IsEmptyString Then url &= $"{Arguments}/"
-                url &= $"{TrueName}/"
+                url &= $"{NameTrue}/"
                 If Page > 1 Then url &= Page - 1
             ElseIf SiteMode = SiteModes.Search Then
-                url = $"https://www.xvideos.com/?k={TrueName}"
+                url = $"https://www.xvideos.com/?k={NameTrue}"
                 If Not Arguments.IsEmptyString Then url &= $"&{Arguments}"
                 If Page > 1 Then url &= $"&p={Page - 1}"
             End If

SCrawler/API/Xhamster/SiteSettings.vb

@@ -86,7 +86,7 @@ Namespace API.Xhamster
                 If Not .SiteMode = SiteModes.User Then
                     Return .GetNonUserUrl(0)
                 Else
-                    Return String.Format(UrlPatternUser, IIf(.IsChannel, ChannelOption, UserOption), .TrueName)
+                    Return String.Format(UrlPatternUser, IIf(.IsChannel, ChannelOption, UserOption), .NameTrue)
                 End If
             End With
         End Function

SCrawler/API/Xhamster/UserData.vb

@@ -29,7 +29,6 @@ Namespace API.Xhamster
         End Property
         Friend Property IsChannel As Boolean = False
         Friend Property IsCreator As Boolean = False
-        Friend Property TrueName As String = String.Empty
         Friend Property Gender As String = String.Empty
         Friend Property SiteMode As SiteModes = SiteModes.User
         Friend Property Arguments As String = String.Empty
@@ -73,17 +72,17 @@ Namespace API.Xhamster
             If Not Force OrElse (Not SiteMode = SiteModes.User AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
                 Dim eObj As Plugin.ExchangeOptions = Nothing
                 If Force Then eObj = MySettings.IsMyUser(NewUrl)
-                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And TrueName.IsEmptyString) Then
+                If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And NameTrue(True).IsEmptyString) Then
                     Dim n$() = If(Force, eObj.UserName, Name).Split("@")
                     If n.ListExists Then
                         If n.Length = 2 And If(Force, eObj.Options, Options).IsEmptyString Then
                             If Force Then Return False
-                            TrueName = n(0)
+                            NameTrue = n(0)
                             IsChannel = n(1) = SiteSettings.ChannelOption
                             IsCreator = n(1) = SiteSettings.P_Creators
                         ElseIf IsChannel Then
                             If Force Then Return False
-                            TrueName = Name
+                            NameTrue = Name
                         ElseIf Not If(Force, eObj.Options, Options).IsEmptyString Then
                             Dim __TrueName$, __Arguments$, __Gender$
                             Dim __Mode As SiteModes
@@ -97,10 +96,10 @@ Namespace API.Xhamster
                                 __Arguments = n2(3)
                                 __TrueName = n2.ListTake(3, 100, EDP.ReturnValue).ListToString(String.Empty)
 
-                                If Force AndAlso (Not TrueName = __TrueName Or Not SiteMode = __Mode Or Not Gender = __Gender) Then
+                                If Force AndAlso (Not NameTrue(True) = __TrueName Or Not SiteMode = __Mode Or Not Gender = __Gender) Then
                                     If ValidateChangeSearchOptions(ToStringForLog,
                                                                    $"{__Mode}{IIf(__Gender.IsEmptyString, String.Empty, $" ({__Gender})")}: {__TrueName}",
-                                                                   $"{SiteMode}{IIf(Gender.IsEmptyString, String.Empty, $" ({Gender})")}: {TrueName}") Then
+                                                                   $"{SiteMode}{IIf(Gender.IsEmptyString, String.Empty, $" ({Gender})")}: {NameTrue(True)}") Then
                                         __ForceApply = True
                                     Else
                                         Return False
@@ -110,17 +109,17 @@ Namespace API.Xhamster
                                 Arguments = __Arguments
                                 Options = If(Force, eObj.Options, Options)
                                 If Not Force Then
-                                    TrueName = __TrueName
+                                    NameTrue = __TrueName
                                     SiteMode = __Mode
                                     Gender = __Gender
 
-                                    UserSiteName = $"{SiteMode}: {TrueName}"
+                                    UserSiteName = $"{SiteMode}: {NameTrue}"
                                     If FriendlyName.IsEmptyString Then FriendlyName = UserSiteName
                                     Settings.Labels.Add(SearchRequestLabelName)
                                     Labels.ListAddValue(SearchRequestLabelName, LNC)
                                     Labels.Sort()
                                 ElseIf Force And __ForceApply Then
-                                    TrueName = __TrueName
+                                    NameTrue = __TrueName
                                     SiteMode = __Mode
                                     Gender = __Gender
                                 End If
@@ -132,7 +131,7 @@ Namespace API.Xhamster
                             End If
                         Else
                             If Force Then Return False
-                            TrueName = n(0)
+                            NameTrue = n(0)
                         End If
                     End If
                 End If
@@ -144,7 +143,6 @@ Namespace API.Xhamster
                 If Loading Then
                     IsChannel = .Value(Name_IsChannel).FromXML(Of Boolean)(False)
                     IsCreator = .Value(Name_IsCreator).FromXML(Of Boolean)(False)
-                    TrueName = .Value(Name_TrueName)
                     Gender = .Value(Name_Gender)
                     SiteMode = .Value(Name_SiteMode).FromXML(Of Integer)(SiteModes.User)
                     Arguments = .Value(Name_Arguments)
@@ -157,7 +155,7 @@ Namespace API.Xhamster
                     End If
                     .Add(Name_IsChannel, IsChannel.BoolToInteger)
                     .Add(Name_IsCreator, IsCreator.BoolToInteger)
-                    .Add(Name_TrueName, TrueName)
+                    .Add(Name_TrueName, NameTrue(True))
                     .Add(Name_Gender, Gender)
                     .Add(Name_SiteMode, CInt(SiteMode))
                     .Add(Name_Arguments, Arguments)
@@ -198,7 +196,7 @@ Namespace API.Xhamster
                     Case SiteModes.User : url &= SiteSettings.P_Creators
                     Case Else : Return String.Empty
                 End Select
-                url &= $"/{TrueName}"
+                url &= $"/{NameTrue}"
 
                 Dim args$ = Arguments
                 If (args.IsEmptyString OrElse Not args.Contains(newest)) And Not SiteMode = SiteModes.Search Then url &= newest
@@ -289,14 +287,14 @@ Namespace API.Xhamster
                     URL = $"https://xhamster.com/my/favorites/{IIf(IsVideo, "videos", "photos-and-galleries")}{IIf(Page = 1, String.Empty, $"/{Page}")}"
                     containerNodes.Add(If(IsVideo, {"favoriteVideoListComponent", "models"}, {"favoritesGalleriesAndPhotosCollection"}))
                 ElseIf IsChannel Then
-                    URL = $"https://xhamster.com/channels/{TrueName}/newest{IIf(Page = 1, String.Empty, $"/{Page}")}"
+                    URL = $"https://xhamster.com/channels/{NameTrue}/newest{IIf(Page = 1, String.Empty, $"/{Page}")}"
                 ElseIf SiteMode = SiteModes.Search Then
                     URL = GetNonUserUrl(Page)
                     containerNodes.Add({"searchResult", "models"})
                 ElseIf IsCreator Or SiteMode = SiteModes.Tags Or SiteMode = SiteModes.Categories Or SiteMode = SiteModes.Pornstars Then
                     URL = GetNonUserUrl(Page)
                 Else
-                    URL = $"https://xhamster.com/users/{TrueName}/{IIf(IsVideo, "videos", "photos")}{IIf(Page = 1, String.Empty, $"/{Page}")}"
+                    URL = $"https://xhamster.com/users/{NameTrue}/{IIf(IsVideo, "videos", "photos")}{IIf(Page = 1, String.Empty, $"/{Page}")}"
                 End If
                 ThrowAny(Token)
 

SCrawler/Download/Feed/DownloadFeedForm.Designer.vb

@@ -61,6 +61,7 @@ Namespace DownloadObjects
             Me.BTT_FEED_DELETE_DAILY_LIST = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_FEED_DELETE_DAILY_DATE = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_CURR_SESSION_SET = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_CURR_SESSION_SET_LAST = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_MERGE_SESSIONS = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_CLEAR_DAILY = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_RESET_DAILY = New System.Windows.Forms.ToolStripMenuItem()
@@ -75,7 +76,7 @@ Namespace DownloadObjects
             Me.BTT_DOWN_SELECTED = New System.Windows.Forms.ToolStripMenuItem()
             Me.BTT_REFRESH = New System.Windows.Forms.ToolStripButton()
             Me.TP_DATA = New System.Windows.Forms.TableLayoutPanel()
-            Me.BTT_CURR_SESSION_SET_LAST = New System.Windows.Forms.ToolStripMenuItem()
+            Me.BTT_CHECK_INVERT = New System.Windows.Forms.ToolStripMenuItem()
             SEP_1 = New System.Windows.Forms.ToolStripSeparator()
             SEP_2 = New System.Windows.Forms.ToolStripSeparator()
             MENU_VIEW = New System.Windows.Forms.ToolStripDropDownButton()
@@ -187,7 +188,7 @@ Namespace DownloadObjects
             '
             Me.MENU_LOAD_SESSION.AutoToolTip = False
             Me.MENU_LOAD_SESSION.DisplayStyle = System.Windows.Forms.ToolStripItemDisplayStyle.Image
-            Me.MENU_LOAD_SESSION.DropDownItems.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_LOAD_SESSION_CURRENT, Me.BTT_LOAD_SESSION_LAST, Me.BTT_LOAD_SESSION_CHOOSE, MENU_LOAD_SEP_0, Me.BTT_COPY_TO, Me.BTT_MOVE_TO, MENU_LOAD_SEP_1, Me.BTT_COPY_SPEC_TO, Me.BTT_MOVE_SPEC_TO, MENU_LOAD_SEP_2, Me.BTT_LOAD_FAV, Me.BTT_LOAD_SPEC, MENU_LOAD_SEP_3, Me.BTT_FEED_ADD_FAV, Me.BTT_FEED_ADD_FAV_REMOVE, Me.BTT_FEED_REMOVE_FAV, MENU_LOAD_SEP_4, Me.BTT_FEED_ADD_SPEC, Me.BTT_FEED_ADD_SPEC_REMOVE, Me.BTT_FEED_REMOVE_SPEC, MENU_LOAD_SEP_5, Me.BTT_FEED_CLEAR_FAV, Me.BTT_FEED_CLEAR_SPEC, Me.BTT_FEED_DELETE_SPEC, Me.BTT_FEED_DELETE_DAILY_LIST, Me.BTT_FEED_DELETE_DAILY_DATE, MENU_LOAD_SEP_6, Me.BTT_CURR_SESSION_SET, Me.BTT_CURR_SESSION_SET_LAST, Me.BTT_MERGE_SESSIONS, Me.BTT_CLEAR_DAILY, Me.BTT_RESET_DAILY, MENU_LOAD_SEP_7, Me.BTT_MERGE_FEEDS, MENU_LOAD_SEP_8, Me.BTT_CHECK_ALL, Me.BTT_CHECK_NONE, MENU_LOAD_SEP_9, Me.BTT_VIEW_SAVE, Me.BTT_VIEW_LOAD})
+            Me.MENU_LOAD_SESSION.DropDownItems.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_LOAD_SESSION_CURRENT, Me.BTT_LOAD_SESSION_LAST, Me.BTT_LOAD_SESSION_CHOOSE, MENU_LOAD_SEP_0, Me.BTT_COPY_TO, Me.BTT_MOVE_TO, MENU_LOAD_SEP_1, Me.BTT_COPY_SPEC_TO, Me.BTT_MOVE_SPEC_TO, MENU_LOAD_SEP_2, Me.BTT_LOAD_FAV, Me.BTT_LOAD_SPEC, MENU_LOAD_SEP_3, Me.BTT_FEED_ADD_FAV, Me.BTT_FEED_ADD_FAV_REMOVE, Me.BTT_FEED_REMOVE_FAV, MENU_LOAD_SEP_4, Me.BTT_FEED_ADD_SPEC, Me.BTT_FEED_ADD_SPEC_REMOVE, Me.BTT_FEED_REMOVE_SPEC, MENU_LOAD_SEP_5, Me.BTT_FEED_CLEAR_FAV, Me.BTT_FEED_CLEAR_SPEC, Me.BTT_FEED_DELETE_SPEC, Me.BTT_FEED_DELETE_DAILY_LIST, Me.BTT_FEED_DELETE_DAILY_DATE, MENU_LOAD_SEP_6, Me.BTT_CURR_SESSION_SET, Me.BTT_CURR_SESSION_SET_LAST, Me.BTT_MERGE_SESSIONS, Me.BTT_CLEAR_DAILY, Me.BTT_RESET_DAILY, MENU_LOAD_SEP_7, Me.BTT_MERGE_FEEDS, MENU_LOAD_SEP_8, Me.BTT_CHECK_ALL, Me.BTT_CHECK_NONE, Me.BTT_CHECK_INVERT, MENU_LOAD_SEP_9, Me.BTT_VIEW_SAVE, Me.BTT_VIEW_LOAD})
             Me.MENU_LOAD_SESSION.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
             Me.MENU_LOAD_SESSION.ImageTransparentColor = System.Drawing.Color.Magenta
             Me.MENU_LOAD_SESSION.Name = "MENU_LOAD_SESSION"
@@ -347,6 +348,13 @@ Namespace DownloadObjects
             Me.BTT_CURR_SESSION_SET.Text = "Set current session..."
             Me.BTT_CURR_SESSION_SET.ToolTipText = "Select one of the download sessions and set it as the current session"
             '
+            'BTT_CURR_SESSION_SET_LAST
+            '
+            Me.BTT_CURR_SESSION_SET_LAST.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
+            Me.BTT_CURR_SESSION_SET_LAST.Name = "BTT_CURR_SESSION_SET_LAST"
+            Me.BTT_CURR_SESSION_SET_LAST.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_CURR_SESSION_SET_LAST.Text = "Set last download session as current session"
+            '
             'BTT_MERGE_SESSIONS
             '
             Me.BTT_MERGE_SESSIONS.AutoToolTip = True
@@ -476,12 +484,11 @@ Namespace DownloadObjects
             Me.TP_DATA.Size = New System.Drawing.Size(484, 436)
             Me.TP_DATA.TabIndex = 1
             '
-            'BTT_CURR_SESSION_SET_LAST
+            'BTT_CHECK_INVERT
             '
-            Me.BTT_CURR_SESSION_SET_LAST.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
+            Me.BTT_CHECK_INVERT.Name = "BTT_CHECK_INVERT"
-            Me.BTT_CURR_SESSION_SET_LAST.Name = "BTT_CURR_SESSION_SET_LAST"
+            Me.BTT_CHECK_INVERT.Size = New System.Drawing.Size(352, 22)
-            Me.BTT_CURR_SESSION_SET_LAST.Size = New System.Drawing.Size(352, 22)
+            Me.BTT_CHECK_INVERT.Text = "Invert selection"
-            Me.BTT_CURR_SESSION_SET_LAST.Text = "Set last download session as current session"
             '
             'DownloadFeedForm
             '
@@ -543,5 +550,6 @@ Namespace DownloadObjects
         Private WithEvents BTT_MOVE_SPEC_TO As ToolStripMenuItem
         Private WithEvents BTT_RESET_DAILY As ToolStripMenuItem
         Private WithEvents BTT_CURR_SESSION_SET_LAST As ToolStripMenuItem
+        Private WithEvents BTT_CHECK_INVERT As ToolStripMenuItem
     End Class
 End Namespace

SCrawler/Download/Feed/DownloadFeedForm.vb

@@ -1081,13 +1081,14 @@ Namespace DownloadObjects
             End Try
         End Sub
 #End Region
-        Private Sub BTT_CHECK_ALL_NONE_Click(sender As Object, e As EventArgs) Handles BTT_CHECK_ALL.Click, BTT_CHECK_NONE.Click
+        Private Sub BTT_CHECK_ALL_NONE_Click(sender As Object, e As EventArgs) Handles BTT_CHECK_ALL.Click, BTT_CHECK_NONE.Click, BTT_CHECK_INVERT.Click
             Try
                 Dim checked As Boolean = sender Is BTT_CHECK_ALL
+                Dim isInvert As Boolean = sender Is BTT_CHECK_INVERT
                 ControlInvokeFast(TP_DATA, Sub()
                                                With TP_DATA
                                                    If .Controls.Count > 0 Then
-                                                       For Each cnt As FeedMedia In .Controls : cnt.Checked = checked : Next
+                                                       For Each cnt As FeedMedia In .Controls : cnt.Checked = If(isInvert, Not cnt.Checked, checked) : Next
                                                    End If
                                                End With
                                            End Sub, EDP.None)

SCrawler/Download/Feed/FeedMedia.vb

@@ -407,7 +407,7 @@ Namespace DownloadObjects
 #End Region
 #Region "Picture / Video objects"
         Private Sub MyPicture_DoubleClick(sender As Object, e As EventArgs) Handles MyPicture.DoubleClick
-            Try : Process.Start(File) : Catch : End Try
+            Try : Process.Start(IIf(IsSubscription, Post.URL_BASE, File.ToString)) : Catch : End Try
         End Sub
 #End Region
 #Region "Context"

SCrawler/Download/Feed/FeedSpecialCollection.vb

@@ -55,8 +55,10 @@ Namespace DownloadObjects
 #Region "FeedsComparer"
         Private Class FeedsComparer : Implements IComparer(Of FeedSpecial)
             Friend Function Compare(ByVal x As FeedSpecial, ByVal y As FeedSpecial) As Integer Implements IComparer(Of FeedSpecial).Compare
-                If x.IsFavorite Then
+                If x.IsFavorite And Not y.IsFavorite Then
                     Return -1
+                ElseIf Not x.IsFavorite And y.IsFavorite Then
+                    Return 1
                 Else
                     Return x.Name.CompareTo(y.Name)
                 End If

SCrawler/Editors/GlobalSettingsForm.Designer.vb

@@ -32,8 +32,6 @@ Namespace Editors
             Dim ActionButton4 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton5 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton6 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton7 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton8 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TP_FILE_NAME As System.Windows.Forms.TableLayoutPanel
             Dim TP_FILE_PATTERNS As System.Windows.Forms.TableLayoutPanel
             Dim LBL_DATE_POS As System.Windows.Forms.Label
@@ -46,26 +44,28 @@ Namespace Editors
             Dim TP_CHANNELS As System.Windows.Forms.TableLayoutPanel
             Dim TAB_BEHAVIOR As System.Windows.Forms.TabPage
             Dim TP_BEHAVIOR As System.Windows.Forms.TableLayoutPanel
-            Dim ActionButton9 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton7 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton10 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton8 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TP_OPEN_INFO As System.Windows.Forms.TableLayoutPanel
             Dim TP_OPEN_PROGRESS As System.Windows.Forms.TableLayoutPanel
             Dim TP_BEHAVIOR_F6 As System.Windows.Forms.TableLayoutPanel
             Dim TAB_DOWN As System.Windows.Forms.TabPage
             Dim TP_DOWNLOADING As System.Windows.Forms.TableLayoutPanel
-            Dim ActionButton11 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton9 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton12 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton10 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TP_MISSING_DATA As System.Windows.Forms.TableLayoutPanel
             Dim TAB_FEED As System.Windows.Forms.TabPage
             Dim TP_FEED As System.Windows.Forms.TableLayoutPanel
             Dim TP_FEED_IMG_COUNT As System.Windows.Forms.TableLayoutPanel
             Dim TP_FEED_SES As System.Windows.Forms.TableLayoutPanel
-            Dim ActionButton13 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton11 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton14 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton12 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TP_FEED_SPEC_SEARCH As System.Windows.Forms.TableLayoutPanel
             Dim TAB_NOTIFY As System.Windows.Forms.TabPage
             Dim TP_NOTIFY_MAIN As System.Windows.Forms.TableLayoutPanel
             Dim TP_ENVIR As System.Windows.Forms.TableLayoutPanel
+            Dim ActionButton13 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton14 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton15 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton16 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton17 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
@@ -78,25 +78,23 @@ Namespace Editors
             Dim ActionButton24 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton25 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton26 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton27 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton28 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TAB_STD As System.Windows.Forms.TabPage
             Dim TP_STD As System.Windows.Forms.TableLayoutPanel
-            Dim ActionButton29 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton27 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ListColumn1 As PersonalUtilities.Forms.Controls.Base.ListColumn = New PersonalUtilities.Forms.Controls.Base.ListColumn()
             Dim ListColumn2 As PersonalUtilities.Forms.Controls.Base.ListColumn = New PersonalUtilities.Forms.Controls.Base.ListColumn()
             Dim TAB_DESIGN As System.Windows.Forms.TabPage
             Dim TP_DESIGN As System.Windows.Forms.TableLayoutPanel
+            Dim ActionButton28 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton29 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton30 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton31 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim TP_HEADERS_DEF As System.Windows.Forms.TableLayoutPanel
             Dim ActionButton32 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton33 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim TP_HEADERS_DEF As System.Windows.Forms.TableLayoutPanel
             Dim ActionButton34 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton35 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton36 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton37 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
-            Dim ActionButton38 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim TAB_HEADERS As System.Windows.Forms.TabPage
             Me.TXT_GLOBAL_PATH = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.TXT_IMAGE_LARGE = New PersonalUtilities.Forms.Controls.TextBoxExtended()
@@ -106,7 +104,6 @@ Namespace Editors
             Me.TXT_MAX_JOBS_CHANNELS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.CH_CHECK_VER_START = New System.Windows.Forms.CheckBox()
             Me.TXT_IMGUR_CLIENT_ID = New PersonalUtilities.Forms.Controls.TextBoxExtended()
-            Me.TXT_USER_AGENT = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.OPT_FILE_NAME_REPLACE = New System.Windows.Forms.RadioButton()
             Me.OPT_FILE_NAME_ADD_DATE = New System.Windows.Forms.RadioButton()
             Me.CH_FILE_NAME_CHANGE = New System.Windows.Forms.CheckBox()
@@ -153,6 +150,7 @@ Namespace Editors
             Me.CH_DOWN_ALL_NOTIFY = New System.Windows.Forms.CheckBox()
             Me.CH_FEED_SPEC_SEARCH = New System.Windows.Forms.CheckBox()
             Me.CH_FEED_SPEC_SEARCH_DEEP = New System.Windows.Forms.CheckBox()
+            Me.CH_FEED_OPEN_CTRLF = New System.Windows.Forms.CheckBox()
             Me.TXT_CHANNELS_ROWS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.TXT_CHANNELS_COLUMNS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.CH_DOWN_IMAGES_NATIVE = New System.Windows.Forms.CheckBox()
@@ -212,7 +210,6 @@ Namespace Editors
             Me.TAB_MAIN = New System.Windows.Forms.TabControl()
             Me.TAB_ENVIR = New System.Windows.Forms.TabPage()
             Me.CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
-            Me.CH_FEED_OPEN_CTRLF = New System.Windows.Forms.CheckBox()
             TP_BASIS = New System.Windows.Forms.TableLayoutPanel()
             TP_IMAGES = New System.Windows.Forms.TableLayoutPanel()
             TP_FILE_NAME = New System.Windows.Forms.TableLayoutPanel()
@@ -256,7 +253,6 @@ Namespace Editors
             CType(Me.TXT_MAX_JOBS_USERS, System.ComponentModel.ISupportInitialize).BeginInit()
             CType(Me.TXT_MAX_JOBS_CHANNELS, System.ComponentModel.ISupportInitialize).BeginInit()
             CType(Me.TXT_IMGUR_CLIENT_ID, System.ComponentModel.ISupportInitialize).BeginInit()
-            CType(Me.TXT_USER_AGENT, System.ComponentModel.ISupportInitialize).BeginInit()
             TP_FILE_NAME.SuspendLayout()
             TP_FILE_PATTERNS.SuspendLayout()
             TP_CHANNELS_IMGS.SuspendLayout()
@@ -333,12 +329,11 @@ Namespace Editors
             TP_BASIS.Controls.Add(Me.TXT_MAX_JOBS_USERS, 0, 3)
             TP_BASIS.Controls.Add(Me.TXT_MAX_JOBS_CHANNELS, 0, 4)
             TP_BASIS.Controls.Add(Me.CH_CHECK_VER_START, 0, 5)
-            TP_BASIS.Controls.Add(Me.TXT_IMGUR_CLIENT_ID, 0, 7)
+            TP_BASIS.Controls.Add(Me.TXT_IMGUR_CLIENT_ID, 0, 6)
-            TP_BASIS.Controls.Add(Me.TXT_USER_AGENT, 0, 6)
             TP_BASIS.Dock = System.Windows.Forms.DockStyle.Fill
             TP_BASIS.Location = New System.Drawing.Point(3, 3)
             TP_BASIS.Name = "TP_BASIS"
-            TP_BASIS.RowCount = 9
+            TP_BASIS.RowCount = 8
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
@@ -346,10 +341,8 @@ Namespace Editors
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
-            TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
             TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
-            TP_BASIS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
             TP_BASIS.Size = New System.Drawing.Size(615, 418)
             TP_BASIS.TabIndex = 0
             '
@@ -489,30 +482,11 @@ Namespace Editors
             Me.TXT_IMGUR_CLIENT_ID.Buttons.Add(ActionButton6)
             Me.TXT_IMGUR_CLIENT_ID.CaptionText = "Imgur Client ID"
             Me.TXT_IMGUR_CLIENT_ID.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.TXT_IMGUR_CLIENT_ID.Location = New System.Drawing.Point(4, 204)
+            Me.TXT_IMGUR_CLIENT_ID.Location = New System.Drawing.Point(4, 175)
             Me.TXT_IMGUR_CLIENT_ID.Name = "TXT_IMGUR_CLIENT_ID"
             Me.TXT_IMGUR_CLIENT_ID.Size = New System.Drawing.Size(607, 22)
             Me.TXT_IMGUR_CLIENT_ID.TabIndex = 7
             '
-            'TXT_USER_AGENT
-            '
-            ActionButton7.BackgroundImage = CType(resources.GetObject("ActionButton7.BackgroundImage"), System.Drawing.Image)
-            ActionButton7.Name = "Refresh"
-            ActionButton7.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton8.BackgroundImage = CType(resources.GetObject("ActionButton8.BackgroundImage"), System.Drawing.Image)
-            ActionButton8.Name = "Clear"
-            ActionButton8.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_USER_AGENT.Buttons.Add(ActionButton7)
-            Me.TXT_USER_AGENT.Buttons.Add(ActionButton8)
-            Me.TXT_USER_AGENT.CaptionText = "UserAgent"
-            Me.TXT_USER_AGENT.CaptionToolTipEnabled = True
-            Me.TXT_USER_AGENT.CaptionToolTipText = "Default user agent to use in requests"
-            Me.TXT_USER_AGENT.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.TXT_USER_AGENT.Location = New System.Drawing.Point(4, 175)
-            Me.TXT_USER_AGENT.Name = "TXT_USER_AGENT"
-            Me.TXT_USER_AGENT.Size = New System.Drawing.Size(607, 22)
-            Me.TXT_USER_AGENT.TabIndex = 6
-            '
             'TP_FILE_NAME
             '
             TP_FILE_NAME.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
@@ -1131,6 +1105,19 @@ Namespace Editors
         "s as well.")
             Me.CH_FEED_SPEC_SEARCH_DEEP.UseVisualStyleBackColor = True
             '
+            'CH_FEED_OPEN_CTRLF
+            '
+            Me.CH_FEED_OPEN_CTRLF.AutoSize = True
+            Me.CH_FEED_OPEN_CTRLF.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_FEED_OPEN_CTRLF.Location = New System.Drawing.Point(4, 299)
+            Me.CH_FEED_OPEN_CTRLF.Name = "CH_FEED_OPEN_CTRLF"
+            Me.CH_FEED_OPEN_CTRLF.Size = New System.Drawing.Size(613, 19)
+            Me.CH_FEED_OPEN_CTRLF.TabIndex = 11
+            Me.CH_FEED_OPEN_CTRLF.Text = "Use 'Ctrl+F' to open the Feed"
+            TT_MAIN.SetToolTip(Me.CH_FEED_OPEN_CTRLF, "If checked, 'Ctrl+F' will be used to open the Feed. Otherwise, 'Alt+F' will be us" &
+        "ed.")
+            Me.CH_FEED_OPEN_CTRLF.UseVisualStyleBackColor = True
+            '
             'TP_CHANNELS_IMGS
             '
             TP_CHANNELS_IMGS.ColumnCount = 2
@@ -1347,11 +1334,11 @@ Namespace Editors
             'TXT_FOLDER_CMD
             '
             Me.TXT_FOLDER_CMD.AutoShowClearButton = True
-            ActionButton9.BackgroundImage = CType(resources.GetObject("ActionButton9.BackgroundImage"), System.Drawing.Image)
+            ActionButton7.BackgroundImage = CType(resources.GetObject("ActionButton7.BackgroundImage"), System.Drawing.Image)
-            ActionButton9.Enabled = False
+            ActionButton7.Enabled = False
-            ActionButton9.Name = "Clear"
+            ActionButton7.Name = "Clear"
-            ActionButton9.Visible = False
+            ActionButton7.Visible = False
-            Me.TXT_FOLDER_CMD.Buttons.Add(ActionButton9)
+            Me.TXT_FOLDER_CMD.Buttons.Add(ActionButton7)
             Me.TXT_FOLDER_CMD.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.CheckBox
             Me.TXT_FOLDER_CMD.CaptionText = "Folder cmd"
             Me.TXT_FOLDER_CMD.CaptionToolTipEnabled = True
@@ -1390,11 +1377,11 @@ Namespace Editors
             'TXT_CLOSE_SCRIPT
             '
             Me.TXT_CLOSE_SCRIPT.AutoShowClearButton = True
-            ActionButton10.BackgroundImage = CType(resources.GetObject("ActionButton10.BackgroundImage"), System.Drawing.Image)
+            ActionButton8.BackgroundImage = CType(resources.GetObject("ActionButton8.BackgroundImage"), System.Drawing.Image)
-            ActionButton10.Enabled = False
+            ActionButton8.Enabled = False
-            ActionButton10.Name = "Clear"
+            ActionButton8.Name = "Clear"
-            ActionButton10.Visible = False
+            ActionButton8.Visible = False
-            Me.TXT_CLOSE_SCRIPT.Buttons.Add(ActionButton10)
+            Me.TXT_CLOSE_SCRIPT.Buttons.Add(ActionButton8)
             Me.TXT_CLOSE_SCRIPT.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.CheckBox
             Me.TXT_CLOSE_SCRIPT.CaptionText = "Close cmd"
             Me.TXT_CLOSE_SCRIPT.CaptionToolTipEnabled = True
@@ -1578,12 +1565,12 @@ Namespace Editors
             '
             'TXT_SCRIPT
             '
-            ActionButton11.BackgroundImage = CType(resources.GetObject("ActionButton11.BackgroundImage"), System.Drawing.Image)
+            ActionButton9.BackgroundImage = CType(resources.GetObject("ActionButton9.BackgroundImage"), System.Drawing.Image)
-            ActionButton11.Name = "Open"
+            ActionButton9.Name = "Open"
-            ActionButton12.BackgroundImage = CType(resources.GetObject("ActionButton12.BackgroundImage"), System.Drawing.Image)
+            ActionButton10.BackgroundImage = CType(resources.GetObject("ActionButton10.BackgroundImage"), System.Drawing.Image)
-            ActionButton12.Name = "Clear"
+            ActionButton10.Name = "Clear"
-            Me.TXT_SCRIPT.Buttons.Add(ActionButton11)
+            Me.TXT_SCRIPT.Buttons.Add(ActionButton9)
-            Me.TXT_SCRIPT.Buttons.Add(ActionButton12)
+            Me.TXT_SCRIPT.Buttons.Add(ActionButton10)
             Me.TXT_SCRIPT.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.CheckBox
             Me.TXT_SCRIPT.CaptionText = "Script"
             Me.TXT_SCRIPT.CaptionToolTipEnabled = True
@@ -1901,14 +1888,14 @@ Namespace Editors
             '
             'NUM_FEED_SES_CURR_LOAD_LAST
             '
-            ActionButton13.BackgroundImage = CType(resources.GetObject("ActionButton13.BackgroundImage"), System.Drawing.Image)
+            ActionButton11.BackgroundImage = CType(resources.GetObject("ActionButton11.BackgroundImage"), System.Drawing.Image)
-            ActionButton13.Name = "Refresh"
+            ActionButton11.Name = "Refresh"
-            ActionButton13.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton11.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton14.BackgroundImage = CType(resources.GetObject("ActionButton14.BackgroundImage"), System.Drawing.Image)
+            ActionButton12.BackgroundImage = CType(resources.GetObject("ActionButton12.BackgroundImage"), System.Drawing.Image)
-            ActionButton14.Name = "Clear"
+            ActionButton12.Name = "Clear"
-            ActionButton14.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton12.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.NUM_FEED_SES_CURR_LOAD_LAST.Buttons.Add(ActionButton13)
+            Me.NUM_FEED_SES_CURR_LOAD_LAST.Buttons.Add(ActionButton11)
-            Me.NUM_FEED_SES_CURR_LOAD_LAST.Buttons.Add(ActionButton14)
+            Me.NUM_FEED_SES_CURR_LOAD_LAST.Buttons.Add(ActionButton12)
             Me.NUM_FEED_SES_CURR_LOAD_LAST.CaptionText = "Load last session"
             Me.NUM_FEED_SES_CURR_LOAD_LAST.CaptionToolTipEnabled = True
             Me.NUM_FEED_SES_CURR_LOAD_LAST.CaptionToolTipText = resources.GetString("NUM_FEED_SES_CURR_LOAD_LAST.CaptionToolTipText")
@@ -2040,19 +2027,19 @@ Namespace Editors
             '
             'TXT_YTDLP
             '
+            ActionButton13.BackgroundImage = CType(resources.GetObject("ActionButton13.BackgroundImage"), System.Drawing.Image)
+            ActionButton13.Name = "Open"
+            ActionButton13.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton14.BackgroundImage = CType(resources.GetObject("ActionButton14.BackgroundImage"), System.Drawing.Image)
+            ActionButton14.Name = "Refresh"
+            ActionButton14.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton14.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
             ActionButton15.BackgroundImage = CType(resources.GetObject("ActionButton15.BackgroundImage"), System.Drawing.Image)
-            ActionButton15.Name = "Open"
+            ActionButton15.Name = "Clear"
-            ActionButton15.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton15.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            ActionButton16.BackgroundImage = CType(resources.GetObject("ActionButton16.BackgroundImage"), System.Drawing.Image)
+            Me.TXT_YTDLP.Buttons.Add(ActionButton13)
-            ActionButton16.Name = "Refresh"
+            Me.TXT_YTDLP.Buttons.Add(ActionButton14)
-            ActionButton16.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton16.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
-            ActionButton17.BackgroundImage = CType(resources.GetObject("ActionButton17.BackgroundImage"), System.Drawing.Image)
-            ActionButton17.Name = "Clear"
-            ActionButton17.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
             Me.TXT_YTDLP.Buttons.Add(ActionButton15)
-            Me.TXT_YTDLP.Buttons.Add(ActionButton16)
-            Me.TXT_YTDLP.Buttons.Add(ActionButton17)
             Me.TXT_YTDLP.CaptionText = "yt-dlp"
             Me.TXT_YTDLP.CaptionToolTipEnabled = True
             Me.TXT_YTDLP.CaptionToolTipText = "Path to yt-dlp.exe file"
@@ -2066,19 +2053,19 @@ Namespace Editors
             '
             'TXT_FFMPEG
             '
+            ActionButton16.BackgroundImage = CType(resources.GetObject("ActionButton16.BackgroundImage"), System.Drawing.Image)
+            ActionButton16.Name = "Open"
+            ActionButton16.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton17.BackgroundImage = CType(resources.GetObject("ActionButton17.BackgroundImage"), System.Drawing.Image)
+            ActionButton17.Name = "Refresh"
+            ActionButton17.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton17.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
             ActionButton18.BackgroundImage = CType(resources.GetObject("ActionButton18.BackgroundImage"), System.Drawing.Image)
-            ActionButton18.Name = "Open"
+            ActionButton18.Name = "Clear"
-            ActionButton18.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton18.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            ActionButton19.BackgroundImage = CType(resources.GetObject("ActionButton19.BackgroundImage"), System.Drawing.Image)
+            Me.TXT_FFMPEG.Buttons.Add(ActionButton16)
-            ActionButton19.Name = "Refresh"
+            Me.TXT_FFMPEG.Buttons.Add(ActionButton17)
-            ActionButton19.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton19.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
-            ActionButton20.BackgroundImage = CType(resources.GetObject("ActionButton20.BackgroundImage"), System.Drawing.Image)
-            ActionButton20.Name = "Clear"
-            ActionButton20.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
             Me.TXT_FFMPEG.Buttons.Add(ActionButton18)
-            Me.TXT_FFMPEG.Buttons.Add(ActionButton19)
-            Me.TXT_FFMPEG.Buttons.Add(ActionButton20)
             Me.TXT_FFMPEG.CaptionText = "ffmpeg"
             Me.TXT_FFMPEG.CaptionToolTipEnabled = True
             Me.TXT_FFMPEG.CaptionToolTipText = "Path to ffmpeg.exe file"
@@ -2092,19 +2079,19 @@ Namespace Editors
             '
             'TXT_CURL
             '
+            ActionButton19.BackgroundImage = CType(resources.GetObject("ActionButton19.BackgroundImage"), System.Drawing.Image)
+            ActionButton19.Name = "Open"
+            ActionButton19.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton20.BackgroundImage = CType(resources.GetObject("ActionButton20.BackgroundImage"), System.Drawing.Image)
+            ActionButton20.Name = "Refresh"
+            ActionButton20.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton20.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
             ActionButton21.BackgroundImage = CType(resources.GetObject("ActionButton21.BackgroundImage"), System.Drawing.Image)
-            ActionButton21.Name = "Open"
+            ActionButton21.Name = "Clear"
-            ActionButton21.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton21.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            ActionButton22.BackgroundImage = CType(resources.GetObject("ActionButton22.BackgroundImage"), System.Drawing.Image)
+            Me.TXT_CURL.Buttons.Add(ActionButton19)
-            ActionButton22.Name = "Refresh"
+            Me.TXT_CURL.Buttons.Add(ActionButton20)
-            ActionButton22.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton22.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
-            ActionButton23.BackgroundImage = CType(resources.GetObject("ActionButton23.BackgroundImage"), System.Drawing.Image)
-            ActionButton23.Name = "Clear"
-            ActionButton23.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
             Me.TXT_CURL.Buttons.Add(ActionButton21)
-            Me.TXT_CURL.Buttons.Add(ActionButton22)
-            Me.TXT_CURL.Buttons.Add(ActionButton23)
             Me.TXT_CURL.CaptionText = "cURL"
             Me.TXT_CURL.CaptionToolTipEnabled = True
             Me.TXT_CURL.CaptionToolTipText = "Path to curl.exe file"
@@ -2118,19 +2105,19 @@ Namespace Editors
             '
             'TXT_GALLERYDL
             '
+            ActionButton22.BackgroundImage = CType(resources.GetObject("ActionButton22.BackgroundImage"), System.Drawing.Image)
+            ActionButton22.Name = "Open"
+            ActionButton22.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton23.BackgroundImage = CType(resources.GetObject("ActionButton23.BackgroundImage"), System.Drawing.Image)
+            ActionButton23.Name = "Refresh"
+            ActionButton23.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton23.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
             ActionButton24.BackgroundImage = CType(resources.GetObject("ActionButton24.BackgroundImage"), System.Drawing.Image)
-            ActionButton24.Name = "Open"
+            ActionButton24.Name = "Clear"
-            ActionButton24.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton24.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            ActionButton25.BackgroundImage = CType(resources.GetObject("ActionButton25.BackgroundImage"), System.Drawing.Image)
+            Me.TXT_GALLERYDL.Buttons.Add(ActionButton22)
-            ActionButton25.Name = "Refresh"
+            Me.TXT_GALLERYDL.Buttons.Add(ActionButton23)
-            ActionButton25.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton25.ToolTipText = "Try to find this program automatically (in SCrawler and/or system environment)"
-            ActionButton26.BackgroundImage = CType(resources.GetObject("ActionButton26.BackgroundImage"), System.Drawing.Image)
-            ActionButton26.Name = "Clear"
-            ActionButton26.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
             Me.TXT_GALLERYDL.Buttons.Add(ActionButton24)
-            Me.TXT_GALLERYDL.Buttons.Add(ActionButton25)
-            Me.TXT_GALLERYDL.Buttons.Add(ActionButton26)
             Me.TXT_GALLERYDL.CaptionText = "gallery-dl"
             Me.TXT_GALLERYDL.CaptionToolTipText = "Path to gallery-dl.exe file"
             Me.TXT_GALLERYDL.CaptionWidth = 80.0R
@@ -2143,14 +2130,14 @@ Namespace Editors
             '
             'TXT_CMD_ENCODING
             '
-            ActionButton27.BackgroundImage = CType(resources.GetObject("ActionButton27.BackgroundImage"), System.Drawing.Image)
+            ActionButton25.BackgroundImage = CType(resources.GetObject("ActionButton25.BackgroundImage"), System.Drawing.Image)
-            ActionButton27.Name = "Refresh"
+            ActionButton25.Name = "Refresh"
-            ActionButton27.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
+            ActionButton25.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Refresh
-            ActionButton28.BackgroundImage = CType(resources.GetObject("ActionButton28.BackgroundImage"), System.Drawing.Image)
+            ActionButton26.BackgroundImage = CType(resources.GetObject("ActionButton26.BackgroundImage"), System.Drawing.Image)
-            ActionButton28.Name = "Clear"
+            ActionButton26.Name = "Clear"
-            ActionButton28.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton26.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_CMD_ENCODING.Buttons.Add(ActionButton27)
+            Me.TXT_CMD_ENCODING.Buttons.Add(ActionButton25)
-            Me.TXT_CMD_ENCODING.Buttons.Add(ActionButton28)
+            Me.TXT_CMD_ENCODING.Buttons.Add(ActionButton26)
             Me.TXT_CMD_ENCODING.CaptionText = "CMD Encoding"
             Me.TXT_CMD_ENCODING.CaptionToolTipEnabled = True
             Me.TXT_CMD_ENCODING.CaptionToolTipText = "Command line encoding"
@@ -2253,10 +2240,10 @@ Namespace Editors
             '
             'CMB_STD_OPEN_DBL
             '
-            ActionButton29.BackgroundImage = CType(resources.GetObject("ActionButton29.BackgroundImage"), System.Drawing.Image)
+            ActionButton27.BackgroundImage = CType(resources.GetObject("ActionButton27.BackgroundImage"), System.Drawing.Image)
-            ActionButton29.Name = "ArrowDown"
+            ActionButton27.Name = "ArrowDown"
-            ActionButton29.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.ArrowDown
+            ActionButton27.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.ArrowDown
-            Me.CMB_STD_OPEN_DBL.Buttons.Add(ActionButton29)
+            Me.CMB_STD_OPEN_DBL.Buttons.Add(ActionButton27)
             Me.CMB_STD_OPEN_DBL.CaptionMode = PersonalUtilities.Forms.Controls.Base.ICaptionControl.Modes.Label
             Me.CMB_STD_OPEN_DBL.CaptionText = "DoubleClick opens"
             Me.CMB_STD_OPEN_DBL.CaptionToolTipEnabled = True
@@ -2338,10 +2325,10 @@ Namespace Editors
             '
             'TXT_PRG_TITLE
             '
-            ActionButton30.BackgroundImage = CType(resources.GetObject("ActionButton30.BackgroundImage"), System.Drawing.Image)
+            ActionButton28.BackgroundImage = CType(resources.GetObject("ActionButton28.BackgroundImage"), System.Drawing.Image)
-            ActionButton30.Name = "Clear"
+            ActionButton28.Name = "Clear"
-            ActionButton30.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton28.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_PRG_TITLE.Buttons.Add(ActionButton30)
+            Me.TXT_PRG_TITLE.Buttons.Add(ActionButton28)
             Me.TXT_PRG_TITLE.CaptionText = "Program title"
             Me.TXT_PRG_TITLE.CaptionToolTipEnabled = True
             Me.TXT_PRG_TITLE.CaptionToolTipText = "Change the title of the main window if you need to"
@@ -2353,10 +2340,10 @@ Namespace Editors
             '
             'TXT_PRG_DESCR
             '
-            ActionButton31.BackgroundImage = CType(resources.GetObject("ActionButton31.BackgroundImage"), System.Drawing.Image)
+            ActionButton29.BackgroundImage = CType(resources.GetObject("ActionButton29.BackgroundImage"), System.Drawing.Image)
-            ActionButton31.Name = "Clear"
+            ActionButton29.Name = "Clear"
-            ActionButton31.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton29.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_PRG_DESCR.Buttons.Add(ActionButton31)
+            Me.TXT_PRG_DESCR.Buttons.Add(ActionButton29)
             Me.TXT_PRG_DESCR.CaptionText = "Program description"
             Me.TXT_PRG_DESCR.CaptionToolTipEnabled = True
             Me.TXT_PRG_DESCR.CaptionToolTipText = "Add some additional info to the program info if you need"
@@ -2368,14 +2355,14 @@ Namespace Editors
             '
             'TXT_USER_LIST_IMAGE
             '
-            ActionButton32.BackgroundImage = CType(resources.GetObject("ActionButton32.BackgroundImage"), System.Drawing.Image)
+            ActionButton30.BackgroundImage = CType(resources.GetObject("ActionButton30.BackgroundImage"), System.Drawing.Image)
-            ActionButton32.Name = "Open"
+            ActionButton30.Name = "Open"
-            ActionButton32.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
+            ActionButton30.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Open
-            ActionButton33.BackgroundImage = CType(resources.GetObject("ActionButton33.BackgroundImage"), System.Drawing.Image)
+            ActionButton31.BackgroundImage = CType(resources.GetObject("ActionButton31.BackgroundImage"), System.Drawing.Image)
-            ActionButton33.Name = "Clear"
+            ActionButton31.Name = "Clear"
-            ActionButton33.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton31.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_USER_LIST_IMAGE.Buttons.Add(ActionButton32)
+            Me.TXT_USER_LIST_IMAGE.Buttons.Add(ActionButton30)
-            Me.TXT_USER_LIST_IMAGE.Buttons.Add(ActionButton33)
+            Me.TXT_USER_LIST_IMAGE.Buttons.Add(ActionButton31)
             Me.TXT_USER_LIST_IMAGE.CaptionText = "Userlist image"
             Me.TXT_USER_LIST_IMAGE.CaptionToolTipEnabled = True
             Me.TXT_USER_LIST_IMAGE.CaptionToolTipText = "Background image for user list"
@@ -2449,10 +2436,10 @@ Namespace Editors
             '
             'TXT_H_DEF_UserAgent
             '
-            ActionButton34.BackgroundImage = CType(resources.GetObject("ActionButton34.BackgroundImage"), System.Drawing.Image)
+            ActionButton32.BackgroundImage = CType(resources.GetObject("ActionButton32.BackgroundImage"), System.Drawing.Image)
-            ActionButton34.Name = "Clear"
+            ActionButton32.Name = "Clear"
-            ActionButton34.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton32.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_H_DEF_UserAgent.Buttons.Add(ActionButton34)
+            Me.TXT_H_DEF_UserAgent.Buttons.Add(ActionButton32)
             Me.TXT_H_DEF_UserAgent.CaptionText = "UserAgent"
             Me.TXT_H_DEF_UserAgent.CaptionWidth = 140.0R
             Me.TXT_H_DEF_UserAgent.Dock = System.Windows.Forms.DockStyle.Fill
@@ -2463,10 +2450,10 @@ Namespace Editors
             '
             'TXT_H_DEF_sec_ch_ua
             '
-            ActionButton35.BackgroundImage = CType(resources.GetObject("ActionButton35.BackgroundImage"), System.Drawing.Image)
+            ActionButton33.BackgroundImage = CType(resources.GetObject("ActionButton33.BackgroundImage"), System.Drawing.Image)
-            ActionButton35.Name = "Clear"
+            ActionButton33.Name = "Clear"
-            ActionButton35.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton33.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_H_DEF_sec_ch_ua.Buttons.Add(ActionButton35)
+            Me.TXT_H_DEF_sec_ch_ua.Buttons.Add(ActionButton33)
             Me.TXT_H_DEF_sec_ch_ua.CaptionText = "sec-ch-ua"
             Me.TXT_H_DEF_sec_ch_ua.CaptionWidth = 140.0R
             Me.TXT_H_DEF_sec_ch_ua.Dock = System.Windows.Forms.DockStyle.Fill
@@ -2477,10 +2464,10 @@ Namespace Editors
             '
             'TXT_H_DEF_sec_ch_ua_full_version_list
             '
-            ActionButton36.BackgroundImage = CType(resources.GetObject("ActionButton36.BackgroundImage"), System.Drawing.Image)
+            ActionButton34.BackgroundImage = CType(resources.GetObject("ActionButton34.BackgroundImage"), System.Drawing.Image)
-            ActionButton36.Name = "Clear"
+            ActionButton34.Name = "Clear"
-            ActionButton36.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton34.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_H_DEF_sec_ch_ua_full_version_list.Buttons.Add(ActionButton36)
+            Me.TXT_H_DEF_sec_ch_ua_full_version_list.Buttons.Add(ActionButton34)
             Me.TXT_H_DEF_sec_ch_ua_full_version_list.CaptionText = "sec-ch-ua-full-version-list"
             Me.TXT_H_DEF_sec_ch_ua_full_version_list.CaptionWidth = 140.0R
             Me.TXT_H_DEF_sec_ch_ua_full_version_list.Dock = System.Windows.Forms.DockStyle.Fill
@@ -2491,10 +2478,10 @@ Namespace Editors
             '
             'TXT_H_DEF_sec_ch_ua_platform
             '
-            ActionButton37.BackgroundImage = CType(resources.GetObject("ActionButton37.BackgroundImage"), System.Drawing.Image)
+            ActionButton35.BackgroundImage = CType(resources.GetObject("ActionButton35.BackgroundImage"), System.Drawing.Image)
-            ActionButton37.Name = "Clear"
+            ActionButton35.Name = "Clear"
-            ActionButton37.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton35.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_H_DEF_sec_ch_ua_platform.Buttons.Add(ActionButton37)
+            Me.TXT_H_DEF_sec_ch_ua_platform.Buttons.Add(ActionButton35)
             Me.TXT_H_DEF_sec_ch_ua_platform.CaptionText = "sec-ch-ua-platform"
             Me.TXT_H_DEF_sec_ch_ua_platform.CaptionWidth = 140.0R
             Me.TXT_H_DEF_sec_ch_ua_platform.Dock = System.Windows.Forms.DockStyle.Fill
@@ -2505,10 +2492,10 @@ Namespace Editors
             '
             'TXT_H_DEF_sec_ch_ua_platform_version
             '
-            ActionButton38.BackgroundImage = CType(resources.GetObject("ActionButton38.BackgroundImage"), System.Drawing.Image)
+            ActionButton36.BackgroundImage = CType(resources.GetObject("ActionButton36.BackgroundImage"), System.Drawing.Image)
-            ActionButton38.Name = "Clear"
+            ActionButton36.Name = "Clear"
-            ActionButton38.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            ActionButton36.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
-            Me.TXT_H_DEF_sec_ch_ua_platform_version.Buttons.Add(ActionButton38)
+            Me.TXT_H_DEF_sec_ch_ua_platform_version.Buttons.Add(ActionButton36)
             Me.TXT_H_DEF_sec_ch_ua_platform_version.CaptionText = "sec-ch-ua-platform-version"
             Me.TXT_H_DEF_sec_ch_ua_platform_version.CaptionWidth = 140.0R
             Me.TXT_H_DEF_sec_ch_ua_platform_version.Dock = System.Windows.Forms.DockStyle.Fill
@@ -2571,19 +2558,6 @@ Namespace Editors
             Me.CONTAINER_MAIN.TabIndex = 0
             Me.CONTAINER_MAIN.TopToolStripPanelVisible = False
             '
-            'CH_FEED_OPEN_CTRLF
-            '
-            Me.CH_FEED_OPEN_CTRLF.AutoSize = True
-            Me.CH_FEED_OPEN_CTRLF.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_FEED_OPEN_CTRLF.Location = New System.Drawing.Point(4, 299)
-            Me.CH_FEED_OPEN_CTRLF.Name = "CH_FEED_OPEN_CTRLF"
-            Me.CH_FEED_OPEN_CTRLF.Size = New System.Drawing.Size(613, 19)
-            Me.CH_FEED_OPEN_CTRLF.TabIndex = 11
-            Me.CH_FEED_OPEN_CTRLF.Text = "Use 'Ctrl+F' to open the Feed"
-            TT_MAIN.SetToolTip(Me.CH_FEED_OPEN_CTRLF, "If checked, 'Ctrl+F' will be used to open the Feed. Otherwise, 'Alt+F' will be us" &
-        "ed.")
-            Me.CH_FEED_OPEN_CTRLF.UseVisualStyleBackColor = True
-            '
             'GlobalSettingsForm
             '
             Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
@@ -2611,7 +2585,6 @@ Namespace Editors
             CType(Me.TXT_MAX_JOBS_USERS, System.ComponentModel.ISupportInitialize).EndInit()
             CType(Me.TXT_MAX_JOBS_CHANNELS, System.ComponentModel.ISupportInitialize).EndInit()
             CType(Me.TXT_IMGUR_CLIENT_ID, System.ComponentModel.ISupportInitialize).EndInit()
-            CType(Me.TXT_USER_AGENT, System.ComponentModel.ISupportInitialize).EndInit()
             TP_FILE_NAME.ResumeLayout(False)
             TP_FILE_NAME.PerformLayout()
             TP_FILE_PATTERNS.ResumeLayout(False)
@@ -2749,7 +2722,6 @@ Namespace Editors
         Private WithEvents CH_NOTIFY_AUTO_DOWN As CheckBox
         Private WithEvents CH_NOTIFY_CHANNELS As CheckBox
         Private WithEvents CH_DOWN_REPARSE_MISSING As CheckBox
-        Private WithEvents TXT_USER_AGENT As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents TXT_USER_LIST_IMAGE As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents TXT_FEED_CENTER_IMAGE As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents CH_NAME_SITE_FRIENDLY As CheckBox

SCrawler/Editors/GlobalSettingsForm.resx

@@ -189,30 +189,6 @@
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
         tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton7.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
-        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
-        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
-        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
-        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
-        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
-        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
-        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
-        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
-        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
-        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
-        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
-</value>
-  </data>
-  <data name="ActionButton8.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
 </value>
   </data>
   <metadata name="TP_FILE_NAME.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
@@ -269,7 +245,7 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_BEHAVIOR.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <data name="ActionButton9.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton7.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -277,7 +253,7 @@ You can find more detailed information about the missing posts in the form that
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
 </value>
   </data>
-  <data name="ActionButton10.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton8.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -300,7 +276,7 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_DOWNLOADING.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <data name="ActionButton11.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton9.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
@@ -311,7 +287,7 @@ You can find more detailed information about the missing posts in the form that
         cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
-  <data name="ActionButton12.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton10.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -334,7 +310,7 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_FEED_SES.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <data name="ActionButton13.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton11.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
         JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
@@ -350,7 +326,7 @@ You can find more detailed information about the missing posts in the form that
         VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
 </value>
   </data>
-  <data name="ActionButton14.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton12.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -376,7 +352,7 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_ENVIR.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <data name="ActionButton15.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton13.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
@@ -387,7 +363,7 @@ You can find more detailed information about the missing posts in the form that
         cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
-  <data name="ActionButton16.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton14.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
         JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
@@ -403,15 +379,50 @@ You can find more detailed information about the missing posts in the form that
         VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
 </value>
   </data>
-  <data name="ActionButton17.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton15.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
         tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton16.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
+        WQwhyWIyJIUW5NqyPb7oCVtIlhVTwYf8nv7/t2zJagel9KmqKsIACYL9RjI8UHz5zshougZr/AEvbxEP
+        aZCDBY3VslixaJvX3wzkkDiOwbZtDRGA5vdNAg+TL27qgmt5XkBG/gTdAG7Gt+3PP9oOaEGFCVEC6rp+
+        5g9MfM/c5e4OsEZMZkQEtGL5H2DdZ5JRArDwPA+iKII0TfkC9vroC9j5vq8JTWw3WzWgLMtZGIaa0MR8
+        vlAD8PYlSaIJTTiOowY0p0Bc19XEJo6HE59FAPuMzyAINKGJ1XLFZxHALtMrnkBXOIQIIIQ8YvF/KrgB
+        cMaRN0UdBBkAAAAASUVORK5CYII=
+</value>
+  </data>
+  <data name="ActionButton17.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
+        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
+        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
+        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
+        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
+        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
+        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
+        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
+        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
+        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
+        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
+        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
 </value>
   </data>
   <data name="ActionButton18.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton19.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
@@ -422,7 +433,7 @@ You can find more detailed information about the missing posts in the form that
         cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
-  <data name="ActionButton19.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton20.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
         JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
@@ -436,17 +447,17 @@ You can find more detailed information about the missing posts in the form that
         HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
         qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
         VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
-</value>
-  </data>
-  <data name="ActionButton20.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
 </value>
   </data>
   <data name="ActionButton21.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton22.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
@@ -457,7 +468,7 @@ You can find more detailed information about the missing posts in the form that
         cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
-  <data name="ActionButton22.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton23.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
         JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
@@ -473,23 +484,12 @@ You can find more detailed information about the missing posts in the form that
         VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
 </value>
   </data>
-  <data name="ActionButton23.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton24.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
         tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton24.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
-        WQwhyWIyJIUW5NqyPb7oCVtIlhVTwYf8nv7/t2zJagel9KmqKsIACYL9RjI8UHz5zshougZr/AEvbxEP
-        aZCDBY3VslixaJvX3wzkkDiOwbZtDRGA5vdNAg+TL27qgmt5XkBG/gTdAG7Gt+3PP9oOaEGFCVEC6rp+
-        5g9MfM/c5e4OsEZMZkQEtGL5H2DdZ5JRArDwPA+iKII0TfkC9vroC9j5vq8JTWw3WzWgLMtZGIaa0MR8
-        vlAD8PYlSaIJTTiOowY0p0Bc19XEJo6HE59FAPuMzyAINKGJ1XLFZxHALtMrnkBXOIQIIIQ8YvF/KrgB
-        cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
   <data name="ActionButton25.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
@@ -514,30 +514,6 @@ You can find more detailed information about the missing posts in the form that
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
         tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton27.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAAB6
-        JQAAgIMAAPn/AACA6QAAdTAAAOpgAAA6mAAAF2+SX8VGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOElE
-        QVQ4T2P4//8/QczOJyyqHpzfiE0OQwAZC8iqszAzs7CJ69o4BR768V/W2jcGXQ0KB4aFNS3dDQtnrbCb
-        ePCK48wTN1wXXXzge/jXf/clV55zC4hIIatF0cjIyMikElzc57z0wX+XHd/+2+//99/ywP//xlu//tdb
-        +eK/4Zp3/1WTOhYzARViNUAluKjTdf37/0ZTTn9TbdhwXblhwwW1/qOP1Ja9+K8w+95/6cm3/6v2Xvkv
-        qKjniGGAoIqRpW3/4e8S9uGdzFz82gwMDFxAzCxm4ZegtuLDf+VJ1/8rZM25IqLvnM/CximCYYCic1QN
-        v7x2JIwPwyrJ3XNUylddE9G2TWNmZOBDl4czmJiZMSRBmFdSyYyJgUEQmxwIYxWEYXZBCUls4sgYq6CA
-        prWNbtG8nXKeaVPR5XiVjSxEzf0yYXy4BBMLO6eQjoOXZvrkbbazrv53Xf/2v4CSbjBMXkhBl1/CMyNZ
-        qWnvGy5pNQ+YONwAfjXzAOupl/47LLr333L50/96q9/8l23YdES6cO5KuYqVW+R7Tj6SnfP0v4hryjyY
-        HhQDmFjYeHVKFp7WX/Xuv9Kq9/+Vd/z7r7rv/3+l7f//y676DEwDN/9L+BVvYkKLCTgDhNkkVUyVlr74
-        qbbz73/VOTc/qsy89kWx+9h7qbQpJwS1bbOAscGGrB6EUTggLOqf16C55ft/HlnNAFZOXgVWdi4FRgYG
-        VnR1MIwhwMTCyqEQ37qEmZVDFF0OE/9nAACtFF4Ey6OP+wAAAABJRU5ErkJggg==
-</value>
-  </data>
-  <data name="ActionButton28.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
 </value>
   </data>
   <metadata name="TAB_STD.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
@@ -546,7 +522,7 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_STD.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
-  <data name="ActionButton29.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton27.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAYAAAD0eNT6AAAABGdBTUEAALGPC/xhBQAAE65JREFUeF7t
         3X2sJWddB/DdLi2lQG2hdOHuvfM887J7Cxca4ELTQMDWKigIFpBAEAgi9g+CJpJo9Q8NJhgBiYZIYspL
@@ -642,23 +618,23 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_DESIGN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
+  <data name="ActionButton28.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton29.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
   <data name="ActionButton30.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton31.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton32.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         wwAADsMBx2+oZAAAAR5JREFUOE+VkjFqwzAUhn2D9iShRyi+QhYbGujg3ZATZPKYdC6FQhPwlAMkg3dP
@@ -669,7 +645,7 @@ You can find more detailed information about the missing posts in the form that
         cMaRN0UdBBkAAAAASUVORK5CYII=
 </value>
   </data>
-  <data name="ActionButton33.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+  <data name="ActionButton31.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
@@ -680,6 +656,22 @@ You can find more detailed information about the missing posts in the form that
   <metadata name="TP_HEADERS_DEF.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
+  <data name="ActionButton32.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton33.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
   <data name="ActionButton34.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
@@ -702,22 +694,6 @@ You can find more detailed information about the missing posts in the form that
         xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
         tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
         AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton37.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
-</value>
-  </data>
-  <data name="ActionButton38.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
-    <value>
-        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
-        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
-        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
 </value>
   </data>
   <metadata name="TAB_HEADERS.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">

SCrawler/Editors/GlobalSettingsForm.vb

@@ -38,7 +38,6 @@ Namespace Editors
                         TXT_MAX_JOBS_USERS.Value = .MaxUsersJobsCount.Value
                         TXT_MAX_JOBS_CHANNELS.Value = .ChannelsMaxJobsCount.Value
                         CH_CHECK_VER_START.Checked = .CheckUpdatesAtStart
-                        TXT_USER_AGENT.Text = .UserAgent
                         TXT_IMGUR_CLIENT_ID.Text = .ImgurClientID
                         'Design
                         TXT_PRG_TITLE.Text = .ProgramText
@@ -255,8 +254,6 @@ Namespace Editors
                     .MaxUsersJobsCount.Value = CInt(TXT_MAX_JOBS_USERS.Value)
                     .ChannelsMaxJobsCount.Value = TXT_MAX_JOBS_CHANNELS.Value
                     .CheckUpdatesAtStart.Value = CH_CHECK_VER_START.Checked
-                    .UserAgent.Value = TXT_USER_AGENT.Text
-                    UserAgentChanged = .UserAgent.ChangesDetected
                     .ImgurClientID.Value = TXT_IMGUR_CLIENT_ID.Text
                     'Design
                     .ProgramText.Value = TXT_PRG_TITLE.Text
@@ -276,6 +273,7 @@ Namespace Editors
                                                  .CMDEncoding.ChangesDetected
                     'Headers
                     .HEADER_UserAgent.Value = TXT_H_DEF_UserAgent.Text
+                    UserAgentChanged = .HEADER_UserAgent.ChangesDetected
                     .HEADER_sec_ch_ua.Value = TXT_H_DEF_sec_ch_ua.Text
                     .HEADER_sec_ch_ua_full_version_list.Value = TXT_H_DEF_sec_ch_ua_full_version_list.Text
                     .HEADER_sec_ch_ua_platform.Value = TXT_H_DEF_sec_ch_ua_platform.Text
@@ -407,9 +405,6 @@ Namespace Editors
         Private Sub TXT_MAX_JOBS_CHANNELS_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles TXT_MAX_JOBS_CHANNELS.ActionOnButtonClick
             If Sender.DefaultButton = ADB.Refresh Then TXT_MAX_JOBS_CHANNELS.Value = SettingsCLS.DefaultMaxDownloadingTasks
         End Sub
-        Private Sub TXT_USER_AGENT_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As ActionButtonEventArgs) Handles TXT_USER_AGENT.ActionOnButtonClick
-            If Sender.DefaultButton = ADB.Refresh Then TXT_USER_AGENT.Text = Settings.UserAgent.Value
-        End Sub
         Private Sub ChangePositionControlsEnabling() Handles OPT_FILE_NAME_REPLACE.CheckedChanged, OPT_FILE_NAME_ADD_DATE.CheckedChanged
             Dim b As Boolean = OPT_FILE_NAME_ADD_DATE.Checked And OPT_FILE_NAME_ADD_DATE.Enabled
             OPT_FILE_DATE_START.Enabled = b

SCrawler/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2025.1.12.0")>
+<Assembly: AssemblyVersion("2025.2.25.0")>
-<Assembly: AssemblyFileVersion("2025.1.12.0")>
+<Assembly: AssemblyFileVersion("2025.2.25.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/PluginsEnvironment/Hosts/PluginHost.vb

@@ -88,6 +88,7 @@ Namespace Plugin.Hosts
                                           ByRef _Vids As XMLValue(Of Boolean)) As IEnumerable(Of PluginHost)
             Return {New PluginHost(GetType(API.Reddit.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(GetType(API.Twitter.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),
+                    New PluginHost(GetType(API.Bluesky.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(GetType(API.Mastodon.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(GetType(API.Instagram.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(GetType(API.ThreadsNet.SiteSettings), _XML, GlobalPath, _Temp, _Imgs, _Vids),

SCrawler/PluginsEnvironment/Hosts/UserDataHost.vb

@@ -48,6 +48,7 @@ Namespace Plugin.Hosts
                 .Thrower = Me
                 .LogProvider = LogConnector
                 .Name = Name
+                .NameTrue = NameTrue(True)
                 .ID = ID
                 .Options = Options
                 .ParseUserMediaOnly = ParseUserMediaOnly
@@ -78,6 +79,7 @@ Namespace Plugin.Hosts
                 UserDescriptionUpdate(.UserDescription)
                 UserExists = .UserExists
                 UserSuspended = .UserSuspended
+                NameTrue = .NameTrue
             End With
         End Sub
         Friend Overrides Sub DownloadSingleObject(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)

SCrawler/SCrawler.vbproj

@@ -168,6 +168,7 @@
     <Compile Include="API\Base\Declarations.vb" />
     <Compile Include="API\Base\DeclaredNames.vb" />
     <Compile Include="API\Base\DownDetector.vb" />
+    <Compile Include="API\Base\EditorExchangeOptionsBase.vb" />
     <Compile Include="API\Base\GDL.vb" />
     <Compile Include="API\Base\IUserData.vb" />
     <Compile Include="API\Base\M3U8Base.vb" />
@@ -189,6 +190,10 @@
     <Compile Include="API\Base\Structures.vb" />
     <Compile Include="API\Base\TokenBatch.vb" />
     <Compile Include="API\Base\YTDLP.vb" />
+    <Compile Include="API\Bluesky\Declarations.vb" />
+    <Compile Include="API\Bluesky\M3U8.vb" />
+    <Compile Include="API\Bluesky\SiteSettings.vb" />
+    <Compile Include="API\Bluesky\UserData.vb" />
     <Compile Include="API\Facebook\Declarations.vb" />
     <Compile Include="API\Facebook\SiteSettings.vb" />
     <Compile Include="API\Facebook\UserData.vb" />
@@ -227,6 +232,7 @@
     <Compile Include="API\PathPlugin\SiteSettings.vb" />
     <Compile Include="API\PathPlugin\UserData.vb" />
     <Compile Include="API\Pinterest\Declarations.vb" />
+    <Compile Include="API\Pinterest\EditorExchangeOptions.vb" />
     <Compile Include="API\Pinterest\SiteSettings.vb" />
     <Compile Include="API\Pinterest\UserData.vb" />
     <Compile Include="API\PornHub\Declarations.vb" />
@@ -660,6 +666,7 @@
   <ItemGroup>
     <None Include=".editorconfig" />
     <None Include="API\OnlyFans\OFScraperConfigPattern.json" />
+    <None Include="Content\Icons\SiteIcons\TwitterIconNew_32.ico" />
     <None Include="Content\Pictures\ApplicationPic_16.png" />
     <None Include="Content\Pictures\BookmarkBlack_16.png" />
     <None Include="Content\Pictures\DBPic_32.png" />
@@ -797,6 +804,8 @@
     <Content Include="API\OnlyFans\DynamicRules.txt" />
     <Content Include="API\OnlyFans\DynamicRulesAll.txt" />
     <Content Include="API\OnlyFans\OFScraperConfigPatternConstants.txt" />
+    <Content Include="Content\Icons\SiteIcons\BlueskyIcon_32.ico" />
+    <Content Include="Content\Pictures\SitePictures\BlueskyPic_32.png" />
     <Content Include="Content\Pictures\CutPic_48.png" />
     <Content Include="Content\Pictures\FindPic_16.png" />
   </ItemGroup>

SCrawler/SettingsCLS.vb

@@ -173,6 +173,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
     Friend ReadOnly Plugins As List(Of PluginHost)
     Friend ReadOnly Property Users As List(Of IUserData)
     Friend ReadOnly Property UsersList As List(Of UserInfo)
+    Private ReadOnly Property UsersListProtected As Boolean = False
     Friend Property Channels As Reddit.ChannelsCollection
     Friend ReadOnly Property Labels As LabelsKeeper
     Friend ReadOnly Property Groups As Groups.DownloadGroupCollection
@@ -193,7 +194,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
     Private ReadOnly BlackListFile As SFile = $"{SettingsFolderName}\BlackList.txt"
     Private ReadOnly UsersSettingsFile As SFile = $"{SettingsFolderName}\Users.xml"
     Private ReadOnly Property SettingsVersion As XMLValue(Of Integer)
-    Private Const SettingsVersionCurrent As Integer = 1
+    Private Const SettingsVersionCurrent As Integer = 2
     Friend ShortcutOpenFeed As New ButtonKey(Keys.F, True)
     Friend ShortcutOpenSearch As New ButtonKey(Keys.F,, True)
     Private Sub ChangeFeedOpenMode()
@@ -228,10 +229,10 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
         EnvironmentProgramsList = New List(Of String)
 
         SettingsVersion = New XMLValue(Of Integer)("SettingsVersion", 0, MyXML)
+        UsersListProtected = MyXML.Value("UsersListProtected").FromXML(Of Boolean)(False)
 
         Dim n() As String = {"Scheduler"}
         AutomationFile = New XMLValue(Of String)("File",, MyXML, n)
-        If SettingsVersion.Value = 0 AndAlso MyXML.Contains(AutomationFile.Name) Then AutomationFile.Value = MyXML.Value(AutomationFile.Name)
         AutomationScript = New XMLValueUse(Of String)("Script", String.Empty,, MyXML, n)
         AutomationScript_ExcludeManual = New XMLValue(Of Boolean)("ScriptExcludeManual", True, MyXML, n)
 
@@ -264,7 +265,6 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
         MaxSmallImageHeight = New XMLValue(Of Integer)("MaxSmallImageHeight", 15, MyXML, n)
         CollectionsPath = New XMLValue(Of String)("CollectionsPath", CollectionsFolderName, MyXML, n)
         MaxUsersJobsCount = New XMLValue(Of Integer)("MaxJobsCount", DefaultMaxDownloadingTasks, MyXML, n)
-        UserAgent = New XMLValue(Of String)("UserAgent",, MyXML, n)
         ImgurClientID = New XMLValue(Of String)("ImgurClientID", String.Empty, MyXML, {Name_Node_Sites})
 
         'Basis: new version
@@ -333,7 +333,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
         DefaultDownloadImages = New XMLValue(Of Boolean)("DownloadImages", True, MyXML, n)
         DefaultDownloadVideos = New XMLValue(Of Boolean)("DownloadVideos", True, MyXML, n)
         DownloadNativeImageFormat = New XMLValue(Of Boolean)("DownloadNativeImageFormat", True, MyXML, n)
-        UserSiteNameAsFriendly = New XMLValue(Of Boolean)("UserSiteNameAsFriendly", False, MyXML, n)
+        UserSiteNameAsFriendly = New XMLValue(Of Boolean)("UserSiteNameAsFriendly", True, MyXML, n)
 
         'STDownloader
         n = {"Downloader"}
@@ -367,6 +367,8 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
         UseDefaultAccountIfMissing = New XMLValue(Of Boolean)("UseDefaultAccountIfMissing", True, MyXML, n)
         AutomationBrushUndownloadedPlansMinutes = New XMLValue(Of Integer)("AutomationBrushUndownloadedPlansMinutes", 10080, MyXML, n)
         DownDetectorEnabled = New XMLValue(Of Boolean)("DownDetectorEnabled", True, MyXML, n)
+        'TODELETE: DownDetectorEnabled change
+        If SettingsVersion.Value < SettingsVersionCurrent Then DownDetectorEnabled.Value = False 'SettingsVersionCurrent = 2
 
         'Downloading: file naming
         n = {"Downloading", "FileName"}
@@ -516,6 +518,8 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
 
                 Dim NeedUpdate As Boolean = False
                 Dim i%, indx%
+                Dim errStr As Func(Of Date, String) = Function(d) IIf(UsersListProtected, String.Empty,
+                                                                      $"It will be removed from SCrawler on {d.ToStringDate(DateTimeDefaultProvider)}.")
                 Dim UsersListInitialCount% = UsersList.Count
                 Dim iUser As UserInfo
                 Dim userFileExists As Boolean, pluginFound As Boolean
@@ -531,7 +535,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
                             If .Plugin.IsEmptyString Then
                                 pluginFound = False
                                 If .Site.IsEmptyString Then
-                                    MyMainLOG = $"The corresponding plugin was not found for the user [{ .Name}]. The user was removed from SCrawler."
+                                    MyMainLOG = $"The corresponding plugin was not found for the user [{ .Name}].{IIf(UsersListProtected, String.Empty, " The user has been removed from SCrawler.")}"
                                 Else
                                     indx = __plugins.FindIndex(Function(p) p.Value.ToLower = .Site.ToLower)
                                     If indx >= 0 Then
@@ -561,19 +565,17 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
                                         If .LastSeen.HasValue Then .LastSeen = Nothing : NeedUpdate = True
                                     Else
                                         .LastSeen = Now
-                                        MyMainLOG = $"The user [{ .Site}: { .Name}]  was not found. " &
+                                        MyMainLOG = $"The user [{ .Site}: { .Name}]  was not found. " & errStr(.LastSeen.Value)
-                                                    $"It will be removed from SCrawler on { .LastSeen.Value.ToStringDate(DateTimeDefaultProvider)}."
                                         NeedUpdate = True
                                     End If
                                 ElseIf userFileExists Then
                                     If .Protected Then
                                         If Not .LastSeen.HasValue Then .LastSeen = Now : NeedUpdate = True
-                                        MyMainLOG = $"The corresponding plugin was not found for the user [{ .Site}: { .Name}]. " &
+                                        MyMainLOG = $"The corresponding plugin was not found for the user [{ .Site}: { .Name}]. " & errStr(.LastSeen.Value)
-                                                    $"It will be removed from SCrawler on { .LastSeen.Value.ToStringDate(DateTimeDefaultProvider)}."
                                     Else
                                         If .LastSeen.HasValue Then .LastSeen = Nothing : NeedUpdate = True
                                     End If
-                                ElseIf If(.LastSeen, Now).AddDays(30) < Now Then
+                                ElseIf If(.LastSeen, Now).AddDays(30) < Now And Not UsersListProtected Then
                                     UsersList.RemoveAt(i)
                                     MyMainLOG = $"The user [{ .Site}: { .Name}] was not found and was removed from SCrawler."
                                     NeedUpdate = True
@@ -633,8 +635,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
                                                                                   uu.LastSeen = Now
                                                                                   NeedUpdate = True
                                                                                   UsersList(uIndex) = uu
-                                                                                  MyMainLOG = $"The user [{uu.Site}: {uu.Name}]  was not found. " &
+                                                                                  MyMainLOG = $"The user [{uu.Site}: {uu.Name}]  was not found. " & errStr(uu.LastSeen.Value)
-                                                                                              $"It will be removed from SCrawler on {uu.LastSeen.Value.ToStringDate(DateTimeDefaultProvider)}."
                                                                               End If
                                                                           End If
                                                                           Return uIndex >= 0
@@ -649,7 +650,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
                                                                        __del = False
                                                                        If Not .Item(i).FileExists Then
                                                                            With DirectCast(.Item(i), UserDataBase).User
-                                                                               If Not findWrongUser(.Self) Then
+                                                                               If Not findWrongUser(.Self) And Not UsersListProtected Then
                                                                                    __del = True
                                                                                    MyMainLOG = $"The user [{ .Site}: { .Name}] was not found and was removed from SCrawler."
                                                                                End If
@@ -875,7 +876,11 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
         End Get
     End Property
     Friend ReadOnly Property MaxUsersJobsCount As XMLValue(Of Integer)
-    Friend ReadOnly Property UserAgent As XMLValue(Of String)
+    Friend ReadOnly Property UserAgent As String
+        Get
+            Return HEADER_UserAgent
+        End Get
+    End Property
     Friend ReadOnly Property ImgurClientID As XMLValue(Of String)
 #End Region
 #Region "Basis: new version"

SCrawler/SiteResources.Designer.vb

@@ -64,6 +64,26 @@ Namespace My.Resources
             End Set
         End Property
         
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
+        '''</summary>
+        Friend Shared ReadOnly Property BlueskyIcon_32() As System.Drawing.Icon
+            Get
+                Dim obj As Object = ResourceManager.GetObject("BlueskyIcon_32", resourceCulture)
+                Return CType(obj,System.Drawing.Icon)
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Bitmap.
+        '''</summary>
+        Friend Shared ReadOnly Property BlueskyPic_32() As System.Drawing.Bitmap
+            Get
+                Dim obj As Object = ResourceManager.GetObject("BlueskyPic_32", resourceCulture)
+                Return CType(obj,System.Drawing.Bitmap)
+            End Get
+        End Property
+        
         '''<summary>
         '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
         '''</summary>
@@ -324,6 +344,16 @@ Namespace My.Resources
             End Get
         End Property
         
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
+        '''</summary>
+        Friend Shared ReadOnly Property TwitterIconNew_32() As System.Drawing.Icon
+            Get
+                Dim obj As Object = ResourceManager.GetObject("TwitterIconNew_32", resourceCulture)
+                Return CType(obj,System.Drawing.Icon)
+            End Get
+        End Property
+        
         '''<summary>
         '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
         '''</summary>

SCrawler/SiteResources.resx

@@ -118,6 +118,12 @@
     <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
   </resheader>
   <assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
+  <data name="BlueskyIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>Content\Icons\SiteIcons\BlueskyIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
+  <data name="BlueskyPic_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>Content\Pictures\SitePictures\BlueskyPic_32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
   <data name="FacebookIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>Content\Icons\SiteIcons\FacebookIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>
@@ -193,6 +199,9 @@
   <data name="TikTokPic_192" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>Content\Pictures\SitePictures\TikTokPic_192.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>
+  <data name="TwitterIconNew_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>Content\Icons\SiteIcons\TwitterIconNew_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
   <data name="TwitterIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>Content\Icons\SiteIcons\TwitterIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>