2026.8.7.0

API: updated EditorExchangeOptionsBase and its inheritances
API.Base.SiteSettingsBase: updated 'Responser' creation algorithm
API.Base.UserDataBase: updated 'DownloadContentDefault_ConvertWebp_Impl' function
API.Instagram: added 'USE_GQL_Highlights' property; updated 'DocId' to get highlights and user info; remove 'ValidateExtension'; add 'DownloadContentDefault_ConvertWebp' to convert 'heic' files; fix highlights downloading; update getting username by ID
API.Reddit: fix missing posts downloading
API.ThisVid: fix a bug with updating/adding cookies
API.Twitter: fix post text is not downloaded completely
API.Xhamster: fix some posts are not downloaded
DownloadFeedForm: improve filters; add loading buttons 'Current', 'Favorite' and 'Load'
FeedMedia: fix an issue where post text would not load in older feeds
FeedFilter: add search by labels; fix saving filters when selecting users; fix an issue with the 'TEMP' filter
SiteEditorForm: get a cookies instance from the source

Changelog.md

@@ -2,18 +2,105 @@
 - [ffmpeg](https://github.com/AAndyProgram/SCrawler/wiki/Settings#ffmpeg)
   - x64 version - [release](https://github.com/GyanD/codexffmpeg/releases/tag/5.1.2); [zip](https://github.com/GyanD/codexffmpeg/releases/download/5.1.2/ffmpeg-5.1.2-full_build.zip); **version `5.1.2-full_build-www.gyan.dev`**
   - x86 version - [release](https://github.com/yt-dlp/FFmpeg-Builds/releases/tag/autobuild-2022-11-30-12-57); [zip](https://github.com/yt-dlp/FFmpeg-Builds/releases/download/autobuild-2022-11-30-12-57/ffmpeg-N-109274-gd7a5f068c2-win32-gpl.zip); **version `N-109457-geeb280f351-20221226`**
-- [Gallery-dl](https://github.com/AAndyProgram/SCrawler/wiki/Settings#gallery-dl) - **1.31.3**
+- [Gallery-dl](https://github.com/AAndyProgram/SCrawler/wiki/Settings#gallery-dl) - **1.32.9**
-- [YT-DLP](https://github.com/AAndyProgram/SCrawler/wiki/Settings#yt-dlp) - **2025.12.08**
+- [YT-DLP](https://github.com/AAndyProgram/SCrawler/wiki/Settings#yt-dlp) - **2026.07.04**
 - [Deno](https://github.com/AAndyProgram/SCrawler/wiki/Settings#deno) - latest *(`2.0.0` or higher)*
 - [OF-Scraper](https://github.com/AAndyProgram/SCrawler/wiki/Settings#of-scraper) - **3.12.9** ([release](https://github.com/datawhores/OF-Scraper/releases/tag/3.12.9))
 
 # 2026
 
+## 2026.8.7.0
+
+*2026-08-07*
+
+- Added
+  - Feed:
+    - improve filters *(add search by labels)*
+    - add `Current`, `Favorite` and `Load` buttons to the toolbar
+  - Minor improvements
+- Updated
+  - gallery-dl up to version **1.32.9**
+  - yt-dlp up to version **2026.07.04**
+- Fixed
+  - Sites:
+    - Instagram: **highlights are not downloading**
+    - Reddit: missing posts are not downloading
+    - ThisVid: a bug with updating/adding cookies
+    - Twitter: **post text is not downloaded completely**
+    - xHamster: some posts are not downloaded
+  - Feed: the post text is not loading in old feeds
+  - Minor bugs
+
+## 2026.5.13.0
+
+*2026-05-13*
+
+- Added
+  - Sites
+    - ThisVid: download undownloaded videos
+  - Feed: the most frequently used buttons are added to the top
+  - Minor improvements
+- Updated
+  - gallery-dl up to version **1.32.0**
+- Fixed
+  - Sites:
+    - **Instagram: data is not downloading**
+    - OnlyFans: cookies error
+    - PornHub: subscriptions are not downloading
+  - Minor bugs
+
+## 2026.3.23.0
+
+*2026-03-23*
+
+- Added
+  - Sites:
+    - Bluesky:
+      - download models (user options), like in Twitter
+      - `did` handle for recognizing
+    - PornHub: `M3U8` download functionality update
+    - xHamster: add `webm` to `jpg` correct conversion
+  - Minor improvements
+- Updated
+  - gallery-dl up to version **1.31.9**
+  - yt-dlp up to version **2026.03.17**
+- Fixed
+  - Sites:
+    - TikTok: gallery-dl downloads stories slowly when the user has no stories
+    - Twitter: downloading a single media file (standalone downloader) fails
+  - Minor bugs
+
+## 2026.2.14.0
+
+*2026-02-14*
+
+- Added
+  - Sites:
+    - Twitter: get a new username based on the user ID
+  - Minor improvements
+- Updated
+  - gallery-dl up to version **1.31.6**
+  - yt-dlp up to version **2026.02.04.233607**
+- Fixed
+  - Sites:
+    - **Instagram: some profiles aren't downloading**
+    - xHamster: videos aren't downloading
+  - Minor bugs
+
+## 2026.1.24.0
+
+*2026-01-24*
+
+- Updated
+  - gallery-dl up to version **1.31.4**
+- Fixed
+  - Minor bugs
+
 ## 2026.1.17.0
 
 *2026-01-17*
 
-- Add
+- Added
   - Sites:
     - OnlyFans: handling error `502`
     - Threads: user name and description extraction

FAQ.md

@@ -33,6 +33,7 @@ I strongly recommend you to **regularly** create backup copies of the settings f
 ## General questions
 - **PROFILES**
     - I added a profile but **nothing downloaded** :arrow_forward: check your cookies and [site requirements](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements). If there are any optional fields that you don't fill in, do so. Still nothing works - [report it](#how-to-report-a-problem)!
+    - :exclamation: **Try to avoid Chinese/Japanese symbols in the paths.**
     - User downloading failed :arrow_forward: check your credentials and **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**. If all settings are set and nothing works, [report it](#how-to-report-a-problem). **Don't forget to attach the LOG.**
     - [How to redownload user](https://github.com/AAndyProgram/SCrawler/wiki#redownload-user)
     - How to **add profile** to download :arrow_forward: copy the **[profile URL](https://github.com/AAndyProgram/SCrawler/wiki#add-user)** and press `Insert` or `Ctrl+Insert`. **ALWAYS PASTE THE USER PROFILE URL**. After that select this user and press `F5` or click the `Download selected` button.

README.md

@@ -1,6 +1,5 @@
-<!--
 # 🏳️‍🌈 Happy LGBT Pride Month 🎉
--->
+
 # 🏳️‍🌈 Social networks crawler 🏳️‍🌈
 
 [![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)

SCrawler/API/Base/Declarations.vb

@@ -11,6 +11,9 @@ Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Base
     Friend Module Declarations
+        Friend Interface IThumbList
+            Property Thumbs As List(Of String)
+        End Interface
         Friend Const UserLabelName As String = "User"
         Friend Const SearchRequestLabelName As String = "Search request"
         Friend ReadOnly LNC As New ListAddParams(LAP.NotContainsOnly)

SCrawler/API/Base/EditorExchangeOptionsBase.vb

@@ -11,6 +11,10 @@ Imports DN = SCrawler.API.Base.DeclaredNames
 Namespace API.Base
     Friend Class EditorExchangeOptionsBase
         Friend Overridable Property SiteKey As String
+        <PSetting(Address:=SettingAddress.User, Caption:="Profile exists")>
+        Friend Overridable Property UserExists As Boolean = True
+        <PSetting(Address:=SettingAddress.User, Caption:="Profile suspended")>
+        Friend Overridable Property UserSuspended As Boolean = False
         <PSetting(Address:=SettingAddress.User, Caption:=DN.UserNameChangeCaption, ToolTip:=DN.UserNameChangeToolTip)>
         Friend Overridable Property UserName As String = String.Empty
         <PSetting(Address:=SettingAddress.User, Caption:=DN.DownloadTextCaption, ToolTip:=DN.DownloadTextTip)>
@@ -20,6 +24,8 @@ Namespace API.Base
         <PSetting(Address:=SettingAddress.User, Caption:=DN.DownloadTextSpecialFolderCaption, ToolTip:=DN.DownloadTextSpecialFolderTip)>
         Friend Overridable Property DownloadTextSpecialFolder As Boolean = False
         Friend Sub New(ByVal u As UserDataBase)
+            UserExists = u.UserExists
+            UserSuspended = u.UserSuspended
             UserName = u.NameTrue(True)
             DownloadText = u.DownloadText
             DownloadTextPosts = u.DownloadTextPosts
@@ -35,6 +41,8 @@ Namespace API.Base
         Protected _ApplyBase_Name As Boolean = True
         Protected _ApplyBase_Text As Boolean = True
         Friend Sub ApplyBase(ByRef u As UserDataBase)
+            u.UserExists = UserExists
+            u.UserSuspended = UserSuspended
             If _ApplyBase_Name Then u.NameTrue = UserName
             If _ApplyBase_Text Then
                 u.DownloadText = DownloadText
@@ -42,5 +50,9 @@ Namespace API.Base
                 u.DownloadTextSpecialFolder = DownloadTextSpecialFolder
             End If
         End Sub
+        Friend Overridable Sub Apply(ByRef u As UserDataBase)
+            ApplyBase(u)
+            If TypeOf u Is IPSite And TypeOf Me Is IPSite Then DirectCast(u, IPSite).QueryString = DirectCast(Me, IPSite).QueryString
+        End Sub
     End Class
 End Namespace

SCrawler/API/Base/EditorExchangeOptionsBase_P.vb

@@ -31,10 +31,6 @@ Namespace API.Base
             MyBase.New(s)
             DisableBase()
         End Sub
-        Friend Overridable Sub Apply(ByRef u As IPSite)
-            ApplyBase(u)
-            u.QueryString = QueryString
-        End Sub
         Protected Overridable Sub DisableBase()
             _ApplyBase_Name = False
             _ApplyBase_Text = False

SCrawler/API/Base/SiteSettingsBase.vb

@@ -53,7 +53,12 @@ Namespace API.Base
             End Get
         End Property
         Private Property Logger As ILogProvider = LogConnector Implements ISiteSettings.Logger
-        Friend Overridable ReadOnly Property Responser As Responser
+        Protected _Responser As Responser = Nothing
+        Friend ReadOnly Property Responser As Responser
+            Get
+                Return _Responser
+            End Get
+        End Property
         Private _UserOptionsExists As Boolean = False
         Private _UserOptionsType As Type = Nothing
         Protected Overridable Function UserOptionsValid(ByVal Options As Object) As Boolean
@@ -122,6 +127,11 @@ Namespace API.Base
             _CookiesNetscapeFile.Name &= "_Cookies_Netscape"
             _CookiesNetscapeFile.Extension = "txt"
         End Sub
+        Protected Overridable Sub ChangeResponserOnInit()
+        End Sub
+        Friend Overridable Function GetCookieKeeperInstance() As CookieKeeper
+            Return New CookieKeeper
+        End Function
 #End Region
 #Region "GetInstance"
         Friend MustOverride Function GetInstance(ByVal What As Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
@@ -131,7 +141,8 @@ Namespace API.Base
             Site = SiteName
             _Icon = __Icon
             _Image = __Image
-            Responser = New Responser With {.DeclaredError = EDP.ThrowException}
+            _Responser = New Responser With {.DeclaredError = EDP.ThrowException}
+            ChangeResponserOnInit()
             SettingsVersion = New PropertyValue(0)
             DownloadText = New PropertyValue(False)
             DownloadTextPosts = New PropertyValue(False)

SCrawler/API/Base/TokenBatch.vb

@@ -13,12 +13,17 @@ Namespace API.Base
         Friend Property TempPostsList As List(Of String)
         Protected ReadOnly Token As CancellationToken
         Friend Property DebugMode As Boolean = False
+        Friend Overridable Property MyWorkingDirectory As SFile = Nothing
         Friend Sub New(ByVal _Token As CancellationToken, Optional ByVal _MainProcessName As String = Nothing, Optional ByVal WorkingDir As SFile = Nothing)
             MyBase.New(True)
             Token = _Token
             MainProcessName = _MainProcessName
             If Not WorkingDir.IsEmptyString Then ChangeDirectory(WorkingDir)
         End Sub
+        Public Overrides Sub ChangeDirectory(ByVal Directory As SFile)
+            MyBase.ChangeDirectory(Directory)
+            If Not Directory.IsEmptyString Then MyWorkingDirectory = Directory
+        End Sub
         Protected Overrides Function Internal_Execute(ByVal Commands As IEnumerable(Of String), ByVal e As ErrorsDescriber) As Boolean
             If Not Encoding.HasValue Then Encoding = UnicodeEncoding
             Return MyBase.Internal_Execute(Commands, e)

SCrawler/API/Base/UserDataBase.vb

@@ -360,8 +360,8 @@ Namespace API.Base
                 Me._UserSiteName = _UserSiteName
             End Set
         End Property
-        Protected Sub UserSiteNameUpdate(ByVal NewName As String)
+        Protected Sub UserSiteNameUpdate(ByVal NewName As String, Optional ByVal Force As Boolean = False)
-            If Not NewName.IsEmptyString And (UserSiteName.IsEmptyString Or Settings.UpdateUserSiteNameEveryTime) Then UserSiteName = NewName
+            If Not NewName.IsEmptyString And (UserSiteName.IsEmptyString Or Settings.UpdateUserSiteNameEveryTime Or Force) Then UserSiteName = NewName
         End Sub
         Friend ReadOnly Property UserModel As UsageModel Implements IUserData.UserModel
             Get
@@ -1030,7 +1030,7 @@ BlockNullPicture:
             End Try
         End Sub
         Private Sub UpdateUserInformation_Ex()
-            If _ForceSaveUserInfoOnException Then UpdateUserInformation()
+            If _ForceSaveUserInfo Then UpdateUserInformation()
         End Sub
         Friend Overridable Overloads Sub UpdateUserInformation() Implements IUserData.UpdateUserInformation
             UpdateUserInformation(False)
@@ -1207,7 +1207,6 @@ BlockNullPicture:
         Protected UseClientTokens As Boolean = False
         Protected _ForceSaveUserData As Boolean = False
         Protected _ForceSaveUserInfo As Boolean = False
-        Protected _ForceSaveUserInfoOnException As Boolean = False
         Private _DownloadInProgress As Boolean = False
         Private _EnvirUserExists As Boolean
         Private _EnvirUserSuspended As Boolean
@@ -1227,7 +1226,6 @@ BlockNullPicture:
             _DescriptionEveryTime = Settings.UpdateUserDescriptionEveryTime
             _ForceSaveUserData = False
             _ForceSaveUserInfo = False
-            _ForceSaveUserInfoOnException = False
             _EnvirUserExists = UserExists
             _EnvirUserSuspended = UserSuspended
             _EnvirCreatedByChannel = CreatedByChannel
@@ -1244,8 +1242,8 @@ BlockNullPicture:
                 Select Case Caller
                     Case NameOf(UserExists) : If Not _EnvirUserExists = CBool(NewValue) Then _EnvirChanged = True : _EnvirInvokeUserUpdated = True
                     Case NameOf(UserSuspended) : If Not _EnvirUserSuspended = CBool(NewValue) Then _EnvirChanged = True : _EnvirInvokeUserUpdated = True
-                    Case NameOf(NameTrue) : _EnvirChanged = True : _ForceSaveUserInfo = True : _ForceSaveUserInfoOnException = True
+                    Case NameOf(NameTrue) : _EnvirChanged = True : _ForceSaveUserInfo = True
-                    Case NameOf(ID) : _EnvirChanged = True : _ForceSaveUserInfo = True : _ForceSaveUserInfoOnException = True
+                    Case NameOf(ID) : _EnvirChanged = True : _ForceSaveUserInfo = True
                     Case Else : _EnvirChanged = True
                 End Select
             End If
@@ -1552,6 +1550,12 @@ BlockNullPicture:
         ''' </summary>
         Protected Overridable Sub ReparseMissing(ByVal Token As CancellationToken)
         End Sub
+        Protected Sub ReparseMissing_ClearList(ByRef rList As List(Of Integer))
+            If rList.Count > 0 Then
+                For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
+                rList.Clear()
+            End If
+        End Sub
 #End Region
 #Region "MD5 support"
         Private Const VALIDATE_MD5_ERROR As String = "VALIDATE_MD5_ERROR"
@@ -1843,7 +1847,8 @@ BlockNullPicture:
 
                                         updateDownCount(False)
 
-                                        v.File = DownloadContentDefault_ConvertWebp(ChangeFileNameByProvider(f, v), postProcessWebp)
+                                        v.File = ChangeFileNameByProvider(f, v)
+                                        v.File = DownloadContentDefault_ConvertWebp(v, postProcessWebp)
                                         v.State = UStates.Downloaded
                                         DownloadContentDefault_PostProcessing(v, f, Token)
                                         If UseMD5Comparison And (v.Type = UTypes.GIF Or v.Type = UTypes.Picture) Then
@@ -1943,7 +1948,11 @@ stxt:
         End Function
         Protected Overridable Sub DownloadContentDefault_PostProcessing(ByRef m As UserMedia, ByVal File As SFile, ByVal Token As CancellationToken)
         End Sub
-        Protected Overridable Function DownloadContentDefault_ConvertWebp(ByVal WebpFile As SFile, ByVal Process As Boolean) As SFile
+        Protected Overridable Function DownloadContentDefault_ConvertWebp(ByVal m As UserMedia, ByVal Process As Boolean) As SFile
+            Return DownloadContentDefault_ConvertWebp_Impl(m, Process)
+        End Function
+        Protected Overridable Function DownloadContentDefault_ConvertWebp_Impl(ByVal m As UserMedia, ByVal Process As Boolean, Optional ByVal ConvertWebpForce As Boolean = False) As SFile
+            Dim WebpFile As SFile = m.File
             Dim f As SFile = WebpFile
             If Process AndAlso f.Exists Then
                 f.Path = $"{f.PathWithSeparator}Sources"
@@ -1951,7 +1960,7 @@ stxt:
                 If WebpFile.Copy(f) Then
                     Dim newFile As SFile = WebpFile
                     newFile.Extension = UserImage.ExtJpg
-                    f = UserImage.ConvertWebp(f, newFile)
+                    f = UserImage.ConvertWebp(f, newFile, ConvertWebpForce)
                     If f.Exists Then WebpFile.Delete(SFO.File, SFODelete.DeletePermanently, EDP.ReturnValue)
                 Else
                     f = WebpFile
@@ -1959,6 +1968,24 @@ stxt:
             End If
             Return f
         End Function
+        Protected Function DownloadContentDefault_ConvertWebp_TestImg(ByVal m As UserMedia, ByVal Process As Boolean) As SFile
+            If m.Type = UTypes.Picture And Settings.DownloadNativeImageFormat Then
+                Using testImg As New UserImage.ImageRenderer2(m.File, EDP.ReturnValue)
+                    If testImg.IsWebP Then
+                        Dim f As SFile = m.File
+                        If f.Extension.IsEmptyString OrElse Not f.Extension = UserImage.ExtWebp Then
+                            f.Extension = UserImage.ExtWebp
+                            f = SFile.Rename(m.File, f,, EDP.ReturnValue).IfNullOrEmpty(m.File)
+                        End If
+                        m.File = f
+                        Return DownloadContentDefault_ConvertWebp_Impl(m, True).IfNullOrEmpty(f)
+                    End If
+                End Using
+            ElseIf Process Then
+                Return DownloadContentDefault_ConvertWebp_Impl(m, Process)
+            End If
+            Return m.File
+        End Function
         Protected Overridable Function DownloadContentDefault_ProcessDownloadException() As Boolean
             Return True
         End Function

SCrawler/API/Bluesky/EditorExchangeOptions.vb

@@ -0,0 +1,46 @@
+' Copyright (C) Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
+Imports SCrawler.Plugin.Attributes
+Namespace API.Bluesky
+    Friend Class EditorExchangeOptions : Inherits Base.EditorExchangeOptionsBase
+        Friend Overrides Property SiteKey As String = BlueskySiteKey
+        <PSetting(NameOf(SiteSettings.DownloadModelMedia), NameOf(MySettings), Address:=SettingAddress.User)>
+        Friend Overridable Property DownloadModelMedia As Boolean = False
+        <PSetting(NameOf(SiteSettings.DownloadModelProfile), NameOf(MySettings), Address:=SettingAddress.User)>
+        Friend Overridable Property DownloadModelProfile As Boolean = False
+        Private ReadOnly Property MySettings As Object
+        Friend Sub New(ByVal s As SiteSettings)
+            MyBase.New(s)
+            DownloadModelMedia = s.DownloadModelMedia.Value
+            DownloadModelProfile = s.DownloadModelProfile.Value
+            MySettings = s
+        End Sub
+        Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
+            DownloadModelMedia = u.DownloadModelMedia
+            DownloadModelProfile = u.DownloadModelProfile
+            MySettings = u.HOST.Source
+        End Sub
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
+            MyBase.Apply(u)
+            If Not DownloadModelMedia And Not DownloadModelProfile Then
+                DownloadModelMedia = True
+            ElseIf DownloadModelMedia Then
+                DownloadModelProfile = False
+            Else
+                DownloadModelMedia = False
+            End If
+            With DirectCast(u, UserData)
+                .DownloadModelMedia = DownloadModelMedia
+                .DownloadModelProfile = DownloadModelProfile
+            End With
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Bluesky/SiteSettings.vb

@@ -26,6 +26,10 @@ Namespace API.Bluesky
         <PXML> Friend ReadOnly Property TokenUpdateTime As PropertyValue
         <PropertyOption(ControlText:="Token update", ControlToolTip:="Token refresh interval (in minutes)." & vbCr & "Default: 120.", IsAuth:=True), PXML, PClonable, HiddenControl>
         Friend ReadOnly Property TokenRefreshInterval As PropertyValue
+        <PropertyOption(ControlText:="Download model 'Media'", ControlToolTip:="Parse the 'Media' block", Category:=DeclaredNames.CAT_UserDefs), PXML, PClonable>
+        Friend ReadOnly Property DownloadModelMedia As PropertyValue
+        <PropertyOption(ControlText:="Download model 'Profile'", ControlToolTip:="Parse the 'Posts' block", Category:=DeclaredNames.CAT_UserDefs), PXML, PClonable>
+        Friend ReadOnly Property DownloadModelProfile As PropertyValue
         Friend Sub New(ByVal AccName As String, ByVal Temp As Boolean)
             MyBase.New("Bluesky", "bsky.app", AccName, Temp, My.Resources.SiteResources.BlueskyIcon_32, My.Resources.SiteResources.BlueskyPic_32)
 
@@ -38,11 +42,14 @@ Namespace API.Bluesky
             TokenUpdateTime = New PropertyValue(Now.AddYears(-1))
             TokenRefreshInterval = New PropertyValue(120)
 
+            DownloadModelMedia = New PropertyValue(True)
+            DownloadModelProfile = New PropertyValue(False)
+
             _AllowUserAgentUpdate = False
             UrlPatternUser = "https://bsky.app/profile/{0}"
             ImageVideoContains = "bsky.app"
             UserRegex = RParams.DMS("bsky.app/profile/([^/\?]+)", 1, EDP.ReturnValue)
-            UserOptionsType = GetType(EditorExchangeOptionsBase)
+            UserOptionsType = GetType(EditorExchangeOptions)
         End Sub
         Protected Overrides Function UserOptionsValid(ByVal Options As Object) As Boolean
             Return DirectCast(Options, EditorExchangeOptionsBase).SiteKey = BlueskySiteKey
@@ -96,5 +103,10 @@ Namespace API.Bluesky
                 _TokenUpdating = False
             End Try
         End Function
+        Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
+            Dim e As ExchangeOptions = MyBase.IsMyUser(UserURL)
+            If Not e.UserName.IsEmptyString AndAlso e.UserName.StartsWith("did:") Then e.UserName = e.UserName.Replace(":", "@")
+            Return e
+        End Function
     End Class
 End Namespace

SCrawler/API/Bluesky/UserData.vb

@@ -16,6 +16,11 @@ Imports UTypes = SCrawler.API.Base.UserMedia.Types
 Imports UStates = SCrawler.API.Base.UserMedia.States
 Namespace API.Bluesky
     Friend Class UserData : Inherits UserDataBase
+#Region "XML names"
+        Private Const Name_DownloadModelMedia As String = "DownloadModelMedia"
+        Private Const Name_DownloadModelProfile As String = "DownloadModelProfile"
+        Private Const Name_ForceParseProfileInfo As String = "ForceParseProfileInfo"
+#End Region
 #Region "Declarations"
         Private ReadOnly Property MySettings As SiteSettings
             Get
@@ -28,16 +33,45 @@ Namespace API.Bluesky
             End Get
         End Property
         Private ReadOnly _TmpPosts2 As List(Of String)
+        Friend Property DownloadModelMedia As Boolean = True
+        Friend Property DownloadModelProfile As Boolean = False
+        Private Property ForceParseProfileInfo As Boolean = False
 #End Region
 #Region "Loader"
+        Private Sub UpdateUserOptions()
+            If ID.IsEmptyString AndAlso Not Name.IsEmptyString AndAlso Name.StartsWith("did@") Then
+                NameTrue = Name.Replace("@", ":")
+                ID = NameTrue
+                ForceParseProfileInfo = True
+            End If
+        End Sub
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+            With Container
+                If Loading Then
+                    If .Contains(Name_DownloadModelMedia) Then
+                        DownloadModelMedia = .Value(Name_DownloadModelMedia).FromXML(Of Boolean)(True)
+                        DownloadModelProfile = .Value(Name_DownloadModelProfile).FromXML(Of Boolean)(False)
+                    Else
+                        DownloadModelMedia = ParseUserMediaOnly
+                        DownloadModelProfile = Not ParseUserMediaOnly
+                    End If
+                    ForceParseProfileInfo = .Value(Name_ForceParseProfileInfo).FromXML(Of Boolean)(False)
+                Else
+                    If ID.IsEmptyString Then
+                        UpdateUserOptions()
+                        .Value(Name_UserID) = ID
+                    End If
+                    .Add(Name_DownloadModelMedia, DownloadModelMedia.BoolToInteger)
+                    .Add(Name_DownloadModelProfile, DownloadModelProfile.BoolToInteger)
+                    .Add(Name_ForceParseProfileInfo, ForceParseProfileInfo.BoolToInteger)
+                End If
+            End With
         End Sub
         Friend Overrides Function ExchangeOptionsGet() As Object
-            Return New EditorExchangeOptionsBase(Me) With {.SiteKey = BlueskySiteKey}
+            Return New EditorExchangeOptions(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
-            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptionsBase AndAlso
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then DirectCast(Obj, EditorExchangeOptions).Apply(Me)
-               DirectCast(Obj, EditorExchangeOptionsBase).SiteKey = BlueskySiteKey Then DirectCast(Obj, EditorExchangeOptionsBase).ApplyBase(Me)
         End Sub
 #End Region
 #Region "Initializer"
@@ -66,6 +100,13 @@ Namespace API.Bluesky
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             _TmpPosts2.Clear()
             Try
+                If Not DownloadModelMedia And Not DownloadModelProfile Then
+                    DownloadModelMedia = True
+                ElseIf DownloadModelMedia Then
+                    DownloadModelProfile = False
+                Else
+                    DownloadModelMedia = False
+                End If
                 If Not CBool(MySettings.CookiesEnabled.Value) Then Responser.Cookies.Clear()
                 UpdateToken(, True)
                 _TokenUpdateCount = 0
@@ -79,7 +120,7 @@ Namespace API.Bluesky
         Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
             Try
-                If Not IsSavedPosts And ID.IsEmptyString Then GetProfileInfo(Token)
+                If (Not IsSavedPosts And ID.IsEmptyString) Or ForceParseProfileInfo Then GetProfileInfo(Token)
                 If Not IsSavedPosts And ID.IsEmptyString Then Throw New ArgumentNullException("ID", "ID is null")
                 If UpdateToken() Then
                     Dim nextCursor$ = String.Empty
@@ -91,7 +132,9 @@ Namespace API.Bluesky
                         n = {"bookmarks"}
                         p = {"item"}
                     Else
-                        URL = $"https://bsky.social/xrpc/app.bsky.feed.getAuthorFeed?actor={ID_Encoded}&filter=posts_and_author_threads&includePins=false&limit=99"
+                        'posts_and_author_threads
+                        'posts_with_media
+                        URL = $"https://bsky.social/xrpc/app.bsky.feed.getAuthorFeed?actor={ID_Encoded}&filter={IIf(DownloadModelMedia, "posts_with_media", "posts_and_author_threads")}&includePins=false&limit=99"
                         If Not Cursor.IsEmptyString Then URL &= $"&cursor={SymbolsConverter.ASCII.EncodeSymbolsOnly(Cursor)}"
                         n = {"feed"}
                         p = {"post"}
@@ -106,7 +149,7 @@ Namespace API.Bluesky
                                     If .ListExists Then
                                         For Each post As EContainer In .Self
                                             With post(p)
-                                                c = DefaultParser(.Self,, nextCursor)
+                                                c = DefaultParser(.Self) ',, nextCursor)
                                                 Select Case c
                                                     Case CInt(DateResult.Skip) * -1 : Continue For
                                                     Case CInt(DateResult.Exit) * -1 : Exit Sub
@@ -238,6 +281,7 @@ Namespace API.Bluesky
 #Region "GetProfileInfo"
         Private Sub GetProfileInfo(ByVal Token As CancellationToken)
             Try
+                If ForceParseProfileInfo Then ForceParseProfileInfo = False : _ForceSaveUserInfo = True
                 If UpdateToken() Then
                     Dim r$ = Responser.GetResponse($"https://bsky.social/xrpc/app.bsky.actor.getProfile?actor={ID.IfNullOrEmpty(NameTrue)}")
                     TokenUpdateCountReset()
@@ -344,6 +388,9 @@ Namespace API.Bluesky
         Protected Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile, ByVal Token As CancellationToken) As SFile
             Return M3U8.Download(URL, DestinationFile, Token, Progress, Not IsSingleObjectDownload)
         End Function
+        Protected Overrides Function DownloadContentDefault_ConvertWebp(ByVal m As UserMedia, ByVal Process As Boolean) As SFile
+            Return DownloadContentDefault_ConvertWebp_TestImg(m, Process)
+        End Function
 #End Region
 #Region "DownloadSingleObject"
         Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)

SCrawler/API/Facebook/UserData.vb

@@ -68,14 +68,7 @@ Namespace API.Facebook
             Return New UserExchangeOptions(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
-            If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then DirectCast(Obj, UserExchangeOptions).Apply(Me)
-                With DirectCast(Obj, UserExchangeOptions)
-                    ParsePhotoBlock = .ParsePhotoBlock
-                    ParseVideoBlock = .ParseVideoBlock
-                    ParseReelsBlock = .ParseReelsBlock
-                    ParseStoriesBlock = .ParseStoriesBlock
-                End With
-            End If
         End Sub
 #End Region
 #Region "Loader"

SCrawler/API/Facebook/UserExchangeOptions.vb

@@ -6,9 +6,10 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
 Imports SCrawler.Plugin.Attributes
 Namespace API.Facebook
-    Friend Class UserExchangeOptions
+    Friend Class UserExchangeOptions : Inherits EditorExchangeOptionsBase
         <PSetting(NameOf(SiteSettings.ParsePhotoBlock), NameOf(MySettings))>
         Friend Property ParsePhotoBlock As Boolean = True
         <PSetting(NameOf(SiteSettings.ParseVideoBlock), NameOf(MySettings))>
@@ -19,6 +20,7 @@ Namespace API.Facebook
         Friend Property ParseStoriesBlock As Boolean = True
         Private ReadOnly Property MySettings As SiteSettings
         Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
             MySettings = u.HostCollection.Default.Source
             ParsePhotoBlock = u.ParsePhotoBlock
             ParseVideoBlock = u.ParseVideoBlock
@@ -26,11 +28,21 @@ Namespace API.Facebook
             ParseStoriesBlock = u.ParseStoriesBlock
         End Sub
         Friend Sub New(ByVal s As SiteSettings)
+            MyBase.New(s)
             MySettings = s
             ParsePhotoBlock = s.ParsePhotoBlock.Value
             ParseVideoBlock = s.ParseVideoBlock.Value
             ParseReelsBlock = s.ParseReelsBlock.Value
             ParseStoriesBlock = s.ParseStoriesBlock.Value
         End Sub
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
+            MyBase.Apply(u)
+            With DirectCast(u, UserData)
+                .ParsePhotoBlock = ParsePhotoBlock
+                .ParseVideoBlock = ParseVideoBlock
+                .ParseReelsBlock = ParseReelsBlock
+                .ParseStoriesBlock = ParseStoriesBlock
+            End With
+        End Sub
     End Class
 End Namespace

SCrawler/API/Instagram/Declarations.vb

@@ -20,6 +20,8 @@ Namespace API.Instagram
         Friend Const PageTokenRegexPatternDefault As String = "\[\],{""token"":""(.*?)""},\d+\]"
         Friend ReadOnly Regex_UserToken_dtsg As RParams = RParams.DMS("DTSGInitialData["":,.\[\]]*?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
         Friend ReadOnly Regex_UserToken_lsd As RParams = RParams.DMS("LSD["":,.\[\]]*?{\s*.token.:\s*""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_BlockVersionID As RParams = RParams.DMS("""versioningID"":""([^""]+)""", 1, EDP.ReturnValue)
+        Friend ReadOnly Regex_ProfileID As RParams = RParams.DMS("profilePage_(\d+)", 1, EDP.ReturnValue)
         Friend Sub UpdateResponser(ByVal Source As IResponse, ByRef Destination As Responser, ByVal UpdateWwwClaim As Boolean)
             Const r_wwwClaimName$ = "x-ig-set-www-claim"
             Const r_tokenName$ = SiteSettings.Header_CSRF_TOKEN_COOKIE

SCrawler/API/Instagram/EditorExchangeOptions.vb

@@ -36,6 +36,8 @@ Namespace API.Instagram
         <PSetting(Caption:="Place the extracted image into the video folder")>
         Friend Property PutImageVideoFolder As Boolean
         Friend Overrides Property UserName As String
+        <PSetting(Address:=SettingAddress.User, Caption:="Verified profile", ToolTip:="This profile has a verified mark")>
+        Friend Property IsVerifiedProfile As Boolean = False
         <PSetting(Address:=SettingAddress.User, Caption:="Force update UserName", ToolTip:="Try to force update UserName if it is not found on the site")>
         Friend Property ForceUpdateUserName As Boolean = False
         <PSetting(Address:=SettingAddress.User, Caption:="Force update user information")>
@@ -57,6 +59,8 @@ Namespace API.Instagram
 
                 PutImageVideoFolder = .PutImageVideoFolder
 
+                IsVerifiedProfile = .IsVerifiedProfile
+
                 ForceUpdateUserName = .ForceUpdateUserName
                 ForceUpdateUserInfo = .ForceUpdateUserInfo
             End With

SCrawler/API/Instagram/SiteSettings.vb

@@ -70,7 +70,7 @@ Namespace API.Instagram
         Friend Const Header_Browser As String = "Sec-Ch-Ua"
         Friend Const Header_BrowserExt As String = "Sec-Ch-Ua-Full-Version-List"
         Friend Const Header_Platform_Verion As String = "Sec-Ch-Ua-Platform-Version"
-        <PropertyOption(ControlText:="x-csrftoken", ControlToolTip:="Can be automatically extracted from cookies", IsAuth:=True, AllowNull:=True), PClonable(Clone:=False)>
+        <PropertyOption(ControlText:="x-csrftoken", ControlToolTip:="Can be automatically extracted from cookies", IsAuth:=True, AllowNull:=False), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_CSRF_TOKEN As PropertyValue
         <CookieValueExtractor(NameOf(HH_CSRF_TOKEN))>
         Private Function GetValueFromCookies(ByVal PropName As String, ByVal c As CookieKeeper) As String
@@ -78,7 +78,7 @@ Namespace API.Instagram
         End Function
         <PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_IG_APP_ID As PropertyValue
-        <PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=True), PClonable(Clone:=False)>
+        <PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=False), PClonable(Clone:=False)>
         Friend ReadOnly Property HH_ASBD_ID As PropertyValue
         'PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=True)
         <PClonable(Clone:=False)>
@@ -89,16 +89,16 @@ Namespace API.Instagram
                 Return Not v.IsEmptyString AndAlso v = "0"
             End Get
         End Property
-        <PropertyOption(ControlText:="sec-ch-ua", IsAuth:=True, AllowNull:=True,
+        <PropertyOption(ControlText:="sec-ch-ua", IsAuth:=True, AllowNull:=False,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_BROWSER As PropertyValue
-        <PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:="sec-ch-ua-full-version-list", IsAuth:=True, AllowNull:=True,
+        <PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:="sec-ch-ua-full-version-list", IsAuth:=True, AllowNull:=False,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua_full_version_list), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_BROWSER_EXT As PropertyValue
-        <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, AllowNull:=True, LeftOffset:=135,
+        <PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, AllowNull:=False, LeftOffset:=135,
                         InheritanceName:=SettingsCLS.HEADER_DEF_sec_ch_ua_platform_version), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_PLATFORM As PropertyValue
-        <PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=True,
+        <PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=False,
                         InheritanceName:=SettingsCLS.HEADER_DEF_UserAgent), PClonable, PXML(OnlyForChecked:=True)>
         Private ReadOnly Property HH_USER_AGENT As PropertyValue
         Friend Overrides Function BaseAuthExists() As Boolean
@@ -146,8 +146,12 @@ Namespace API.Instagram
         <Provider(NameOf(HH_IG_WWW_CLAIM_UPDATE_INTERVAL), FieldsChecker:=True)>
         Private ReadOnly Property TokenUpdateIntervalProvider As IFormatProvider
 #End Region
-        <PropertyOption(ControlText:="Use GraphQL to download", IsAuth:=True), PXML, PClonable>
+        <PropertyOption(ControlText:="Use GraphQL to download", ControlToolTip:="This feature is in test mode", IsAuth:=True), PXML, PClonable>
         Friend ReadOnly Property USE_GQL As PropertyValue
+        <PropertyOption(ControlText:="Use GraphQL to download user data", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property USE_GQL_UserData As PropertyValue
+        <PropertyOption(ControlText:="Use GraphQL to download user stories (highlights)", IsAuth:=True), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property USE_GQL_Highlights As PropertyValue
 #End Region
 #Region "Download data"
         <PropertyOption(ControlText:="Download timeline", Category:=CAT_DOWN), PXML, PClonable>
@@ -165,6 +169,14 @@ Namespace API.Instagram
         <PropertyOption(ControlText:="Download tagged posts", Category:=CAT_DOWN), PXML, PClonable>
         Friend ReadOnly Property DownloadTagged As PropertyValue
         <PXML> Private ReadOnly Property DownloadTagged_Def As PropertyValue
+        <PropertyOption(ControlText:="Number of posts (verified)", ControlToolTip:="The number of posts received per request if the profile has a verified mark", Category:=CAT_DOWN), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property PostNumberVerified As PropertyValue
+        <Provider(NameOf(PostNumberVerified), FieldsChecker:=True)>
+        Private ReadOnly Property PostNumberVerifiedProvider As IFormatProvider
+        <PropertyOption(ControlText:="Number of posts (unverified)", ControlToolTip:="The number of posts received per request if the profile doesn't have a verified mark", Category:=CAT_DOWN), PXML, PClonable, HiddenControl>
+        Friend ReadOnly Property PostNumberVerifiedNot As PropertyValue
+        <Provider(NameOf(PostNumberVerifiedNot), FieldsChecker:=True)>
+        Private ReadOnly Property PostNumberVerifiedNotProvider As IFormatProvider
 #End Region
 #Region "Timers"
         Friend Const TimersUrgentTip As String = vbCr & "It is highly recommended not to change the default value."
@@ -485,6 +497,8 @@ Namespace API.Instagram
             HH_IG_WWW_CLAIM_USE_DEFAULT_ALGO = New PropertyValue(True)
             TokenUpdateIntervalProvider = New TokenRefreshIntervalProvider
             USE_GQL = New PropertyValue(False)
+            USE_GQL_UserData = New PropertyValue(True)
+            USE_GQL_Highlights = New PropertyValue(True)
 
             DownloadTimeline = New PropertyValue(True)
             DownloadTimeline_Def = New PropertyValue(DownloadTimeline.Value, GetType(Boolean))
@@ -496,6 +510,10 @@ Namespace API.Instagram
             DownloadStoriesUser_Def = New PropertyValue(DownloadStoriesUser.Value, GetType(Boolean))
             DownloadTagged = New PropertyValue(False)
             DownloadTagged_Def = New PropertyValue(DownloadTagged.Value, GetType(Boolean))
+            PostNumberVerified = New PropertyValue(50)
+            PostNumberVerifiedProvider = New TimersChecker(12)
+            PostNumberVerifiedNot = New PropertyValue(12)
+            PostNumberVerifiedNotProvider = New TimersChecker(12)
 
             RequestsWaitTimer_Any = New PropertyValue(1000)
             RequestsWaitTimer_AnyProvider = New TimersChecker(0)
@@ -545,18 +563,17 @@ Namespace API.Instagram
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "instagram.com/"), 1)
             ImageVideoContains = "instagram.com"
         End Sub
-        Private Const SettingsVersionCurrent As Integer = 2
+        Private Const SettingsVersionCurrent As Integer = 3
         Friend Overrides Sub EndInit()
             Try : MyLastRequests.Add(LastDownloadDate.Value, LastRequestsCount.Value) : Catch : End Try
             If Not CBool(HH_IG_WWW_CLAIM_USE.Value) Then Responser.Headers.Remove(Header_IG_WWW_CLAIM)
             If CInt(SettingsVersion.Value) < SettingsVersionCurrent Then
                 SettingsVersion.Value = SettingsVersionCurrent
-                HH_IG_WWW_CLAIM_UPDATE_INTERVAL.Value = 120
+                HH_IG_WWW_CLAIM_RESET_EACH_TARGET.Value = False
-                HH_IG_WWW_CLAIM_ALWAYS_ZERO.Value = False
+                RequestsWaitTimer_Any.Value = 5000
-                HH_IG_WWW_CLAIM_RESET_EACH_SESSION.Value = True
+                TaggedNotifyLimit.Value = 50
-                HH_IG_WWW_CLAIM_RESET_EACH_TARGET.Value = True
+                DownDetectorValue.Value = 30
-                HH_IG_WWW_CLAIM_USE.Value = True
+                DownDetectorValueAddToLog.Value = True
-                HH_IG_WWW_CLAIM_USE_DEFAULT_ALGO.Value = True
             End If
             MyBase.EndInit()
         End Sub

SCrawler/API/Instagram/UserData.GQL.vb

@@ -6,12 +6,13 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Security.Cryptography
 Imports System.Threading
-Imports SCrawler.API.Base
-Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports SCrawler.API.Base
 Namespace API.Instagram
     Partial Friend Class UserData
 #Region "Tokens"
@@ -26,26 +27,30 @@ Namespace API.Instagram
             Token_dtsg = String.Empty
             Token_lsd = String.Empty
         End Sub
+        Protected Property BlockVersionID As String = String.Empty
 #End Region
 #Region "Headers"
         Friend Const GQL_HEADER_FB_FRINDLY_NAME As String = "x-fb-friendly-name"
         Friend Const GQL_HEADER_FB_LSD As String = "x-fb-lsd"
+        Friend Const GQL_HEADER_BLOCK_VERSION_ID As String = "x-bloks-version-id"
+        Friend Const GQL_HEADER_ROOT_FIELD_NAME As String = "x-root-field-name"
+        Friend Const GQL_HEADER_ROOT_FIELD_NAME_Value As String = "fetch__XDTUserDict"
 #End Region
 #Region "Data constants"
-        Private Const GQL_UserData_DocId As String = "7381344031985950"
+        Private Const GQL_UserData_DocId As String = "26672929172408668" '"35710877621861450" '"7381344031985950"
         Private Const GQL_UserData_FbFriendlyName As String = "PolarisProfilePageContentQuery"
 
-        Private Const GQL_Highlights_DocId As String = "8298007123561120"
+        Private Const GQL_Highlights_DocId As String = "36997000523232338" '"8298007123561120"
-        Private Const GQL_Highlights_DocId_Second As String = "7559771384111300"
+        Private Const GQL_Highlights_DocId_Second As String = "27618287167780519" '"7559771384111300"
         Private Const GQL_Highlights_FbFriendlyName As String = "PolarisProfileStoryHighlightsTrayContentQuery"
         Private Const GQL_Highlights_FbFriendlyName_Second As String = "PolarisStoriesV3HighlightsPageQuery"
 
         Private Const GQL_UserStories_DocId As String = "25231722019806941"
         Private Const GQL_UserStories_FbFriendlyName As String = "PolarisStoriesV3ReelPageStandaloneQuery"
 
-        Private Const GQL_Timeline_DocId As String = "7268577773270422"
+        Private Const GQL_Timeline_DocId As String = "7268577773270422" '"34579740524958711" '"7268577773270422"
         Private Const GQL_Timeline_FbFriendlyName As String = "PolarisProfilePostsQuery"
-        Private Const GQL_Timeline_DocId_Second As String = "7286316061475375"
+        Private Const GQL_Timeline_DocId_Second As String = "7286316061475375" '"33944389991841132" '"7286316061475375"
         Private Const GQL_Timeline_FbFriendlyName_Second As String = "PolarisProfilePostsTabContentQuery_connection"
 
         Private Const GQL_Reels_DocId As String = "7191572580905225"
@@ -60,52 +65,80 @@ Namespace API.Instagram
         Private Const GQL_URL_Q As String = "https://www.instagram.com/graphql/query"
 #End Region
 #Region "Download functions"
-        Protected Sub UpdateHeadersGQL(ByVal HeaderValue As String)
+        Protected Sub UpdateHeadersGQL(ByVal HeaderValue As String, Optional ByVal Add As Boolean = True)
-            Responser.Headers.Add(GQL_HEADER_FB_FRINDLY_NAME, HeaderValue)
+            With Responser.Headers
-            Responser.Headers.Add(GQL_HEADER_FB_LSD, Token_lsd)
+                If Add Then
+                    .Add(GQL_HEADER_FB_FRINDLY_NAME, HeaderValue)
+                    .Add(GQL_HEADER_FB_LSD, Token_lsd)
+                    '.Add(GQL_HEADER_BLOCK_VERSION_ID, BlockVersionID)
+                    '.Add(GQL_HEADER_ROOT_FIELD_NAME, GQL_HEADER_ROOT_FIELD_NAME_Value)
+                Else
+                    .Remove(GQL_HEADER_FB_FRINDLY_NAME)
+                    .Remove(GQL_HEADER_FB_LSD)
+                    '.Remove(GQL_HEADER_BLOCK_VERSION_ID)
+                    '.Remove(GQL_HEADER_ROOT_FIELD_NAME)
+                End If
+            End With
         End Sub
-        <Obsolete("Use 'GET' function: 'GetUserData'", False)>
+        '<Obsolete("Use 'GET' function: 'GetUserData'", False)>
-        Private Sub GetUserDataGQL(ByVal Token As CancellationToken)
+        Private Function GetUserDataGQL(ByVal Token As CancellationToken) As String
+            UpdateTokens(Not ValidateBaseTokens())
+            'Dim vars$ = String.Format(GQL_URL_PATTERN_VARS, GQL_UserData_DocId, Token_lsd, Token_dtsg_Var, GQL_UserData_FbFriendlyName,
+            '                          SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""id"":""{ID}"",""relay_header"":false,""render_surface"":""PROFILE""" & "}"))
             Dim vars$ = String.Format(GQL_URL_PATTERN_VARS, GQL_UserData_DocId, Token_lsd, Token_dtsg_Var, GQL_UserData_FbFriendlyName,
-                                      SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""id"":""{ID}"",""relay_header"":false,""render_surface"":""PROFILE""" & "}"))
+                                      SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""enable_integrity_filters"":true,""id"":""{ID}"",""__relay_internal__pv__PolarisCannesGuardianExperienceEnabledrelayprovider"":true,""__relay_internal__pv__PolarisCASB976ProfileEnabledrelayprovider"":false,""__relay_internal__pv__PolarisWebSchoolsEnabledrelayprovider"":false,""__relay_internal__pv__PolarisRepostsConsumptionEnabledrelayprovider"":true" & "}"))
             UpdateRequestNumber()
             ChangeResponserMode(True)
             UpdateHeadersGQL(GQL_UserData_FbFriendlyName)
-            Dim r$ = Responser.GetResponse(GQL_URL, vars)
+            Dim r$ = Responser.GetResponse(GQL_URL_Q, vars)
-            If Not r.IsEmptyString Then
+            Return r
-                Using j As EContainer = JsonDocument.Parse(r)
+            'If Not r.IsEmptyString Then
-                    If j.ListExists Then
+            '    Using j As EContainer = JsonDocument.Parse(r)
-                        With j({"data", "user"})
+            '        If j.ListExists Then
-                            If .ListExists Then
+            '            With j({"data", "user"})
-                                UserSiteName = .Value("full_name").IfNullOrEmpty(UserSiteName)
+            '                If .ListExists Then
-                                Dim f As New SFile With {.Path = DownloadContentDefault_GetRootDir(), .Name = "ProfilePicture", .Extension = "jpg"}
+            '                    UserSiteName = .Value("full_name").IfNullOrEmpty(UserSiteName)
-                                Dim pic$ = .Value({"hd_profile_pic_url_info"}, "url").IfNullOrEmpty(.Value("profile_pic_url"))
+            '                    IsVerifiedProfile = .Value("is_verified").FromXML(Of Boolean)(False)
-                                If Not pic.IsEmptyString Then GetWebFile(pic, f, EDP.ReturnValue)
+            '                    IsVerifiedProfile_Checked = True
-                                UserDescriptionUpdate(.Value("biography"))
+            '                    Dim descr$ = .Value("biography")
-                            End If
+            '                    If If(.Item("bio_links")?.Count, 0) > 0 Then descr.StringAppend(.Item("bio_links").Select(Function(bl) bl.Value("url")).ListToString(vbNewLine), vbNewLine)
-                        End With
+            '                    Dim eUrl$ = .Value("external_url")
-                    End If
+            '                    If Not eUrl.IsEmptyString AndAlso (descr.IsEmptyString OrElse Not descr.Contains(eUrl)) Then descr.StringAppendLine(eUrl)
-                End Using
+            '                    UserDescriptionUpdate(descr)
-            End If
+
-        End Sub
+            '                    Dim f As New SFile With {.Path = DownloadContentDefault_GetRootDir(), .Name = "ProfilePicture", .Extension = "jpg"}
+            '                    Dim pic$ = .Value({"hd_profile_pic_url_info"}, "url").IfNullOrEmpty(.Value("profile_pic_url"))
+            '                    If Not pic.IsEmptyString Then GetWebFile(pic, f, EDP.ReturnValue)
+            '                End If
+            '            End With
+            '        End If
+            '    End Using
+            'End If
+        End Function
         Private Function GetTimelineGQL(ByVal Cursor As String, ByVal Token As CancellationToken) As String
             Const none_cursor$ = "none"
-            Dim nextCursor$ = String.Empty, hasNextPage$ = String.Empty
+            Dim nextCursor$ = String.Empty
+            Dim hasNextPage As Boolean = False
             Dim vars$
 
             ThrowAny(Token)
+            UpdateTokens(Not ValidateBaseTokens())
             UpdateRequestNumber()
             ChangeResponserMode(True)
 
             If Cursor.IsEmptyString Then
-                vars = "{""data"":{""count"":50,""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""username"":""" &
+                vars = "{""data"":{""count"":" & PostNumberPerRequest & ",""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""username"":""" &
                         NameTrue & """,""__relay_internal__pv__PolarisShareMenurelayprovider"":false}"
+                'vars = "{""data"":{""count"":" & PostNumberPerRequest & ",""include_reel_media_seen_timestamp"":true,""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""username"":""" &
+                '        NameTrue & """,""__relay_internal__pv__PolarisShareMenurelayprovider"":false}"
                 vars = String.Format(GQL_URL_PATTERN_VARS, GQL_Timeline_DocId, Token_lsd, Token_dtsg_Var, GQL_Timeline_FbFriendlyName,
                                      SymbolsConverter.ASCII.EncodeSymbolsOnly(vars))
                 UpdateHeadersGQL(GQL_Timeline_FbFriendlyName)
             Else
-                vars = "{""after"":""" & Cursor & """,""before"":null,""data"":{""count"":50,""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""first"":50,""last"":null,""username"":""" &
+                vars = "{""after"":""" & Cursor & """,""before"":null,""data"":{""count"":" & PostNumberPerRequest & ",""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""first"":" & PostNumberPerRequest & ",""last"":null,""username"":""" &
                         NameTrue & """,""__relay_internal__pv__PolarisShareMenurelayprovider"":false}"
+                'vars = "{""after"":""" & Cursor & """,""before"":null,""data"":{""count"":" & PostNumberPerRequest & ",""include_reel_media_seen_timestamp"":true,""include_relationship_info"":true,""latest_besties_reel_media"":true,""latest_reel_media"":true},""first"":" & PostNumberPerRequest & ",""last"":null,""username"":""" &
+                '        NameTrue & """}"
                 vars = String.Format(GQL_URL_PATTERN_VARS, GQL_Timeline_DocId_Second, Token_lsd, Token_dtsg_Var, GQL_Timeline_FbFriendlyName_Second,
                                      SymbolsConverter.ASCII.EncodeSymbolsOnly(vars))
                 UpdateHeadersGQL(GQL_Timeline_FbFriendlyName_Second)
@@ -139,8 +172,9 @@ Namespace API.Instagram
             Return If(hasNextPage And (Not nextCursor.IsEmptyString AndAlso Not nextCursor.StringToLower = none_cursor), nextCursor, String.Empty)
         End Function
         Private Function GetHighlightsGQL_List() As List(Of String)
-
+            UpdateTokens(Not ValidateBaseTokens())
-            Dim nextCursor$ = String.Empty, hasNextPage$ = String.Empty
+            Dim nextCursor$ = String.Empty
+            Dim hasNextPage As Boolean = False
             Dim i% = -1
             Dim hList As New List(Of String)
             Dim tmpList As New List(Of String)
@@ -174,12 +208,15 @@ Namespace API.Instagram
             Return hList
         End Function
         Private Sub GetHighlightsGQL(ByRef StoriesList As List(Of String), ByVal Token As CancellationToken)
-            Const highlightData$ = """first"":50,""initial_reel_id"":""{0}"",""last"":2,""reel_ids"":[{1}]"
+            Const highlightData$ = """initial_reel_id"":""{0}"",""reel_ids"":[{1}],""first"":50,""last"":2,""__relay_internal__pv__PolarisCommunityNoteStoriesLabelEnabledrelayprovider"":true"
             Dim tmpList As New List(Of String)
             Dim i% = -1
             If StoriesList.ListExists Then
-                tmpList.AddRange(StoriesList.Take(10))
+                'TODO: 5 Instagram stories
+                'tmpList.AddRange(StoriesList.Take(10))
+                tmpList.AddRange(StoriesList.Take(5))
                 StoriesList.RemoveRange(0, tmpList.Count)
+                UpdateTokens(Not ValidateBaseTokens())
 
                 Dim vars$ = String.Format(GQL_URL_PATTERN_VARS, GQL_Highlights_DocId_Second, Token_lsd, Token_dtsg_Var, GQL_Highlights_FbFriendlyName_Second,
                                           SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & String.Format(highlightData, tmpList(0), tmpList.Select(Function(hl) $"""{hl}""").ListToString(",")) & "}"))
@@ -206,6 +243,7 @@ Namespace API.Instagram
             tmpList.Clear()
         End Sub
         Private Sub GetUserStoriesGQL(ByVal Token As CancellationToken)
+            UpdateTokens(Not ValidateBaseTokens())
             '"{" & $"""user_id"":""{ID}""" & "}"
             Dim vars$ = String.Format(GQL_URL_PATTERN_VARS, GQL_UserStories_DocId, Token_lsd, Token_dtsg_Var, GQL_UserStories_FbFriendlyName,
                                       SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""reel_ids_arr"":[""{ID}""]" & "}"))
@@ -238,11 +276,9 @@ Namespace API.Instagram
         Private Function GetReelsGQL(ByVal Cursor As String) As String
             GetReelsGQL_SetEnvir = True
 
-            Dim errData$ = String.Empty
+            UpdateTokens(Cursor.IsEmptyString Or Not ValidateBaseTokens())
-            If Cursor.IsEmptyString And Not ValidateBaseTokens() Then GetPageTokens()
-            If Cursor.IsEmptyString And Not ValidateBaseTokens(errData) Then ValidateBaseTokens_Error(errData)
 
-            Dim vars$ = """data"":{""include_feed_video"":true,""page_size"":50,""target_user_id"":""" & ID & """}"
+            Dim vars$ = """data"":{""include_feed_video"":true,""page_size"":" & PostNumberPerRequest & ",""target_user_id"":""" & ID & """}"
             If Not Cursor.IsEmptyString Then vars = $"""after"":""{Cursor}"",""before"":null,{vars},""first"":4,""last"":null"
             vars = String.Format(GQL_URL_PATTERN_VARS, GQL_Reels_DocId, Token_lsd, Token_dtsg_Var, GQL_Reels_FbFriendlyName,
                                  SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & vars & "}"))
@@ -253,15 +289,16 @@ Namespace API.Instagram
         End Function
         ''' <summary>Response</summary>
         Private Function GetTaggedGQL(ByVal Cursor As String) As String
+            UpdateTokens(Not ValidateBaseTokens())
             'default count = 12
             'max count = 21
             Dim vars$
             If Cursor.IsEmptyString Then
                 vars = String.Format(GQL_URL_PATTERN_VARS, GQL_Tagged_DocId, Token_lsd, Token_dtsg_Var, GQL_Tagged_FbFriendlyName,
-                                     SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""count"":50,""user_id"":""{ID}""" & "}"))
+                                     SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""count"":{PostNumberPerRequest},""user_id"":""{ID}""" & "}"))
             Else
                 vars = String.Format(GQL_URL_PATTERN_VARS, GQL_Tagged_DocId, Token_lsd, Token_dtsg_Var, GQL_Tagged_FbFriendlyName,
-                                     SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""after"":""{Cursor}"",""before"":null,""count"":50,""first"":50,""last"":null,""user_id"":""{ID}""" & "}"))
+                                     SymbolsConverter.ASCII.EncodeSymbolsOnly("{" & $"""after"":""{Cursor}"",""before"":null,""count"":{PostNumberPerRequest},""first"":{PostNumberPerRequest},""last"":null,""user_id"":""{ID}""" & "}"))
             End If
             UpdateRequestNumber()
             ChangeResponserMode(True)
@@ -270,6 +307,13 @@ Namespace API.Instagram
         End Function
 #End Region
 #Region "ValidateBaseTokens"
+        Private Sub UpdateTokens(ByVal process As Boolean)
+            If process Then
+                Dim TokensErrData$ = String.Empty
+                If Not ValidateBaseTokens() Then GetPageTokens()
+                If Not ValidateBaseTokens(TokensErrData) Then ValidateBaseTokens_Error(TokensErrData)
+            End If
+        End Sub
         Protected Overridable Overloads Function ValidateBaseTokens() As Boolean
             Return ValidateBaseTokens(Nothing)
         End Function
@@ -296,7 +340,12 @@ Namespace API.Instagram
                         .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode, "navigate"))
                     End With
                 End With
-                Dim r$ = Responser.GetResponse(MySiteSettings.GetUserUrl(Me))
+                Dim r$
+                If UserNameRequested Then
+                    r = Responser.GetResponse("https://www.instagram.com/")
+                Else
+                    r = Responser.GetResponse(MySiteSettings.GetUserUrl(Me))
+                End If
                 ParseTokens(r, 0)
             Catch ex As Exception
             Finally
@@ -307,6 +356,10 @@ Namespace API.Instagram
             Try
                 If Not r.IsEmptyString Then
                     ResetBaseTokens()
+                    If ID.IsEmptyString Then
+                        Dim __id$ = RegexReplace(r, Regex_ProfileID)
+                        If CLng(AConvert(Of Long)(__id, 0, EDP.ReturnValue)) <> 0 Then ID = __id
+                    End If
                     Select Case Attempt
                         Case 0
                             Dim rr As RParams = RParams.DM(PageTokenRegexPatternDefault, 0, RegexReturn.List, EDP.ReturnValue)
@@ -318,6 +371,7 @@ Namespace API.Instagram
                                     .MatchSub = 1
                                     .WhatGet = RegexReturn.Value
                                 End With
+                                BlockVersionID = RegexReplace(r, Regex_BlockVersionID)
                                 For Each tt In tokens
                                     If Not Token_lsd.IsEmptyString And Not Token_dtsg.IsEmptyString Then
                                         Exit For
@@ -336,6 +390,7 @@ Namespace API.Instagram
                         Case 1
                             Token_dtsg = RegexReplace(r, Regex_UserToken_dtsg)
                             Token_lsd = RegexReplace(r, Regex_UserToken_lsd)
+                            BlockVersionID = RegexReplace(r, Regex_BlockVersionID)
                     End Select
                     If Not ValidateBaseTokens() And Attempt = 0 Then ParseTokens(r, Attempt + 1)
                 End If

SCrawler/API/Instagram/UserData.vb

@@ -39,6 +39,8 @@ Namespace API.Instagram
         Private Const Name_TaggedChecked As String = "TaggedChecked"
         Private Const Name_ForceUpdateUserName As String = "ForceUpdateUserName"
         Private Const Name_ForceUpdateUserInfo As String = "ForceUpdateUserInfo"
+        Private Const Name_IsVerifiedProfile As String = "IsVerifiedProfile"
+        Private Const Name_IsVerifiedProfile_Checked As String = "IsVerifiedProfile_Checked"
 #End Region
 #Region "Declarations"
         Friend Structure PostKV : Implements IEContainerProvider
@@ -115,6 +117,13 @@ Namespace API.Instagram
         Private UserNameRequested As Boolean = False
         Friend Property ForceUpdateUserName As Boolean = False
         Friend Property ForceUpdateUserInfo As Boolean = False
+        Friend Property IsVerifiedProfile As Boolean = False
+        Friend Property IsVerifiedProfile_Checked As Boolean = False
+        Private ReadOnly Property PostNumberPerRequest As Integer
+            Get
+                With MySiteSettings : Return If(IsVerifiedProfile, .PostNumberVerified, .PostNumberVerifiedNot).Value : End With
+            End Get
+        End Property
 #End Region
 #Region "Loader"
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
@@ -136,6 +145,8 @@ Namespace API.Instagram
                     TaggedChecked = .Value(Name_TaggedChecked).FromXML(Of Boolean)(False)
                     ForceUpdateUserName = .Value(Name_ForceUpdateUserName).FromXML(Of Boolean)(False)
                     ForceUpdateUserInfo = .Value(Name_ForceUpdateUserInfo).FromXML(Of Boolean)(False)
+                    IsVerifiedProfile = .Value(Name_IsVerifiedProfile).FromXML(Of Boolean)(False)
+                    IsVerifiedProfile_Checked = .Value(Name_IsVerifiedProfile_Checked).FromXML(Of Boolean)(False)
                 Else
                     .Add(Name_LastCursor, LastCursor)
                     .Add(Name_FirstLoadingDone, FirstLoadingDone.BoolToInteger)
@@ -153,6 +164,8 @@ Namespace API.Instagram
                     .Add(Name_TaggedChecked, TaggedChecked.BoolToInteger)
                     .Add(Name_ForceUpdateUserName, ForceUpdateUserName.BoolToInteger)
                     .Add(Name_ForceUpdateUserInfo, ForceUpdateUserInfo.BoolToInteger)
+                    .Add(Name_IsVerifiedProfile, IsVerifiedProfile.BoolToInteger)
+                    .Add(Name_IsVerifiedProfile_Checked, IsVerifiedProfile_Checked.BoolToInteger)
                 End If
             End With
         End Sub
@@ -164,7 +177,7 @@ Namespace API.Instagram
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
             If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then
                 With DirectCast(Obj, EditorExchangeOptions)
-                    .ApplyBase(Me)
+                    .Apply(Me)
                     GetTimeline = .GetTimeline
                     GetReels = .GetReels
                     GetStories = .GetStories
@@ -179,6 +192,9 @@ Namespace API.Instagram
 
                     PutImageVideoFolder = .PutImageVideoFolder
 
+                    IsVerifiedProfile = .IsVerifiedProfile
+                    If IsVerifiedProfile Then IsVerifiedProfile_Checked = True
+
                     ForceUpdateUserName = .ForceUpdateUserName
                     ForceUpdateUserInfo = .ForceUpdateUserInfo
                 End With
@@ -317,6 +333,7 @@ Namespace API.Instagram
         Private _LastWwwClaim As String = String.Empty
         Private _ResponserGQLMode As Boolean = False
         Private _UseGQL As Boolean = False
+        Private Const ImgHeic$ = "heic"
         Private Sub ChangeResponserMode(ByVal GQL As Boolean, Optional ByVal Force As Boolean = False)
             If Not _ResponserGQLMode = GQL Or Force Then
                 _ResponserGQLMode = GQL
@@ -346,8 +363,7 @@ Namespace API.Instagram
                         With .Headers
                             .Remove("origin")
                             .Remove("authority")
-                            .Remove(GQL_HEADER_FB_FRINDLY_NAME)
+                            UpdateHeadersGQL(String.Empty, False)
-                            .Remove(GQL_HEADER_FB_LSD)
                             Dim hv$ = MySiteSettings.Responser.Headers.Value(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchDest)).IfNullOrEmpty("empty")
                             .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchDest, hv))
                             hv = MySiteSettings.Responser.Headers.Value(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode)).IfNullOrEmpty("cors")
@@ -491,26 +507,26 @@ Namespace API.Instagram
                 If Not errorFound Then LoadSavePostsKV(False)
             End Try
         End Sub
+        'TODELETE: ValidateExtension
         Protected Sub ValidateExtension()
-            Dim tmpList As List(Of UserMedia) = Nothing
+            'Dim tmpList As List(Of UserMedia) = Nothing
-            Try
+            'Try
-                Const heic$ = "heic"
+            '    If _TempMediaList.Count > 0 AndAlso _TempMediaList.Exists(Function(mm) mm.File.Extension = ImgHeic) Then
-                If _TempMediaList.Count > 0 AndAlso _TempMediaList.Exists(Function(mm) mm.File.Extension = heic) Then
+            '        Dim m As UserMedia
-                    Dim m As UserMedia
+            '        tmpList = New List(Of UserMedia)
-                    tmpList = New List(Of UserMedia)
+            '        tmpList.ListAddList(_TempMediaList)
-                    tmpList.ListAddList(_TempMediaList)
+            '        _TempMediaList.Clear()
-                    _TempMediaList.Clear()
+            '        For i% = 0 To tmpList.Count - 1
-                    For i% = 0 To tmpList.Count - 1
+            '            m = tmpList(i)
-                        m = tmpList(i)
+            '            _TempMediaList.Add(m)
-                        _TempMediaList.Add(m)
+            '            If m.Type = UTypes.Picture AndAlso Not m.File.Extension.IsEmptyString AndAlso m.File.Extension = ImgHeic Then _
-                        If m.Type = UTypes.Picture AndAlso Not m.File.Extension.IsEmptyString AndAlso m.File.Extension = heic Then _
+            '               m.File.Extension = "jpg" : _TempMediaList.Add(m)
-                           m.File.Extension = "jpg" : _TempMediaList.Add(m)
+            '        Next
-                    Next
+            '        tmpList.Clear()
-                    tmpList.Clear()
+            '    End If
-                End If
+            'Catch ex As Exception
-            Catch ex As Exception
+            '    If tmpList.ListExists Then _TempMediaList.Clear() : _TempMediaList.ListAddList(tmpList) : tmpList.Clear()
-                If tmpList.ListExists Then _TempMediaList.Clear() : _TempMediaList.ListAddList(tmpList) : tmpList.Clear()
+            'End Try
-            End Try
         End Sub
         Protected Overridable Sub UpdateResponser()
             Try
@@ -663,6 +679,7 @@ Namespace API.Instagram
             Dim StoriesList As List(Of String) = Nothing
             Dim StoriesRequested As Boolean = False
             Dim dValue% = 1
+            Dim __idIsEmpty As Boolean = ID.IsEmptyString
             LastCursor = Cursor
             Try
                 Do While dValue = 1
@@ -676,7 +693,6 @@ Namespace API.Instagram
                         Dim HasNextPage As Boolean = False
                         Dim EndCursor$ = String.Empty
                         Dim PostID$ = String.Empty, PostDate$ = String.Empty, SpecFolder$ = String.Empty
-                        Dim TokensErrData$ = String.Empty
                         Dim PostIDKV As PostKV
                         Dim ENode() As Object = Nothing
                         Dim processGetResponse As Boolean = True
@@ -684,14 +700,11 @@ Namespace API.Instagram
 
                         'Check environment
                         If Not IsSavedPosts Then
-                            If ID.IsEmptyString Then GetUserData()
+                            If _UseGQL And Cursor.IsEmptyString And Not Section = Sections.SavedPosts Then UpdateTokens(True)
-                            If ID.IsEmptyString Then UserExists = False : _ForceSaveUserInfoOnException = True : Throw New Plugin.ExitException("can't get user ID")
+                            If ID.IsEmptyString Or __idIsEmpty Or Not IsVerifiedProfile_Checked Then GetUserData(Token)
-                            If _UseGQL And Cursor.IsEmptyString And Not Section = Sections.SavedPosts Then
+                            If ID.IsEmptyString Then UserExists = False : _ForceSaveUserInfo = True : Throw New Plugin.ExitException("can't get user ID")
-                                If Not ValidateBaseTokens() Then GetPageTokens()
+                            If ForceUpdateUserName Then GetUserNameById(Token)
-                                If Not ValidateBaseTokens(TokensErrData) Then ValidateBaseTokens_Error(TokensErrData)
+                            If ForceUpdateUserInfo Then GetUserData(Token)
-                            End If
-                            If ForceUpdateUserName Then GetUserNameById()
-                            If ForceUpdateUserInfo Then GetUserData()
                         End If
 
                         'Create query
@@ -703,7 +716,7 @@ Namespace API.Instagram
                                     MySiteSettings.TooManyRequests(False)
                                     GoTo NextPageBlock
                                 Else
-                                    URL = $"https://www.instagram.com/api/v1/feed/user/{NameTrue}/username/?count=50" &
+                                    URL = $"https://www.instagram.com/api/v1/feed/user/{NameTrue}/username/?count={PostNumberPerRequest}" &
                                            If(Cursor.IsEmptyString, String.Empty, $"&max_id={Cursor}")
                                     ENode = Nothing
                                 End If
@@ -726,7 +739,7 @@ Namespace API.Instagram
                                     ENode = {"data", "xdt_api__v1__usertags__user_id__feed_connection"}
                                     processGetResponse = False
                                 Else
-                                    Dim vars$ = "{""id"":" & ID & ",""first"":50,""after"":""" & Cursor & """}"
+                                    Dim vars$ = "{""id"":" & ID & $",""first"":{PostNumberPerRequest},""after"":""" & Cursor & """}"
                                     vars = SymbolsConverter.ASCII.EncodeSymbolsOnly(vars)
                                     URL = $"https://www.instagram.com/graphql/query/?doc_id=17946422347485809&variables={vars}"
                                     ENode = {"data", "user", "edge_user_to_photos_of_you"}
@@ -1241,25 +1254,38 @@ NextPageBlock:
         End Sub
 #End Region
 #Region "GetUserId, GetUserName"
-        Private Sub GetUserData()
+        Private Sub GetUserData(ByVal Token As CancellationToken)
             Dim __idFound As Boolean = False
             If ForceUpdateUserInfo Then ForceUpdateUserInfo = False : _ForceSaveUserInfo = True
             Try
+                Dim r$
+                Dim ____dataGql As Boolean = _UseGQL Or CBool(MySiteSettings.USE_GQL_UserData.Value)
+                If ____dataGql Then
+                    UpdateTokens(True)
+                    r = GetUserDataGQL(Token)
+                    If Not _UseGQL Then ChangeResponserMode(_UseGQL)
+                Else
                     ChangeResponserMode(False)
                     UpdateRequestNumber()
-                Dim r$ = Responser.GetResponse($"https://i.instagram.com/api/v1/users/web_profile_info/?username={NameTrue}")
+                    r = Responser.GetResponse($"https://i.instagram.com/api/v1/users/web_profile_info/?username={NameTrue}")
+                End If
+
                 If Not r.IsEmptyString Then
                     Using j As EContainer = JsonDocument.Parse(r)
                         If Not j Is Nothing AndAlso j.Contains({"data", "user"}) Then
                             With j({"data", "user"})
-                                ID = .Value("id")
+                                If Not ____dataGql Or ID.IsEmptyString Then ID = .Value("id")
                                 __idFound = True
+                                If Not .Value("username").IsEmptyString Then NameTrue = .Value("username")
                                 UserSiteNameUpdate(.Value("full_name"))
+                                IsVerifiedProfile = .Value("is_verified").FromXML(Of Boolean)(False)
+                                IsVerifiedProfile_Checked = True
                                 Dim descr$ = .Value("biography")
                                 If If(.Item("bio_links")?.Count, 0) > 0 Then descr.StringAppend(.Item("bio_links").Select(Function(bl) bl.Value("url")).ListToString(vbNewLine), vbNewLine)
                                 Dim eUrl$ = .Value("external_url")
                                 If Not eUrl.IsEmptyString AndAlso (descr.IsEmptyString OrElse Not descr.Contains(eUrl)) Then descr.StringAppendLine(eUrl)
                                 UserDescriptionUpdate(descr)
+
                                 Dim f As New SFile With {.Path = DownloadContentDefault_GetRootDir(), .Name = "ProfilePicture", .Extension = "jpg"}
                                 f = SFile.IndexReindex(f)
                                 If Not f.Exists Then
@@ -1283,9 +1309,17 @@ NextPageBlock:
                 ChangeResponserMode(_UseGQL)
             End Try
         End Sub
-        Private Function GetUserNameById() As Boolean
+        Private Function GetUserNameById(ByVal Token As CancellationToken) As Boolean
             UserNameRequested = True
             If ForceUpdateUserName Then ForceUpdateUserName = False : _ForceSaveUserInfo = True
+            If Not ID.IsEmptyString Then
+                Dim currentName$ = NameTrue
+                GetUserData(Token)
+                If Not currentName = NameTrue Then UserNameRequested = False : Return True
+            End If
+
+            Return False
+
             Try
                 If Not ID.IsEmptyString Then
                     UpdateRequestNumber()
@@ -1319,6 +1353,7 @@ NextPageBlock:
 #Region "Pinned stories"
         Private Sub GetStoriesData(ByRef StoriesList As List(Of String), ByVal GetUserStory As Boolean, ByVal Token As CancellationToken)
             Const ReqUrl$ = "https://i.instagram.com/api/v1/feed/reels_media/?{0}"
+            Const hPattern$ = "highlight:"
             Dim tmpList As IEnumerable(Of String) = Nothing
             Dim qStr$, r$
             Dim i% = -1
@@ -1331,7 +1366,7 @@ NextPageBlock:
                     If GetUserStory Then
                         qStr = $"https://www.instagram.com/api/v1/feed/reels_media/?reel_ids={ID}"
                     Else
-                        qStr = String.Format(ReqUrl, tmpList.Select(Function(q) $"reel_ids=highlight:{q}").ListToString("&"))
+                        qStr = String.Format(ReqUrl, tmpList.Select(Function(q) $"reel_ids={hPattern}{q.Replace(hPattern, String.Empty)}").ListToString("&"))
                     End If
                     UpdateRequestNumber()
                     r = Responser.GetResponse(qStr,, EDP.ThrowException)
@@ -1380,6 +1415,11 @@ NextPageBlock:
             End If
         End Sub
         Private Function GetStoriesList() As List(Of String)
+            If CBool(MySiteSettings.USE_GQL_Highlights.Value) Then
+                Dim hList As List(Of String) = GetHighlightsGQL_List()
+                If Not _UseGQL Then ChangeResponserMode(_UseGQL)
+                Return hList
+            Else
                 UpdateRequestNumber()
                 Dim r$ = Responser.GetResponse($"https://i.instagram.com/api/v1/highlights/{ID}/highlights_tray/",, EDP.ThrowException)
                 If Not r.IsEmptyString Then
@@ -1389,12 +1429,20 @@ NextPageBlock:
                     End Using
                 End If
                 Return Nothing
+            End If
         End Function
 #End Region
 #Region "Download content"
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
             DownloadContentDefault(Token)
         End Sub
+        Protected Overrides Function DownloadContentDefault_ConvertWebp(ByVal m As UserMedia, ByVal Process As Boolean) As SFile
+            If m.Type = UTypes.Picture AndAlso Not m.File.Extension.IsEmptyString AndAlso m.File.Extension = ImgHeic Then
+                Return DownloadContentDefault_ConvertWebp_Impl(m, True, True)
+            Else
+                Return MyBase.DownloadContentDefault_ConvertWebp(m, Process)
+            End If
+        End Function
 #End Region
 #Region "Erase"
         Protected Overrides Sub EraseData_AdditionalDataFiles()
@@ -1417,7 +1465,7 @@ NextPageBlock:
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal s As Object = Nothing) As Integer
             If Responser.StatusCode = HttpStatusCode.NotFound Then '404
-                If Not UserNameRequested AndAlso GetUserNameById() Then Return 1 Else UserExists = False
+                If Not UserNameRequested AndAlso GetUserNameById(New CancellationToken) Then Return 1 Else UserExists = False
             ElseIf Responser.StatusCode = HttpStatusCode.BadRequest Or Responser.StatusCode = HttpStatusCode.Unauthorized Then '400, 401
                 HasError = True
                 MyMainLOG = $"Instagram credentials have expired [{CInt(Responser.StatusCode)}]: {ToStringForLog()} [{s}]"
@@ -1434,7 +1482,7 @@ NextPageBlock:
                 MyMainLOG = $"Number of requests before error 429: {RequestsCount}"
                 Return 1
             ElseIf Responser.StatusCode = 560 Or Responser.StatusCode = HttpStatusCode.InternalServerError Then '560, 500
-                If Responser.StatusCode = 560 And s = Sections.Stories And MySiteSettings.IgnoreStoriesDownloadingErrors Then
+                If Responser.StatusCode = 560 And s = Sections.Stories And MySiteSettings.IgnoreStoriesDownloadingErrors.Value Then
                     MyMainLOG = $"{ToStringForLog()}: Stories downloading skipped (560)"
                     Return ErrHandlingValueStories
                 Else

SCrawler/API/OnlyFans/Declarations.vb

@@ -17,5 +17,23 @@ Namespace API.OnlyFans
             {{"files", "full", "url"}}
         }
         Friend Property Rules As DynamicRulesEnv
+        Friend Class CookieDateProvider : Implements ICustomProvider
+            Private ReadOnly DefaultProvider As IFormatProvider
+            Friend Sub New()
+                DefaultProvider = PersonalUtilities.Tools.Web.Cookies.CookieKeeper.DateProviderDefault
+            End Sub
+            Friend Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider,
+                                    Optional ByVal NothingArg As Object = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Object Implements ICustomProvider.Convert
+                If Not IsNothing(Value) AndAlso TypeOf Value Is String AndAlso Not CStr(Value).IsEmptyString Then
+                    Dim v$ = Value
+                    If v.Contains(",_") Then v = v.Substring(0, v.IndexOf(",_"))
+                    Value = v
+                End If
+                Return AConvert(Of Date)(Value, DefaultProvider, Nothing)
+            End Function
+            Private Function GetFormat(ByVal FormatType As Type) As Object Implements IFormatProvider.GetFormat
+                Throw New NotImplementedException("'GetFormat' is not available in the 'CookieDateProvider'")
+            End Function
+        End Class
     End Module
 End Namespace

SCrawler/API/OnlyFans/SiteSettings.vb

@@ -99,6 +99,19 @@ Namespace API.OnlyFans
             End Get
         End Property
 #End Region
+#Region "Other"
+        <PClonable, PXML("OpenPostsUsingID")> Private ReadOnly Property OpenPostsUsingID_XML As PropertyValue
+        <PropertyOption(ControlText:="Open posts using ID", ControlToolTip:="Open posts using the user ID instead of the user name"), HiddenControl>
+        Private ReadOnly Property OpenPostsUsingID As PropertyValue
+            Get
+                If Not DefaultInstance Is Nothing Then
+                    Return DirectCast(DefaultInstance, SiteSettings).OpenPostsUsingID_XML
+                Else
+                    Return OpenPostsUsingID_XML
+                End If
+            End Get
+        End Property
+#End Region
 #Region "OFScraper"
         <PClonable, PXML("OFScraperPath")> Private ReadOnly Property OFScraperPath_XML As PropertyValue
         <PropertyOption(ControlText:="OF-Scraper path", ControlToolTip:="The path to the 'ofscraper.exe'", Category:=CAT_OFS)>
@@ -240,6 +253,7 @@ Namespace API.OnlyFans
                 .CookiesUpdateMode = CookieKeeper.UpdateModes.Disabled
                 .Cookies.ChangedAllowInternalDrop = False
                 .Cookies.Changed = False
+                .Cookies.DateProvider = New CookieDateProvider
                 With .Headers
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform))
                     .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaMobile))
@@ -285,6 +299,8 @@ Namespace API.OnlyFans
 
             UpdateRules401_XML = New PropertyValue(False)
 
+            OpenPostsUsingID_XML = New PropertyValue(True)
+
             UserRegex = RParams.DMS(String.Format(UserRegexDefaultPattern, "onlyfans.com/"), 1, EDP.ReturnValue)
             UrlPatternUser = "https://onlyfans.com/{0}"
             ImageVideoContains = "onlyfans.com"
@@ -363,8 +379,9 @@ Namespace API.OnlyFans
                 End If
                 If p.IsEmptyString Then
                     Return GetUserUrl(User)
+                ElseIf CBool(OpenPostsUsingID.Value) Then
+                    Return String.Format(UserPostPattern, p, If(User.ID.IsEmptyString, User.NameTrue, $"u{User.ID}"))
                 Else
-                    'Return String.Format(UserPostPattern, p, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
                     Return String.Format(UserPostPattern, p, User.NameTrue)
                 End If
             Else

SCrawler/API/OnlyFans/UserData.vb

@@ -66,7 +66,7 @@ Namespace API.OnlyFans
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
             If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
                 With DirectCast(Obj, UserExchangeOptions)
-                    .ApplyBase(Me)
+                    .Apply(Me)
                     MediaDownloadTimeline = .DownloadTimeline
                     MediaDownloadStories = .DownloadStories
                     MediaDownloadHighlights = .DownloadHighlights

SCrawler/API/Pinterest/EditorExchangeOptions.vb

@@ -6,15 +6,21 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
 Imports SCrawler.Plugin.Attributes
 Namespace API.Pinterest
-    Friend Class EditorExchangeOptions
+    Friend Class EditorExchangeOptions : Inherits EditorExchangeOptionsBase
         <PSetting(Caption:="Get Sub-Boards", ToolTip:="Extract the Sub-Boards from the boards and download them")>
         Friend Property ExtractSubBoards As Boolean = True
         Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
             ExtractSubBoards = u.ExtractSubBoards
         End Sub
         Friend Sub New()
         End Sub
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
+            MyBase.Apply(u)
+            DirectCast(u, UserData).ExtractSubBoards = ExtractSubBoards
+        End Sub
     End Class
 End Namespace

SCrawler/API/Pinterest/UserData.vb

@@ -97,7 +97,7 @@ Namespace API.Pinterest
             Return New EditorExchangeOptions(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
-            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then ExtractSubBoards = DirectCast(Obj, EditorExchangeOptions).ExtractSubBoards
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then DirectCast(Obj, EditorExchangeOptions).Apply(Me)
         End Sub
 #End Region
 #Region "Download overrides"

SCrawler/API/PornHub/Declarations.vb

@@ -24,7 +24,12 @@ Namespace API.PornHub
                                                                 0, RegexOptions.Singleline, RegexReturn.List, EDP.ReturnValue, UnicodeHexConverter)
         Friend ReadOnly RegexVideo_Video_VideoKey As RParams = RParams.DMS("viewkey=([\w\d]+)", 1, EDP.ReturnValue)
         Friend ReadOnly RegexVideoPageTitle As RParams = RParams.DMS("meta (property|name)=""[^:]+?:title"" content=""([^""]+)""", 2, EDP.ReturnValue)
+        Friend ReadOnly RegexVideoPageTitle2 As RParams = RParams.DMS("\<title\>([^\<]+)\</title>", 1, EDP.ReturnValue)
+        Friend ReadOnly RegexVideoPageTitle3 As RParams = RParams.DMS("videoTitle...([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly RegexVideoPageTitle_NoTitle As RParams = RParams.DMS("(?<=(\=|/))([\w\d]+)(?=(\?|\&|\Z))", 2, EDP.ReturnValue)
         Friend ReadOnly RegexDataToken As RParams = RParams.DMS("data-token=""([^""]+)", 1, EDP.ReturnValue)
+        Friend ReadOnly RegexVideoAdditImg As RParams = RParams.DMS("twitter.image..content=""([^""]+)""", 1, EDP.ReturnValue)
+        Friend ReadOnly RegexVideoAdditImg2 As RParams = RParams.DMS("thumbnailUrl""[^""]*?""([^""]+)""", 1, EDP.ReturnValue)
 #End Region
 #Region "Declarations M3U8"
         Friend ReadOnly Regex_M3U8_FilesList As RParams = RParams.DM("RESOLUTION=\d+x(\d+).*?[\r\n]*?(.+?m3u8.*)", 0, RegexReturn.List, EDP.ReturnValue)

SCrawler/API/PornHub/M3U8.vb

@@ -22,7 +22,7 @@ Namespace API.PornHub
             If Not r.IsEmptyString Then
                 Dim files As List(Of Sizes) = RegexFields(Of Sizes)(r, {Regex_M3U8_FilesList}, {1, 2}, EDP.ReturnValue)
                 Dim file$
-                If files.ListExists Then files.RemoveAll(Function(f) f.Value = 0 Or (Not DownloadUHD And f.Value > 1080))
+                If files.ListExists Then files.RemoveAll(Function(f) f.Value = 0 Or (Not DownloadUHD And f.Value > 1080) Or (f.Data.IsEmptyString OrElse f.Data.Contains("""")))
                 If files.ListExists Then
                     files.Sort()
                     file = files(0).Data

SCrawler/API/PornHub/UserData.vb

@@ -6,13 +6,14 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Security.Policy
 Imports System.Threading
-Imports SCrawler.API.Base
-Imports SCrawler.API.YouTube.Objects
-Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports SCrawler.API.Base
+Imports SCrawler.API.YouTube.Objects
 Imports UTypes = SCrawler.API.Base.UserMedia.Types
 Namespace API.PornHub
     Friend Class UserData : Inherits UserDataBase : Implements IPSite
@@ -96,6 +97,9 @@ Namespace API.PornHub
                 Return Me
             End Function
         End Structure
+        Private Structure PHObj : Implements IThumbList
+            Friend Property Thumbs As List(Of String) Implements IThumbList.Thumbs
+        End Structure
 #End Region
 #Region "Enums"
         Private Enum VideoTypes
@@ -258,7 +262,6 @@ Namespace API.PornHub
         Private _PageVideosRepeat As Integer = 0
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
-                UpdateM3U8URLS = False
                 PlaylistToken = String.Empty
                 Responser.ResetStatus()
                 _PageVideosRepeat = 0
@@ -648,6 +651,21 @@ Namespace API.PornHub
 #End Region
 #End Region
 #Region "ReparseVideo"
+        Private Function GetVideoTitle(ByVal r As String, ByVal URL As String) As String
+            Dim tmpName$ = String.Empty
+            Dim rr As RParams
+            For i As Byte = 0 To 3
+                Select Case i
+                    Case 0 : rr = RegexVideoPageTitle
+                    Case 1 : rr = RegexVideoPageTitle2
+                    Case 2 : rr = RegexVideoPageTitle3
+                    Case Else : rr = RegexVideoPageTitle_NoTitle
+                End Select
+                tmpName = RegexReplace(IIf(i = 3, URL, r), rr)
+                If Not tmpName.IsEmptyString Then Exit For
+            Next
+            Return tmpName
+        End Function
         Protected Overloads Overrides Sub ReparseVideo(ByVal Token As CancellationToken)
             If IsSubscription Then
                 ReparseVideoSubscriptions(Token)
@@ -663,6 +681,7 @@ Namespace API.PornHub
                 If _TempMediaList.Count > 0 AndAlso _TempMediaList.Exists(Function(tm) tm.Type = UTypes.VideoPre) Then
                     Dim m As UserMedia
                     Dim r$, NewUrl$, tmpName$
+                    Dim isCurl%
                     ProgressPre.ChangeMax(_TempMediaList.Count)
                     For i% = _TempMediaList.Count - 1 To 0 Step -1
                         ProgressPre.Perform()
@@ -671,7 +690,8 @@ Namespace API.PornHub
                             ThrowAny(Token)
                             Try
                                 URL = m.URL
-                                r = Responser.Curl(URL)
+                                For isCurl = 1 To 0 Step -1
+                                    If CBool(isCurl) Then r = Responser.Curl(URL) Else r = Responser.GetResponse(URL)
                                     If Not r.IsEmptyString Then
                                         NewUrl = CreateVideoURL(r)
                                         If NewUrl.IsEmptyString Then
@@ -680,7 +700,7 @@ Namespace API.PornHub
                                             m.URL = NewUrl
                                             m.Type = UTypes.m3u8
                                             If CreateFileName Then
-                                            tmpName = RegexReplace(r, RegexVideoPageTitle)
+                                                tmpName = GetVideoTitle(r, URL)
                                                 If Not tmpName.IsEmptyString Then
                                                     If Not Data Is Nothing Then Data.Title = tmpName
                                                     m.File.Name = TitleHtmlConverter(tmpName)
@@ -689,9 +709,11 @@ Namespace API.PornHub
                                             End If
                                             _TempMediaList(i) = m
                                         End If
-                                Else
+                                        Exit For
+                                    ElseIf Not CBool(isCurl) Then
                                         _TempMediaList.RemoveAt(i)
                                     End If
+                                Next
                             Catch mid_ex As Exception
                                 If mid_ex.HelpLink = ERR_NEW_URL OrElse DownloadingException(mid_ex, "") = 1 Then
                                     m.State = UserMedia.States.Missing
@@ -713,6 +735,10 @@ Namespace API.PornHub
                     Dim m As UserMedia
                     Dim r$, URL$, tmpName$, thumb$
                     Dim c% = 0
+                    Dim thumbObj As PHObj
+                    Dim thumbAdded As Boolean
+                    Dim rp As RParams
+                    Dim rpa() As RParams = {RegexVideoAdditImg, RegexVideoAdditImg2}
                     Dim rErr As New ErrorsDescriber(EDP.ReturnValue)
                     Progress.Maximum += _TempMediaList.Count
                     For i% = _TempMediaList.Count - 1 To 0 Step -1
@@ -727,10 +753,24 @@ Namespace API.PornHub
                                     If Not r.IsEmptyString Then
                                         m.Type = UTypes.m3u8
 
+                                        thumbObj = New PHObj With {.Thumbs = New List(Of String)}
+                                        thumbAdded = False
                                         thumb = RegexReplace(r, Regex_VideosThumb_OG_IMAGE)
-                                        If Not thumb.IsEmptyString Then m.URL = thumb
+                                        If Not thumb.IsEmptyString Then m.URL = thumb : thumbAdded = True
+                                        For Each rp In rpa
+                                            thumb = RegexReplace(r, rp)
+                                            If Not thumb.IsEmptyString Then
+                                                If Not thumbAdded Then
+                                                    m.URL = thumb
+                                                    thumbAdded = True
+                                                Else
+                                                    thumbObj.Thumbs.Add(thumb)
+                                                End If
+                                            End If
+                                        Next
+                                        If thumbObj.Thumbs.Count > 0 Then m.Object = thumbObj
 
-                                        tmpName = RegexReplace(r, RegexVideoPageTitle)
+                                        tmpName = GetVideoTitle(r, URL)
                                         If Not tmpName.IsEmptyString Then
                                             m.File.Name = TitleHtmlConverter(tmpName)
                                             m.File.Extension = "mp4"
@@ -785,39 +825,34 @@ Namespace API.PornHub
             Catch ex As Exception
                 ProcessException(ex, Token, "missing data downloading error")
             Finally
-                If rList.Count > 0 Then
+                ReparseMissing_ClearList(rList)
-                    For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
-                    rList.Clear()
-                End If
             End Try
         End Sub
 #End Region
 #Region "Download content"
-        Private UpdateM3U8URLS As Boolean = False
-        Private UpdateM3U8URLS_Error As Boolean = False
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
-            Try : DownloadContentDefault(Token) : Finally : UpdateM3U8URLS = False : End Try
+            DownloadContentDefault(Token)
         End Sub
         Protected Overloads Overrides Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile,
                                                             ByVal Token As CancellationToken) As SFile
-            UpdateM3U8URLS_Error = False
+            Return DownloadM3U8(URL, Media, DestinationFile, Token, 0)
-            Return DownloadM3U8(URL, Media, DestinationFile, Token, UpdateM3U8URLS)
         End Function
         Private Overloads Function DownloadM3U8(ByVal URL As String, ByVal Media As UserMedia, ByVal DestinationFile As SFile,
-                                                ByVal Token As CancellationToken, ByVal Second As Boolean) As SFile
+                                                ByVal Token As CancellationToken, ByVal Round As Integer) As SFile
+            Const MaxRound% = 2
             Try
-                If Second Then
+                If Round > 0 Then
                     Dim r$ = Responser.Curl(Media.URL_BASE,, EDP.ReturnValue)
                     If Not r.IsEmptyString Then Media.URL = CreateVideoURL(r).IfNullOrEmpty(URL) : URL = Media.URL
                 End If
                 Dim f As SFile = M3U8.Download(URL, Responser, DestinationFile, DownloadUHD, Token, Progress, Not IsSingleObjectDownload)
-                If Not f.Exists And Not Second Then UpdateM3U8URLS = True : f = DownloadM3U8(URL, Media, DestinationFile, Token, True)
+                If Not f.Exists Then f = Nothing
+                If f.IsEmptyString And Round < MaxRound Then f = DownloadM3U8(URL, Media, DestinationFile, Token, Round + 1)
                 Return f
             Catch ex As Exception
-                If Not UpdateM3U8URLS_Error Then
+                If Round < MaxRound Then
-                    UpdateM3U8URLS_Error = True
                     Thread.Sleep(1000)
-                    Return DownloadM3U8(URL, Media, DestinationFile, Token, True)
+                    Return DownloadM3U8(URL, Media, DestinationFile, Token, Round + 1)
                 End If
                 Return Nothing
             End Try
@@ -917,7 +952,6 @@ Namespace API.PornHub
 #End Region
 #Region "DownloadSingleObject"
         Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
-            UpdateM3U8URLS = False
             _TempMediaList.Add(New UserMedia(Data.URL, UTypes.VideoPre))
             ReparseVideo(Token, True, Data)
         End Sub

SCrawler/API/PornHub/UserExchangeOptions.vb

@@ -44,7 +44,7 @@ Namespace API.PornHub
             DownloadGifs = Not v = CheckState.Unchecked
             MySettings = s
         End Sub
-        Friend Overrides Sub Apply(ByRef u As IPSite)
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
             MyBase.Apply(u)
             With DirectCast(u, UserData)
                 .DownloadUHD = DownloadUHD

SCrawler/API/Reddit/UserData.vb

@@ -136,7 +136,7 @@ Namespace API.Reddit
                     DownloadTextSpecialFolder = .DownloadTextSpecialFolder
                     RedGifsAccount = .RedGifsAccount
                     RedditAccount = .RedditAccount
-                    If TypeOf Options Is RedditViewExchange Then DirectCast(Options, RedditViewExchange).ApplyBase(Me)
+                    If TypeOf Options Is RedditViewExchange Then DirectCast(Options, RedditViewExchange).Apply(Me)
                 End With
             End If
         End Sub
@@ -1022,11 +1022,11 @@ Namespace API.Reddit
                                             If .ListExists AndAlso ParseContainer(.Self, m.Post.ID, String.Empty,,, GetTextDocument(.Self)) Then
                                                 If lastCount <> _TempMediaList.Count Then
                                                     For li = IIf(lastCount < 0, 0, lastCount) To _TempMediaList.Count - 1
-                                                        m2 = _TempMediaList(i)
+                                                        m2 = _TempMediaList(li)
                                                         m2.Post.Date = m.Post.Date
                                                         m2.State = UStates.Missing
                                                         m2.Attempts = m.Attempts
-                                                        _TempMediaList(i) = m2
+                                                        _TempMediaList(li) = m2
                                                     Next
                                                 End If
                                                 rList.Add(i)

SCrawler/API/ThisVid/Declarations.vb

@@ -7,6 +7,8 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Cookies
+Imports PersonalUtilities.Tools.Web.Clients
 Namespace API.ThisVid
     Friend Module Declarations
         Friend Const ThisVidSiteKey As String = "AndyProgram_ThisVid"
@@ -22,5 +24,27 @@ Namespace API.ThisVid
         Friend ReadOnly RegExVideosThumb1 As RParams = RParams.DMS("preview_url:\s*'([^""']+)'", 1, EDP.ReturnValue)
         Friend ReadOnly RegExVideosThumb2 As RParams = RParams.DMS("preview_url1:\s*'([^""']+)'", 1, EDP.ReturnValue)
         Friend ReadOnly RegExVideoTitle As RParams = RParams.DMS("meta property=.og:title..content=""([^""]*)""", 1, EDP.ReturnValue)
+        Friend Class CookieKeeper2 : Inherits CookieKeeper
+            Public Overrides Function Update(Optional ByVal e As ErrorsDescriber = Nothing) As Boolean
+                If Count > 0 Then
+                    If Not ConvertToContainer(_Container, CookieList, EDP.ReturnValue) Then CookieList.RemoveAt(CookieList.Count - 1) Else Return True
+                End If
+                Return MyBase.Update(e)
+            End Function
+        End Class
+        Friend Class Responser2 : Inherits Responser
+            Friend Sub New()
+                MyBase.New
+                ChangeCookieKeeper()
+            End Sub
+            Friend Sub New(ByVal f As SFile)
+                MyBase.New(f)
+                ChangeCookieKeeper()
+            End Sub
+            Private Sub ChangeCookieKeeper()
+                _Cookies = New CookieKeeper2
+                Reset()
+            End Sub
+        End Class
     End Module
 End Namespace

SCrawler/API/ThisVid/SiteSettings.vb

@@ -50,6 +50,14 @@ Namespace API.ThisVid
             UrlPatternUser = "https://thisvid.com/members/{0}/"
             ImageVideoContains = "https://thisvid.com/videos/"
         End Sub
+        Protected Overrides Sub ChangeResponserOnInit()
+            _Responser.DisposeIfReady(False)
+            _Responser = Nothing
+            _Responser = New Responser2 With {.DeclaredError = EDP.ThrowException}
+        End Sub
+        Friend Overrides Function GetCookieKeeperInstance() As CookieKeeper
+            Return New CookieKeeper2
+        End Function
 #End Region
 #Region "GetInstance"
         Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider

SCrawler/API/ThisVid/UserData.vb

@@ -403,6 +403,30 @@ Namespace API.ThisVid
         End Sub
 #End Region
 #Region "ReparseVideo"
+        Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
+            Dim rList As New List(Of Integer)
+            Try
+                If Not IsSubscription And ContentMissingExists Then
+                    Dim m As UserMedia
+                    ProgressPre.ChangeMax(_ContentList.Count)
+                    For i% = 0 To _ContentList.Count - 1
+                        ProgressPre.Perform()
+                        m = _ContentList(i)
+                        If m.State = UserMedia.States.Missing AndAlso Not m.URL_BASE.IsEmptyString Then
+                            ThrowAny(Token)
+                            m.URL = m.URL_BASE
+                            m.Type = UserMedia.Types.VideoPre
+                            _TempMediaList.ListAddValue(m, LNC)
+                            rList.Add(i)
+                        End If
+                    Next
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, "missing data downloading error")
+            Finally
+                ReparseMissing_ClearList(rList)
+            End Try
+        End Sub
         Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken)
             If IsSubscription Then
                 ReparseVideoSubscriptions(Token)
@@ -440,11 +464,12 @@ Namespace API.ThisVid
                                     If n.IsEmptyString Then n = u.Post.ID
                                     If n.IsEmptyString Then n = "VideoFile"
                                     u.File = $"{n}.mp4"
-                                    If u.URL.IsEmptyString OrElse (Not u.Post.ID.IsEmptyString AndAlso _TempPostsList.Contains(u.Post.ID)) Then
+                                    If Not u.State = UserMedia.States.Missing AndAlso
+                                       (u.URL.IsEmptyString OrElse (Not u.Post.ID.IsEmptyString AndAlso _TempPostsList.Contains(u.Post.ID))) Then
                                         _TempMediaList.RemoveAt(i)
                                     Else
                                         u.Type = UserMedia.Types.Video
-                                        _TempPostsList.Add(u.Post.ID)
+                                        _TempPostsList.ListAddValue(u.Post.ID, LNC)
                                         _TempMediaList(i) = u
                                     End If
                                     e.Dispose()

SCrawler/API/ThisVid/UserExchangeOptions.vb

@@ -35,7 +35,7 @@ Namespace API.ThisVid
             DifferentFolders = u.DifferentFolders
             MySettings = u.HOST.Source
         End Sub
-        Friend Overrides Sub Apply(ByRef u As IPSite)
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
             MyBase.Apply(u)
             With DirectCast(u, UserData)
                 .DownloadPublic = DownloadPublic

SCrawler/API/ThreadsNet/UserData.vb

@@ -55,7 +55,7 @@ Namespace API.ThreadsNet
             Return New EditorExchangeOptionsBase(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
-            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptionsBase Then DirectCast(Obj, EditorExchangeOptionsBase).ApplyBase(Me)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptionsBase Then DirectCast(Obj, EditorExchangeOptionsBase).Apply(Me)
         End Sub
 #End Region
 #Region "Initializer"
@@ -184,7 +184,6 @@ Namespace API.ThreadsNet
             If uex.UserNotFound Then
                 UserExists = False
                 _ForceSaveUserInfo = True
-                _ForceSaveUserInfoOnException = True
             ElseIf ThrowEx Then
                 Throw New ExitException(uex.ErrMessage) With {.SimpleLogLine = True}
             Else
@@ -404,7 +403,6 @@ Namespace API.ThreadsNet
                     If _IdChanged Then
                         If Not idStr.IsEmptyString Then UserDescriptionUpdate(idStr, True, True, True)
                         _ForceSaveUserInfo = True
-                        _ForceSaveUserInfoOnException = True
                     End If
                 End If
                 Return Valid

SCrawler/API/TikTok/UserData.vb

@@ -80,20 +80,7 @@ Namespace API.TikTok
             Return New UserExchangeOptions(Me)
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
-            If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then DirectCast(Obj, UserExchangeOptions).Apply(Me)
-                With DirectCast(Obj, UserExchangeOptions)
-                    .ApplyBase(Me)
-                    GetTimeline = .GetTimeline
-                    GetStoriesUser = .GetStoriesUser
-                    GetReposts = .GetReposts
-                    RemoveTagsFromTitle = .RemoveTagsFromTitle
-                    TitleUseNative = .TitleUseNative
-                    TitleAddVideoID = .TitleAddVideoID
-                    TitleUseRegexForTitle = .TitleUseRegexForTitle
-                    TitleUseRegexForTitle_Value = .TitleUseRegexForTitle_Value
-                    TitleUseGlobalRegexOptions = .TitleUseGlobalRegexOptions
-                End With
-            End If
         End Sub
 #End Region
 #Region "Loader"
@@ -192,10 +179,14 @@ Namespace API.TikTok
             UserCache.DisposeIfReady(False)
             UserCache = Nothing
         End Sub
+        Private StoryStatus As Boolean = True
+        Private StoryStatusFound As Boolean = False
         Protected Overloads Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             ValidateCache()
+            StoryStatus = True
+            StoryStatusFound = False
             If GetTimeline Then DownloadDataF(Sections.Timeline, Token)
-            If GetStoriesUser Then DownloadDataF(Sections.UserStories, Token)
+            If GetStoriesUser And StoryStatus Then DownloadDataF(Sections.UserStories, Token)
             If GetReposts Then DownloadDataF(Sections.Reposts, Token)
         End Sub
         Protected Overloads Sub DownloadDataF(ByVal Section As Sections, ByVal Token As CancellationToken)
@@ -271,8 +262,8 @@ Namespace API.TikTok
                             Else
                                 .TempPostsList = New List(Of String)
                             End If
-                            .Execute(CreateGDLCommand(URL, gdlCmd))
+                            .Execute(CreateGDLCommand(URL, gdlCmd,,,, CObj(IIf(Section = Sections.Timeline, dateAfter, Nothing))))
-                            If Not PhotosDownloaded Then _ForceSaveUserInfo = True : _ForceSaveUserInfoOnException = True
+                            If Not PhotosDownloaded Then _ForceSaveUserInfo = True
                             PhotosDownloaded = True
                         End With
                     End Using
@@ -436,6 +427,10 @@ Namespace API.TikTok
                                             End If
                                         End With
                                     Else
+                                        If Section = Sections.Timeline And Not StoryStatusFound And j.ContainsF({0, "webapp.user-detail", "userInfo", "user"}, "UserStoryStatus") Then
+                                            StoryStatusFound = True
+                                            StoryStatus = j.ItemF({0, "webapp.user-detail", "userInfo", "user"}, "UserStoryStatus").Value.FromXML(Of Boolean)(False)
+                                        End If
                                         With j.ItemF({0, "webapp.video-detail", "itemInfo", "itemStruct"})
                                             If .ListExists Then
                                                 postID = .Value("id")
@@ -604,13 +599,17 @@ Namespace API.TikTok
 #End Region
 #Region "GDL Support"
         Private Function CreateGDLCommand(ByVal URL As String, Optional ByVal SectionCommand As String = Nothing,
-                                          Optional ByVal IsDownload As Boolean = False, Optional ByVal Output As SFile = Nothing) As String
+                                          Optional ByVal IsDownload As Boolean = False, Optional ByVal Output As SFile = Nothing,
+                                          Optional ByVal DateBefore As Date? = Nothing, Optional ByVal DateAfter As Date? = Nothing) As String
             Dim command$ = $"""{Settings.GalleryDLFile}"" "
             If Not IsDownload Then
                 command &= "--verbose --no-download --no-skip --write-pages "
             Else
                 command &= $"--dest ""{Output.PathNoSeparator}"" "
             End If
+            If DateBefore.HasValue Or DateAfter.HasValue Then
+                With If(DateAfter, DateBefore).Value : command &= $"--filter ""date {IIf(DateAfter.HasValue, ">", "<")} datetime({ .Year}, { .Month}, { .Day}) or terminate()"" " : End With
+            End If
             If Not CBool(If(IsSingleObjectDownload, MySettings.UseParsedVideoDateSTD, MySettings.UseParsedVideoDate).Value) Then command &= "--no-mtime "
             command &= $"{SectionCommand} {URL}"
             Return command

SCrawler/API/TikTok/UserExchangeOptions.vb

@@ -6,10 +6,11 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
 Namespace API.TikTok
-    Friend Class UserExchangeOptions : Inherits Base.EditorExchangeOptionsBase
+    Friend Class UserExchangeOptions : Inherits EditorExchangeOptionsBase
         <PSetting(NameOf(SiteSettings.GetTimeline), NameOf(MySettings))>
         Friend Property GetTimeline As Boolean
         <PSetting(NameOf(SiteSettings.GetStoriesUser), NameOf(MySettings))>
@@ -57,5 +58,19 @@ Namespace API.TikTok
             TitleUseRegexForTitle = s.TitleUseRegexForTitle.Value
             TitleUseRegexForTitle_Value = s.TitleUseRegexForTitle_Value.Value
         End Sub
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
+            MyBase.Apply(u)
+            With DirectCast(u, UserData)
+                .GetTimeline = GetTimeline
+                .GetStoriesUser = GetStoriesUser
+                .GetReposts = GetReposts
+                .RemoveTagsFromTitle = RemoveTagsFromTitle
+                .TitleUseNative = TitleUseNative
+                .TitleAddVideoID = TitleAddVideoID
+                .TitleUseRegexForTitle = TitleUseRegexForTitle
+                .TitleUseRegexForTitle_Value = TitleUseRegexForTitle_Value
+                .TitleUseGlobalRegexOptions = TitleUseGlobalRegexOptions
+            End With
+        End Sub
     End Class
 End Namespace

SCrawler/API/Twitter/Declarations.vb

@@ -17,6 +17,10 @@ Namespace API.Twitter
         Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue)
         Friend ReadOnly StatusRegEx As RParams = RParams.DM(".*?(twitter|x)\.com/\S+/status/\d+", 0, EDP.ReturnValue)
         Friend ReadOnly BroadcastsUrls As Object() = {"entities", "urls", 0, "expanded_url"}
+        Friend ReadOnly RichTextNode As New List(Of String()) From {
+            {{"content", "itemContent", "tweet_results", "result", "note_tweet", "note_tweet_results", "result", "text"}},
+            {{"item", "itemContent", "tweet_results", "result", "note_tweet", "note_tweet_results", "result", "text"}}
+        }
         Friend ReadOnly GdlLimitRegEx As RParams = RParams.DM("Waiting until[\s\W\d\:]+\(rate limit\)", 0, RegexOptions.IgnoreCase, EDP.ReturnValue)
         Friend ReadOnly GdlPostCoutNumberNodes As String() = {"data", "user", "result", "legacy", "statuses_count"}
         Private Function GetDateProvider() As ADateTime

SCrawler/API/Twitter/EditorExchangeOptions.vb

@@ -66,6 +66,7 @@ Namespace API.Twitter
             MySettings = s
         End Sub
         Friend Sub New(ByVal s As Mastodon.SiteSettings)
+            MyBase.New(s)
             GifsDownload = s.GifsDownload.Value
             GifsSpecialFolder = s.GifsSpecialFolder.Value
             GifsPrefix = s.GifsPrefix.Value

SCrawler/API/Twitter/UserData.vb

@@ -123,7 +123,7 @@ Namespace API.Twitter
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
             If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then
                 With DirectCast(Obj, EditorExchangeOptions)
-                    .ApplyBase(Me)
+                    .Apply(Me)
                     GifsDownload = .GifsDownload
                     GifsSpecialFolder = .GifsSpecialFolder
                     GifsPrefix = .GifsPrefix
@@ -297,10 +297,17 @@ Namespace API.Twitter
         Private Const DEBUG_PROFILE As Boolean = False
         Private Const DEBUG_LEAVE_CACHE As Boolean = False
         Private JsonNullErr As Boolean = False
+        Private ____UserExists As Boolean = True
+        Private NotUserExistsAttempts As Integer = 0
+        Friend Overrides Sub DownloadData(ByVal Token As CancellationToken)
+            ____UserExists = UserExists
+            MyBase.DownloadData(Token)
+        End Sub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
                 GDL_REQUESTS_COUNT = 0
                 JsonNullErr = False
+                NotUserExistsAttempts = 0
                 If MySettings.LIMIT_ABORT Then
                     Throw New TwitterLimitException(Me)
                 Else
@@ -317,6 +324,14 @@ Namespace API.Twitter
                         End If
                         LikesPosts.Clear()
                         If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
+                        If Not ____UserExists Then
+                            For i% = 0 To 1
+                                NotUserExistsAttempts += 1
+                                DownloadData_Timeline(Token)
+                                If UserExists Then ____UserExists = True : Exit For
+                            Next
+                        End If
+                        If Not UserExists Then Exit Sub
                         DownloadData_Timeline(Token)
                         If _TempMediaList.Count = 0 And LikesPosts.Count = 0 And JsonNullErr Then Throw New Plugin.ExitException("No deserialized data found")
                         LoadSavePostsKV(False)
@@ -366,6 +381,7 @@ Namespace API.Twitter
                 Dim j As EContainer, rootNode As EContainer, optionalNode As EContainer, workingNode As EContainer, tmpNode As EContainer, nn As EContainer = Nothing
                 Dim multiMode As Boolean = IsMultiMode
                 Dim currentModel As DownloadModels = DownloadModels.Undefined
+                Dim onlyUpdateUser As Boolean = Not ____UserExists
 
                 Dim __parseContainer As Func(Of EContainer, Boolean) =
                     Function(ByVal ee As EContainer) As Boolean
@@ -426,7 +442,7 @@ Namespace API.Twitter
                                         End If
                                     End If
                                 Else
-                                    ObtainMedia(nn, PostID, PostDate)
+                                    ObtainMedia(nn, PostID, PostDate,,,, ee)
                                 End If
                             End If
                         End If
@@ -478,6 +494,7 @@ Namespace API.Twitter
                                 End If
                                 'parse files
                                 For i = 0 To timelineFiles.Count - 1
+                                    If userInfoParsed And onlyUpdateUser Then Exit Sub
                                     j = JsonDocument.Parse(timelineFiles(i).GetText, jsonArgs)
                                     If jsonArgs.State = WebDocumentEventArgs.States.Error Then
                                         jsonArgs.Reset(Token)
@@ -506,7 +523,7 @@ Namespace API.Twitter
                                                                     With .ItemF({0, "content", "items", 0, "item", "itemContent", "tweet_results", "result", "tweet", "community_results", "result"})
                                                                         If .ListExists Then
                                                                             If ID = .Value("id_str") Then
-                                                                                UserSiteNameUpdate(.Value("name"))
+                                                                                UserSiteNameUpdate(.Value("name"), True)
                                                                                 UserDescriptionUpdate(.Value("description"))
 
                                                                                 icon = .Value({"custom_banner_media", "media_info"}, "original_img_url").
@@ -526,12 +543,20 @@ Namespace API.Twitter
                                                         If .ListExists Then
                                                             If ID.IsEmptyString Then ID = .Value("rest_id")
                                                             icon = .Value({"avatar"}, "image_url")
-                                                            UserSiteNameUpdate(.Value({"core"}, "name"))
+                                                            UserSiteNameUpdate(.Value({"core"}, "name"), True)
                                                             Dim tScreenName$ = .Value({"core"}, "screen_name")
                                                             With .Item({"legacy"})
                                                                 If .ListExists Then
+                                                                    If onlyUpdateUser Then
+                                                                        If Not NameTrue = tScreenName Then
+                                                                            Dim uStr$ = $"username changed from '{NameTrue}' to '{tScreenName}'"
+                                                                            LogError(Nothing, uStr)
+                                                                            UserDescriptionUpdate(uStr, True, True, True)
+                                                                        End If
+                                                                        NameTrue = tScreenName
+                                                                    End If
                                                                     If .Value("screen_name").IfNullOrEmpty(tScreenName).StringToLower = NameTrue.ToLower Then
-                                                                        UserSiteNameUpdate(.Value("name"))
+                                                                        UserSiteNameUpdate(.Value("name"), True)
                                                                         UserDescriptionUpdate(.Value("description"))
 
                                                                         If icon.IsEmptyString Then icon = .Value("profile_image_url_https")
@@ -692,7 +717,7 @@ nextpIndx:
                                             With jj(node)
                                                 If .ListExists Then
                                                     id = .Value("id_str")
-                                                    If _TempPostsList.Contains(id) Then j.Dispose() : Exit Sub Else ObtainMedia(.Self, id, .Value("created_at"))
+                                                    If _TempPostsList.Contains(id) Then j.Dispose() : Exit Sub Else ObtainMedia(.Self, id, .Value("created_at"),,,, jj)
                                                     Exit For
                                                 End If
                                             End With
@@ -713,11 +738,27 @@ nextpIndx:
 #End Region
 #Region "Obtain media"
         Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown,
-                                Optional ByVal Attempts As Integer = 0, Optional ByVal SpecialFolder As String = Nothing)
+                                Optional ByVal Attempts As Integer = 0, Optional ByVal SpecialFolder As String = Nothing,
+                                Optional ByVal RootElem As EContainer = Nothing)
             Dim s As EContainer = e({"extended_entities", "media"})
             If If(s?.Count, 0) = 0 Then s = e({"retweeted_status", "extended_entities", "media"})
             If If(s?.Count, 0) = 0 Then s = e({"retweeted_status_result", "result", "legacy", "extended_entities", "media"})
-            Dim txt$ = If(DownloadText, e.Value("full_text"), String.Empty)
+            Dim txt$ = String.Empty
+
+            If DownloadText Then
+                txt = e.Value("full_text")
+                If RootElem.ListExists Then
+                    For Each n As String() In RichTextNode
+                        With RootElem(n)
+                            If Not .Self Is Nothing AndAlso Not .Value.IsEmptyString Then
+                                If Not txt.IsEmptyString AndAlso .Value.Contains(txt) Then txt = String.Empty
+                                If txt.IsEmptyString Then txt = .Value Else txt = $"{ .Value}{vbCr}{vbCr}----{vbCr}{vbCr}{txt}"
+                                Exit For
+                            End If
+                        End With
+                    Next
+                End If
+            End If
 
             Dim mUrl$
             Dim media As UserMedia
@@ -846,6 +887,23 @@ nextpIndx:
         Private Class TwitterGDL : Inherits GDL.GDLBatch
             Private ReadOnly KillOnLimit As Boolean
             Friend LimitReached As Boolean = False
+            Private _GetOnlyUserInfo As Boolean = False
+            Friend Overrides Property MyWorkingDirectory As SFile
+                Get
+                    Return If(MyBase.MyWorkingDirectory.IsEmptyString, If(FileExchanger?.RootDirectory, MyBase.MyWorkingDirectory), MyBase.MyWorkingDirectory)
+                End Get
+                Set(ByVal dir As SFile)
+                    MyBase.MyWorkingDirectory = dir
+                End Set
+            End Property
+            Friend Property GetOnlyUserInfo As Boolean
+                Get
+                    Return _GetOnlyUserInfo And Not MyWorkingDirectory.IsEmptyString
+                End Get
+                Set(ByVal __GetOnlyUserInfo As Boolean)
+                    _GetOnlyUserInfo = __GetOnlyUserInfo
+                End Set
+            End Property
             Friend Sub New(ByVal Dir As SFile, ByVal _Token As CancellationToken, ByVal _KillOnLimit As Boolean)
                 MyBase.New(_Token,, Dir)
                 KillOnLimit = _KillOnLimit
@@ -855,11 +913,15 @@ nextpIndx:
             End Function
             Private Function IdExists(ByVal Value As String) As Boolean
                 Try
+                    If GetOnlyUserInfo Then
+                        Return CheckForData()
+                    Else
                         Value = Value.StringTrim
                         If Not Value.IsEmptyString AndAlso (Value.StartsWith("*") Or Value.StartsWith(".\gallery-dl\")) Then
                             Dim id$ = Value.Split("\").Last.Split(".").First.Split("_").First
                             If Not id.IsEmptyString Then Return TempPostsList.Contains(id)
                         End If
+                    End If
                 Catch ex As Exception
                 End Try
                 Return False
@@ -867,8 +929,14 @@ nextpIndx:
             Protected Overrides Async Sub ErrorDataReceiver(ByVal Sender As Object, ByVal e As DataReceivedEventArgs)
                 Await Task.Run(Sub() CheckForLimit(e.Data))
             End Sub
+            Private Function CheckForData()
+                If GetOnlyUserInfo Then
+                    If SFile.GetFiles(MyWorkingDirectory, "*.txt",, EDP.ReturnValue).Count > 2 Then Return True
+                End If
+                Return False
+            End Function
             Private Sub CheckForLimit(ByVal Value As String)
-                If Token.IsCancellationRequested Or (KillOnLimit AndAlso Not ProcessKilled AndAlso
+                If CheckForData() Or Token.IsCancellationRequested Or (KillOnLimit AndAlso Not ProcessKilled AndAlso
                    Not Value.IsEmptyString AndAlso (Value.ToLower.Contains("for rate limit reset") OrElse
                                                     Not CStr(RegexReplace(Value, GdlLimitRegEx)).IsEmptyString)) Then
                     LimitReached = True
@@ -1017,11 +1085,12 @@ nextpIndx:
                     .AutoClear = True,
                     .AutoReset = True,
                     .CommandPermanent = $"chcp {BatchExecutor.UnicodeEncoding}",
-                    .FileExchanger = confCache
+                    .FileExchanger = confCache,
+                    .GetOnlyUserInfo = NotUserExistsAttempts > 0
                 }
                     tgdl.FileExchanger.DeleteCacheOnDispose = False
                     tgdl.FileExchanger.DeleteRootOnDispose = False
-                    For i As Byte = 0 To IIf(IsCommunity, 0, 3)
+                    For i As Byte = 0 To IIf(IsCommunity Or NotUserExistsAttempts > 0, 0, 3)
                         dir = rootDir.NewPath
                         dir.Exists(SFO.Path, True, EDP.ThrowException)
                         outList.Add(dir)
@@ -1032,6 +1101,20 @@ nextpIndx:
                         Else
                             command &= GdlGetIdFilterString()
                         End If
+                        If NotUserExistsAttempts > 0 Then
+                            Select Case NotUserExistsAttempts
+                                Case 1 : command &= $"{urlPrePattern}{NameTrue}/media" : currentModel = DownloadModels.Media : process = True
+                                Case 2
+                                    If ID.IsEmptyString Then
+                                        process = False
+                                    Else
+                                        command &= $"https://twitter.com/intent/user?user_id={ID}"
+                                        currentModel = DownloadModels.Media
+                                        process = True
+                                    End If
+                                Case Else : process = False
+                            End Select
+                        Else
                             Select Case i
                                 Case 0 : command &= $"{urlPrePattern}{NameTrue}/media" : currentModel = DownloadModels.Media : process = dm.Contains(currentModel) Or IsCommunity
                                 Case 1 : command &= $"{urlPrePattern}{NameTrue}" : currentModel = DownloadModels.Profile : process = dm.Contains(currentModel)
@@ -1039,6 +1122,7 @@ nextpIndx:
                                 Case 3 : command &= $"{urlPrePattern}{NameTrue}/likes" : currentModel = DownloadModels.Likes : process = dm.Contains(currentModel)
                                 Case Else : process = False
                             End Select
+                        End If
                         '#If DEBUG Then
                         'Debug.WriteLine(command)
                         '#End If
@@ -1122,10 +1206,12 @@ nextpIndx:
             Dim cache As CacheKeeper = Nothing
             Try
                 If ContentMissingExists Or (_ReparseLikes And LikesPosts.Count > 0) Then
+                    Const __entries$ = "entries"
                     Dim m As UserMedia, mTmp As UserMedia
                     Dim PostDate$
                     Dim nodes As List(Of String()) = GetContainerSubnodes()
                     Dim node$()
+                    Dim entriesNode As Predicate(Of EContainer) = Function(ee) ee.Contains(__entries)
                     Dim j As EContainer, n As EContainer
                     Dim f As SFile
                     Dim i%, ii%
@@ -1161,7 +1247,7 @@ nextpIndx:
                                             f = GDLRenameFile(files(ii), ii)
                                             j = JsonDocument.Parse(f.GetText)
                                             If Not j Is Nothing Then
-                                                With j.ItemF({"data", 0, "instructions", 0, "entries"})
+                                                With j.ItemF({"data", 0, "instructions", entriesNode, __entries})
                                                     If .ListExists Then
                                                         If IsSingleObjectDownload Or DownloadBroadcasts Then
                                                             mTmp = ExtractBroadcast(.Self, m.Post.ID, String.Empty, nodes)
@@ -1176,7 +1262,7 @@ nextpIndx:
                                                                     If .ListExists Then
                                                                         PostDate = String.Empty
                                                                         If .Contains("created_at") Then PostDate = .Value("created_at") Else PostDate = String.Empty
-                                                                        ObtainMedia(.Self, m.Post.ID, PostDate, UStates.Missing, m.Attempts, specFolder)
+                                                                        ObtainMedia(.Self, m.Post.ID, PostDate, UStates.Missing, m.Attempts, specFolder, n)
                                                                         rList.ListAddValue(i, LNC)
                                                                     End If
                                                                 End With

SCrawler/API/Xhamster/SiteSettings.vb

@@ -109,7 +109,7 @@ Namespace API.Xhamster
         End Function
         Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
             With DirectCast(User, UserData)
-                If Not .SiteMode = SiteModes.User Then
+                If Not .SiteMode = SiteModes.User OrElse .IsCreator Then
                     Return .GetNonUserUrl(0)
                 Else
                     Return String.Format(UrlPatternUser, IIf(.IsChannel, ChannelOption, UserOption), .NameTrue)

SCrawler/API/Xhamster/UserData.vb

@@ -53,7 +53,7 @@ Namespace API.Xhamster
         End Property
         Friend Property QueryString As String Implements IPSite.QueryString
             Get
-                If SiteMode = SiteModes.User Then
+                If SiteMode = SiteModes.User And Not IsCreator Then
                     Return String.Empty
                 Else
                     Return GetNonUserUrl(0)
@@ -73,7 +73,7 @@ Namespace API.Xhamster
         End Structure
         Private ReadOnly _TempPhotoData As List(Of UserMedia)
         Private Function UpdateUserOptions(Optional ByVal Force As Boolean = False, Optional ByVal NewUrl As String = Nothing) As Boolean
-            If Not Force OrElse (Not SiteMode = SiteModes.User AndAlso Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
+            If Not Force OrElse (Not NewUrl.IsEmptyString AndAlso MyFileSettings.Exists) Then
                 Dim eObj As Plugin.ExchangeOptions = Nothing
                 If Force Then eObj = MySettings.IsMyUser(NewUrl)
                 If (Force And Not eObj.UserName.IsEmptyString) Or (Not Force And NameTrue(True).IsEmptyString) Then
@@ -193,7 +193,7 @@ Namespace API.Xhamster
         End Sub
 #End Region
 #Region "Download functions"
-        Friend Function GetNonUserUrl(ByVal Page As Integer) As String
+        Friend Function GetNonUserUrl(ByVal Page As Integer, Optional ByVal GetShorts As Boolean = False) As String
             Const newest$ = "/newest"
             If SiteMode = SiteModes.User And Not IsCreator Then
                 Return String.Empty
@@ -210,8 +210,10 @@ Namespace API.Xhamster
                 End Select
                 url &= $"/{NameTrue}"
 
+                If GetShorts And (SiteMode = SiteModes.User Or IsCreator) Then url &= "/shorts"
+
                 Dim args$ = Arguments
-                If (args.IsEmptyString OrElse Not args.Contains(newest)) And Not SiteMode = SiteModes.Search Then url &= newest
+                If (args.IsEmptyString OrElse Not args.Contains(newest)) And Not SiteMode = SiteModes.Search And Not IsCreator Then url &= newest
                 If Page > 1 Then
                     If args.IsEmptyString Then
                         If SiteMode = SiteModes.Search Then
@@ -288,7 +290,10 @@ Namespace API.Xhamster
                     End If
                 ElseIf Not SiteMode = SiteModes.Search Then
                     If IsVideo Then
-                        If GetMoments Then containerNodes.Add({"momentListComponent", "videoThumbProps"})
+                        If GetMoments Then
+                            containerNodes.Add({"momentListComponent", "videoThumbProps"})
+                            containerNodes.Add({"momentsComponent", "videoListProps", "videoThumbProps"})
+                        End If
                         containerNodes.Add({"trendingVideoListComponent", "models"})
                         containerNodes.Add({"pagesCategoryComponent", "trendingVideoListProps", "models"})
                         containerNodes.Add({"trendingVideoSectionComponent", "videoModels"})
@@ -298,6 +303,7 @@ Namespace API.Xhamster
                         containerNodes.Add({"videoListComponent", "videoThumbProps"})
                     Else
                         containerNodes.Add({"userGalleriesCollection"})
+                        containerNodes.Add({"contentComponent", "items"})
                     End If
                 End If
 
@@ -310,7 +316,7 @@ Namespace API.Xhamster
                     URL = GetNonUserUrl(Page)
                     containerNodes.Add({"searchResult", "models"})
                 ElseIf IsCreator Or SiteMode = SiteModes.Tags Or SiteMode = SiteModes.Categories Or SiteMode = SiteModes.Pornstars Then
-                    URL = GetNonUserUrl(Page)
+                    URL = GetNonUserUrl(Page, IsCreator And GetMoments)
                 Else
                     URL = $"https://xhamster.com/{SiteSettings.UserOption}/{NameTrue}/{If(GetMoments, "moments", IIf(IsVideo, "videos", "photos"))}{IIf(Page = 1, String.Empty, $"/{Page}")}"
                 End If
@@ -744,12 +750,13 @@ Namespace API.Xhamster
                 If IsSingleObjectDownload Then
                     cc = Settings.Cache
                 Else
-                    If MyCache Is Nothing Then MyCache = CreateCache() : MyCache.Validate()
+                    If MyCache Is Nothing Then MyCache = CreateCache()
                     cc = MyCache
                 End If
+                cc.Validate()
                 Dim path As SFile = cc.NewPath
                 Dim c$ = If(MySettings.CookiesNetscapeFile.Exists, $" --no-cookies-from-browser --cookies ""{MySettings.CookiesNetscapeFile}""", String.Empty)
-                Dim cmd$ = $"{Settings.YtdlpFile} --write-info-json --skip-download{c} {URL} -o ""{path.PathWithSeparator}file"""
+                Dim cmd$ = $"""{Settings.YtdlpFile}"" --write-info-json --skip-download{c} {URL} -o ""{path.PathWithSeparator}file"""
                 path.Exists()
                 Using ytdlp As New YTDLP.YTDLPBatch(TokenPersonal,, path) : ytdlp.Encoding = Settings.CMDEncoding : ytdlp.Execute(cmd) : End Using
                 Return SFile.GetFiles(path, "*.json",, EDP.ReturnValue).FirstOrDefault
@@ -794,10 +801,13 @@ Namespace API.Xhamster
         Private Function YTDLPDownload(ByVal Media As UserMedia, ByVal DestinationFile As SFile, ByVal Token As CancellationToken) As SFile
             DestinationFile.Extension = "mp4"
             Dim c$ = If(MySettings.CookiesNetscapeFile.Exists, $" --no-cookies-from-browser --cookies ""{MySettings.CookiesNetscapeFile}""", String.Empty)
-            Dim cmd$ = $"{Settings.YtdlpFile} --format {DirectCast(Media.Object, XMMediaInfo).FormatID}{c} {Media.URL_BASE} -o ""{DestinationFile}"""
+            Dim cmd$ = $"""{Settings.YtdlpFile}"" --format {DirectCast(Media.Object, XMMediaInfo).FormatID}{c} {Media.URL_BASE} -o ""{DestinationFile}"""
             Using ytdlp As New YTDLP.YTDLPBatch(TokenPersonal,, DestinationFile) : ytdlp.Encoding = Settings.CMDEncoding : ytdlp.Execute(cmd) : End Using
             Return DestinationFile
         End Function
+        Protected Overrides Function DownloadContentDefault_ConvertWebp(ByVal m As UserMedia, ByVal Process As Boolean) As SFile
+            Return DownloadContentDefault_ConvertWebp_TestImg(m, Process)
+        End Function
 #End Region
 #Region "Create media"
         Private Function ExtractMedia(ByVal j As EContainer, ByVal t As UTypes, Optional ByVal UrlNode As String = "pageURL",

SCrawler/API/Xhamster/UserExchangeOptions.vb

@@ -13,7 +13,6 @@ Namespace API.Xhamster
         <PSetting(Address:=SettingAddress.User, Caption:=SiteSettings.GetMomentsCaption)>
         Friend Property GetMoments As Boolean = False
         Friend Sub New()
-            MyBase.New
         End Sub
         Friend Sub New(ByVal u As IPSite)
             MyBase.New(DirectCast(u, UserData))
@@ -23,7 +22,7 @@ Namespace API.Xhamster
             MyBase.New(s)
             GetMoments = s.GetMoments.Value
         End Sub
-        Friend Overrides Sub Apply(ByRef u As IPSite)
+        Friend Overrides Sub Apply(ByRef u As UserDataBase)
             MyBase.Apply(u)
             DirectCast(u, UserData).GetMoments = GetMoments
         End Sub

SCrawler/API/YouTube/UserData.vb

@@ -121,6 +121,7 @@ Namespace API.YouTube
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
             If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
                 With DirectCast(Obj, UserExchangeOptions)
+                    .Apply(Me)
                     DownloadYTVideos = .DownloadVideos
                     DownloadYTShorts = .DownloadShorts
                     DownloadYTPlaylists = .DownloadPlaylists

SCrawler/API/YouTube/UserExchangeOptions.vb

@@ -8,7 +8,7 @@
 ' but WITHOUT ANY WARRANTY
 Imports SCrawler.Plugin.Attributes
 Namespace API.YouTube
-    Friend Class UserExchangeOptions
+    Friend Class UserExchangeOptions : Inherits SCrawler.API.Base.EditorExchangeOptionsBase
         <PSetting(Caption:="Download videos")>
         Friend Property DownloadVideos As Boolean
         <PSetting(Caption:="Download shorts")>
@@ -23,7 +23,14 @@ Namespace API.YouTube
         Friend Property UseCookies As Boolean
         <PSetting(Caption:="Channel ID", Address:=SettingAddress.User)>
         Friend Property ChannelID As String
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property UserName As String
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadText As Boolean
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadTextPosts As Boolean
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadTextSpecialFolder As Boolean
         Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
+            _ApplyBase_Name = False
+            _ApplyBase_Text = False
             DownloadVideos = u.DownloadYTVideos
             DownloadShorts = u.DownloadYTShorts
             DownloadPlaylists = u.DownloadYTPlaylists
@@ -33,6 +40,9 @@ Namespace API.YouTube
             ChannelID = u.ChannelID
         End Sub
         Friend Sub New(ByVal s As SiteSettings)
+            MyBase.New(s)
+            _ApplyBase_Name = False
+            _ApplyBase_Text = False
             DownloadVideos = s.DownloadVideos.Value
             DownloadShorts = s.DownloadShorts.Value
             DownloadPlaylists = s.DownloadPlaylists.Value

SCrawler/Download/Feed/DownloadFeedForm.vb

@@ -26,7 +26,19 @@ Namespace DownloadObjects
         Friend WithEvents MyDefs As DefaultFormOptions
         Private WithEvents MyRange As RangeSwitcherToolbar(Of UserMediaD)
         Private ReadOnly DataList As List(Of UserMediaD)
+#Region "Buttons"
         Private WithEvents BTT_DELETE_SELECTED As ToolStripButton
+        Private WithEvents BTT_COPY_SELECTED_T As ToolStripButton
+        Private WithEvents BTT_MOVE_SELECTED_T As ToolStripButton
+        Private WithEvents BTT_FAV_T As ToolStripButton
+        Private WithEvents BTT_FEED_ADD_T As ToolStripDropDownButton
+        Private WithEvents BTT_SELECT_ALL_T As ToolStripButton
+        Private WithEvents BTT_SELECT_NONE_T As ToolStripButton
+        Private WithEvents BTT_INVERT_T As ToolStripButton
+        Private WithEvents BTT_LOAD_CURRENT_T As ToolStripButton
+        Private WithEvents BTT_LOAD_FAV_T As ToolStripButton
+        Private WithEvents BTT_LOAD_T As ToolStripDropDownButton
+#End Region
         Private DataRows As Integer = 10
         Private DataColumns As Integer = 1
         Private FeedEndless As Boolean = False
@@ -65,8 +77,10 @@ Namespace DownloadObjects
                 If Not CurrentFilter Is Nothing Then
                     With CurrentFilter
                         If .Types.Count > 0 AndAlso Not .Types.Any(Function(t) FilterTypesConversion(t, d)) Then Return False
-#Disable Warning BC42109
+#Disable Warning BC42109, BC42104
+                        If .Sites.Count > 0 AndAlso Not .Sites.Contains(d.UserInfo.Site) Then Return False
                         If .Users.Count > 0 AndAlso Not .Users.Contains(d.UserInfo) Then Return False
+                        If .Labels.Count > 0 AndAlso d.User.Labels.Count > 0 AndAlso Not d.User.Labels.ListContains(.Labels) Then Return False
 #Enable Warning
                     End With
                 End If
@@ -99,15 +113,15 @@ Namespace DownloadObjects
         End Sub
         Private Sub FeedRemoveCheckedMedia(ByVal MediaList As IEnumerable(Of UserMediaD), Optional ByVal OverriddenNames As List(Of String) = Nothing,
                                            Optional ByVal RemoveChecked As Boolean = True, Optional ByVal ExcludingNames As IEnumerable(Of String) = Nothing,
-                                           Optional ByVal RemoveFromDataListOnly As Boolean = False)
+                                           Optional ByVal IsAddAndRemove As Boolean = False)
             Try
-                If FeedMode = FeedModes.Special Then
+                If FeedMode = FeedModes.Saved Then Exit Sub
-                    If LoadedFeedNames.Count > 0 Then
+
                 Dim dataRemoved As Boolean = False
-                        If OverriddenNames.ListExists And Not LoadedFeedNames.ListContains(OverriddenNames) Then Exit Sub
+                If FeedMode = FeedModes.Special And OverriddenNames.ListExists And Not LoadedFeedNames.ListContains(OverriddenNames) Then Exit Sub
-                        If Not RemoveFromDataListOnly Then
                 Dim eNames As IEnumerable(Of String) = If(ExcludingNames, New String() {})
                 With If(OverriddenNames, LoadedFeedNames)
+                    If FeedMode = FeedModes.Special And .ListExists Then
                         .ForEach(Sub(ByVal feedName As String)
                                      If Not eNames.Contains(feedName) Then
                                          Dim indx% = Settings.Feeds.IndexOf(feedName)
@@ -116,25 +130,33 @@ Namespace DownloadObjects
                                          End If
                                      End If
                                  End Sub)
-                            End With
+                    ElseIf FeedMode = FeedModes.Current And Not OverriddenNames.ListExists And IsAddAndRemove Then
+                        dataRemoved = Downloader.Files.ListDisposeRemove(MediaList) > 0
+                        'Downloader.FilesSave()
+                    Else
+                        Exit Sub
                     End If
-                        If RemoveFromDataListOnly Then
+                End With
-                            RefillSpecialFeedsData()
+
-                        ElseIf dataRemoved Then
+                If dataRemoved Then DataList.ListDisposeRemove(MediaList)
-                            DataList.ListDisposeRemove(MediaList)
+
-                            If RemoveChecked Then
+                Select Case FeedMode
+                    Case FeedModes.Special
+                        If RemoveChecked And IsAddAndRemove Then
                             If RemoveCheckedMedia(False) Then RefillAfterDelete()
                         Else
                             RefillSpecialFeedsData()
                         End If
-                        End If
+                    Case FeedModes.Current
-                    End If
+                        If dataRemoved Then
-                ElseIf FeedMode = FeedModes.Current Then
+                            If RemoveChecked Then
-                    If OverriddenNames Is Nothing AndAlso Downloader.Files.ListDisposeRemove(MediaList) > 0 AndAlso RemoveCheckedMedia(False) Then
+                                RemoveCheckedMedia(False)
-                        DataList.ListDisposeRemove(MediaList)
                                 RefillAfterDelete()
+                            Else
+                                RefillList()
                             End If
                         End If
+                End Select
             Catch ex As Exception
                 ErrorsDescriber.Execute(EDP.SendToLog, ex, "[DownloadFeedForm.FeedRemoveCheckedMedia]")
             End Try
@@ -149,13 +171,84 @@ Namespace DownloadObjects
             MyRange = New RangeSwitcherToolbar(Of UserMediaD)(ToolbarTOP)
             DataList = New List(Of UserMediaD)
             LoadedFeedNames = New List(Of String)
+
             BTT_DELETE_SELECTED = New ToolStripButton With {
-                .Text = "Delete selected",
+                .Text = "Delete",
                 .AutoToolTip = True,
-                .ToolTipText = "Delete marked files",
+                .ToolTipText = "Delete selected files",
                 .Image = My.Resources.DeletePic_24,
                 .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
             }
+            BTT_COPY_SELECTED_T = New ToolStripButton With {
+                .Text = "Copy",
+                .AutoToolTip = True,
+                .ToolTipText = "Copy selected files",
+                .Image = My.Resources.PastePic_32,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+            BTT_MOVE_SELECTED_T = New ToolStripButton With {
+                .Text = "Move",
+                .AutoToolTip = True,
+                .ToolTipText = "Move selected files",
+                .Image = My.Resources.CutPic_48,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+
+            BTT_FAV_T = New ToolStripButton With {
+                .Text = "Favorite",
+                .AutoToolTip = True,
+                .ToolTipText = "Add checked to Favorite",
+                .Image = My.Resources.HeartPic_32,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+            BTT_FEED_ADD_T = New ToolStripDropDownButton With {
+                .Text = "Add",
+                .AutoToolTip = True,
+                .ToolTipText = "Add checked to special feed..." & vbCr & "Right click to add to multiple feeds",
+                .Image = My.Resources.RSSPic_512,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+
+            BTT_LOAD_CURRENT_T = New ToolStripButton With {
+                .Text = "Current",
+                .AutoToolTip = True,
+                .ToolTipText = "Load current feed",
+                .Image = My.Resources.ArrowDownPic_Blue_24,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+            BTT_LOAD_FAV_T = New ToolStripButton With {
+                .Text = "Favorite",
+                .AutoToolTip = True,
+                .ToolTipText = "Load Favorite",
+                .Image = My.Resources.HeartPic_32,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+            BTT_LOAD_T = New ToolStripDropDownButton With {
+                .Text = "Load",
+                .AutoToolTip = True,
+                .ToolTipText = "Load special feed..." & vbCr & "Right click to load multiple feeds",
+                .Image = My.Resources.RSSPic_512,
+                .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText
+            }
+
+            BTT_SELECT_ALL_T = New ToolStripButton With {
+                .Text = "All",
+                .AutoToolTip = True,
+                .ToolTipText = "Select all",
+                .DisplayStyle = ToolStripItemDisplayStyle.Text
+            }
+            BTT_SELECT_NONE_T = New ToolStripButton With {
+                .Text = "None",
+                .AutoToolTip = True,
+                .ToolTipText = "Select none",
+                .DisplayStyle = ToolStripItemDisplayStyle.Text
+            }
+            BTT_INVERT_T = New ToolStripButton With {
+                .Text = "Invert",
+                .AutoToolTip = True,
+                .ToolTipText = "Invert selection",
+                .DisplayStyle = ToolStripItemDisplayStyle.Text
+            }
             FILTERS = New FeedFilterCollection
             BTT_FILTER.Image = My.Resources.FilterPic
             BTT_FILTER_SIMPLE.Image = My.Resources.FilterPic
@@ -181,17 +274,29 @@ Namespace DownloadObjects
                     .ToolTip(RCI.GoTo) = "GoTo (Ctrl+G)"
                     .AddThisToolbar()
                 End With
-                ToolbarTOP.Items.AddRange({New ToolStripSeparator, BTT_DELETE_SELECTED})
+                ToolbarTOP.Items.AddRange({New ToolStripSeparator, BTT_DELETE_SELECTED,
+                                           New ToolStripSeparator, BTT_COPY_SELECTED_T, BTT_MOVE_SELECTED_T,
+                                           New ToolStripSeparator, BTT_FAV_T, BTT_FEED_ADD_T,
+                                           New ToolStripSeparator, BTT_LOAD_CURRENT_T, BTT_LOAD_FAV_T, BTT_LOAD_T,
+                                           New ToolStripSeparator, BTT_SELECT_ALL_T, BTT_SELECT_NONE_T, BTT_INVERT_T})
                 With Settings
                     With .Feeds
                         .Load()
                         AddHandler .FeedAdded, AddressOf Feed_FeedAdded
                         AddHandler .FeedRemoved, AddressOf Feed_FeedRemoved
+                        If .Count = 0 OrElse (.Count = 1 AndAlso .Item(0).IsFavorite) Then
+                            BTT_FEED_ADD_T.Enabled = False
+                            BTT_FEED_ADD_T.Visible = False
+                            BTT_LOAD_T.Enabled = False
+                            BTT_LOAD_T.Visible = False
+                        End If
                         If .Count > 0 Then
                             For Each feed As FeedSpecial In .Self
                                 If Not feed.IsFavorite Then
                                     AddNewFeedItem(BTT_LOAD_SPEC, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_LOAD)
+                                    AddNewFeedItem(BTT_LOAD_T, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_LOAD)
                                     AddNewFeedItem(BTT_FEED_ADD_SPEC, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD)
+                                    AddNewFeedItem(BTT_FEED_ADD_T, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD)
                                     AddNewFeedItem(BTT_FEED_ADD_SPEC_REMOVE, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD_REMOVE)
                                     AddNewFeedItem(BTT_FEED_REMOVE_SPEC, feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_REMOVE)
                                     AddNewFeedItem(BTT_FEED_DELETE_SPEC, feed, My.Resources.DeletePic_24, AddressOf Feed_SPEC_DELETE)
@@ -260,11 +365,11 @@ Namespace DownloadObjects
         End Sub
 #End Region
 #Region "Feeds handlers"
-        Private Overloads Sub AddNewFeedItem(ByVal Destination As ToolStripMenuItem, ByVal Feed As FeedSpecial, ByVal Image As Image,
+        Private Overloads Sub AddNewFeedItem(Of T As ToolStripDropDownItem)(ByVal Destination As T, ByVal Feed As FeedSpecial, ByVal Image As Image,
                                                                             ByVal Handler As EventHandler, Optional ByVal Insert As Boolean = False)
             AddNewFeedItem(Destination, ToolbarTOP, Feed, Image, Handler, Insert)
         End Sub
-        Friend Overloads Shared Function AddNewFeedItem(ByVal Destination As ToolStripMenuItem, ByVal Toolbar As ToolStrip,
+        Friend Overloads Shared Function AddNewFeedItem(ByVal Destination As ToolStripDropDownItem, ByVal Toolbar As ToolStrip,
                                                         ByVal Feed As FeedSpecial, ByVal Image As Image,
                                                         ByVal Handler As EventHandler, Optional ByVal Insert As Boolean = False) As ToolStripMenuItem
             Dim item As New ToolStripMenuItem(Feed.Name, Image) With {.Tag = Feed}
@@ -281,6 +386,7 @@ Namespace DownloadObjects
         Private Sub Feed_FeedAdded(ByVal Source As FeedSpecialCollection, ByVal Feed As FeedSpecial)
             AddNewFeedItem(BTT_LOAD_SPEC, Feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_LOAD, True)
             AddNewFeedItem(BTT_FEED_ADD_SPEC, Feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD, True)
+            AddNewFeedItem(BTT_FEED_ADD_T, Feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD, True)
             AddNewFeedItem(BTT_FEED_ADD_SPEC_REMOVE, Feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_ADD_REMOVE, True)
             AddNewFeedItem(BTT_FEED_REMOVE_SPEC, Feed, My.Resources.RSSPic_512, AddressOf Feed_SPEC_REMOVE, True)
             AddNewFeedItem(BTT_FEED_DELETE_SPEC, Feed, My.Resources.DeletePic_24, AddressOf Feed_SPEC_DELETE, True)
@@ -289,14 +395,15 @@ Namespace DownloadObjects
         Private Overloads Sub Feed_FeedRemoved(ByVal Source As FeedSpecialCollection, ByVal Feed As FeedSpecial)
             Feed_FeedRemoved(BTT_LOAD_SPEC, Feed)
             Feed_FeedRemoved(BTT_FEED_ADD_SPEC, Feed)
+            Feed_FeedRemoved(BTT_FEED_ADD_T, Feed)
             Feed_FeedRemoved(BTT_FEED_REMOVE_SPEC, Feed)
             Feed_FeedRemoved(BTT_FEED_DELETE_SPEC, Feed)
             Feed_FeedRemoved(BTT_FEED_CLEAR_SPEC, Feed)
         End Sub
-        Private Overloads Sub Feed_FeedRemoved(ByVal Destination As ToolStripMenuItem, ByVal Feed As FeedSpecial)
+        Private Overloads Sub Feed_FeedRemoved(ByVal Destination As ToolStripDropDownItem, ByVal Feed As FeedSpecial)
             Feed_FeedRemoved(Destination, ToolbarTOP, Feed)
         End Sub
-        Friend Overloads Shared Sub Feed_FeedRemoved(ByVal Destination As ToolStripMenuItem, ByVal Toolbar As ToolStrip, ByVal Feed As FeedSpecial)
+        Friend Overloads Shared Sub Feed_FeedRemoved(ByVal Destination As ToolStripDropDownItem, ByVal Toolbar As ToolStrip, ByVal Feed As FeedSpecial)
             Try
                 With Destination
                     ControlInvokeFast(Toolbar, .Self,
@@ -333,7 +440,7 @@ Namespace DownloadObjects
                 Dim c As IEnumerable(Of UserMediaD) = GetCheckedMedia()
                 If c.ListExists Then
                     f.Add(c)
-                    FeedRemoveCheckedMedia(c,,, {f.Name})
+                    FeedRemoveCheckedMedia(c,,, {f.Name}, True)
                 End If
             End If
         End Sub
@@ -352,7 +459,7 @@ Namespace DownloadObjects
                 Dim m As IEnumerable(Of UserMediaD) = GetCheckedMedia()
                 If m.ListExists Then
                     f.Remove(m)
-                    FeedRemoveCheckedMedia(m, {f.Name}.ToList)
+                    FeedRemoveCheckedMedia(m, {f.Name}.ToList,,, False)
                 End If
             End If
         End Sub
@@ -471,7 +578,7 @@ Namespace DownloadObjects
 #Region "Toolbar controls"
 #Region "Feed"
 #Region "Load"
-        Private Sub BTT_LOAD_SESSION_CURRENT_Click(sender As Object, e As EventArgs) Handles BTT_LOAD_SESSION_CURRENT.Click
+        Private Sub BTT_LOAD_SESSION_CURRENT_Click(sender As Object, e As EventArgs) Handles BTT_LOAD_SESSION_CURRENT.Click, BTT_LOAD_CURRENT_T.Click
             FeedChangeMode(FeedModes.Current)
             RefillList(True, False)
         End Sub
@@ -586,7 +693,7 @@ Namespace DownloadObjects
         End Sub
 #End Region
 #Region "Move/Copy"
-        Private Sub BTT_COPY_MOVE_TO_Click(sender As Object, e As EventArgs) Handles BTT_COPY_TO.Click, BTT_MOVE_TO.Click
+        Private Sub BTT_COPY_MOVE_TO_Click(sender As Object, e As EventArgs) Handles BTT_COPY_TO.Click, BTT_MOVE_TO.Click, BTT_COPY_SELECTED_T.Click, BTT_MOVE_SELECTED_T.Click
             MoveCopyFiles(True, sender, Nothing, Nothing)
         End Sub
         Private Sub BTT_COPY_MOVE_SPEC_TO_Click(sender As Object, e As EventArgs) Handles BTT_COPY_SPEC_TO.Click, BTT_MOVE_SPEC_TO.Click
@@ -596,7 +703,7 @@ Namespace DownloadObjects
                                        ByVal FeedMediaData As FeedMedia, Optional ByVal GetChecked As Boolean = True) As Boolean
             Dim MsgTitle$ = "Copy/Move checked files"
             Try
-                Dim isCopy As Boolean = Not Sender Is Nothing AndAlso (Sender Is BTT_COPY_TO OrElse Sender Is BTT_COPY_SPEC_TO)
+                Dim isCopy As Boolean = Not Sender Is Nothing AndAlso (Sender Is BTT_COPY_TO OrElse Sender Is BTT_COPY_SPEC_TO OrElse Sender Is BTT_COPY_SELECTED_T)
                 Dim moveOptions As FeedMoveCopyTo = Nothing
                 Dim ff As SFile = Nothing, ffInit As SFile = Nothing, df As SFile = Nothing
                 Dim data As IEnumerable(Of UserMediaD) = Nothing
@@ -959,10 +1066,13 @@ Namespace DownloadObjects
         End Sub
 #End Region
 #Region "Load fav, spec"
-        Private Sub BTT_LOAD_FAV_Click(sender As Object, e As EventArgs) Handles BTT_LOAD_FAV.Click
+        Private Sub BTT_LOAD_FAV_Click(sender As Object, e As EventArgs) Handles BTT_LOAD_FAV.Click, BTT_LOAD_FAV_T.Click
             FeedChangeMode(FeedModes.Special, {FeedSpecial.FavoriteName})
             RefillSpecialFeedsData(False)
         End Sub
+        Private Sub BTT_LOAD_T_MouseDown(sender As Object, e As MouseEventArgs) Handles BTT_LOAD_T.MouseDown
+            If e.Button = MouseButtons.Right Then BTT_LOAD_SPEC_Click(sender, e)
+        End Sub
         Private Sub BTT_LOAD_SPEC_Click(sender As Object, e As EventArgs) Handles BTT_LOAD_SPEC.Click
             With FeedSpecialCollection.ChooseFeeds(False)
                 If .ListExists Then
@@ -996,20 +1106,23 @@ Namespace DownloadObjects
         End Sub
 #End Region
 #Region "Add remove fav spec"
-        Private Sub BTT_FEED_ADD_FAV_Click(sender As Object, e As EventArgs) Handles BTT_FEED_ADD_FAV.Click, BTT_FEED_ADD_FAV_REMOVE.Click
+        Private Sub BTT_FEED_ADD_FAV_Click(sender As Object, e As EventArgs) Handles BTT_FEED_ADD_FAV.Click, BTT_FEED_ADD_FAV_REMOVE.Click, BTT_FAV_T.Click
             Dim m As IEnumerable(Of UserMediaD) = GetCheckedMedia()
             If m.ListExists Then
                 Settings.Feeds.Favorite.Add(m)
-                If sender Is BTT_FEED_ADD_FAV_REMOVE Then FeedRemoveCheckedMedia(m,,, {FeedSpecial.FavoriteName})
+                If sender Is BTT_FEED_ADD_FAV_REMOVE Then FeedRemoveCheckedMedia(m,,, {FeedSpecial.FavoriteName}, True)
             End If
         End Sub
         Private Sub BTT_FEED_REMOVE_FAV_Click(sender As Object, e As EventArgs) Handles BTT_FEED_REMOVE_FAV.Click
             Dim m As IEnumerable(Of UserMediaD) = GetCheckedMedia()
             If m.ListExists Then
                 Settings.Feeds.Favorite.Remove(m)
-                If FeedMode = FeedModes.Special Then FeedRemoveCheckedMedia(m, {FeedSpecial.FavoriteName}.ToList)
+                If FeedMode = FeedModes.Special Then FeedRemoveCheckedMedia(m, {FeedSpecial.FavoriteName}.ToList,,, False)
             End If
         End Sub
+        Private Sub BTT_FEED_ADD_T_MouseDown(sender As Object, e As MouseEventArgs) Handles BTT_FEED_ADD_T.MouseDown
+            If e.Button = MouseButtons.Right Then BTT_FEED_ADD_SPEC_Click(sender, e)
+        End Sub
         Private Sub BTT_FEED_ADD_SPEC_Click(sender As Object, e As EventArgs) Handles BTT_FEED_ADD_SPEC.Click, BTT_FEED_ADD_SPEC_REMOVE.Click
             Dim c As IEnumerable(Of UserMediaD) = GetCheckedMedia()
             If c.ListExists Then
@@ -1020,7 +1133,7 @@ Namespace DownloadObjects
                                                      f.Add(c)
                                                  End Sub)
                 End With
-                If sender Is BTT_FEED_ADD_SPEC_REMOVE Then FeedRemoveCheckedMedia(c,,, names)
+                If sender Is BTT_FEED_ADD_SPEC_REMOVE Then FeedRemoveCheckedMedia(c,,, names, True)
                 names.Clear()
             Else
                 MsgBoxE({"You haven't selected media to add to your feed(s)", "Add to feed(s)"}, vbExclamation)
@@ -1036,7 +1149,7 @@ Namespace DownloadObjects
                                                      f.Remove(c)
                                                  End Sub)
                 End With
-                If FeedMode = FeedModes.Special Then FeedRemoveCheckedMedia(c, names)
+                If FeedMode = FeedModes.Special Then FeedRemoveCheckedMedia(c, names,,, False)
             Else
                 MsgBoxE({"You haven't selected media to remove from your feed(s)", "Remove from feed(s)"}, vbExclamation)
             End If
@@ -1268,10 +1381,10 @@ Namespace DownloadObjects
             End Try
         End Sub
 #End Region
-        Private Sub BTT_CHECK_ALL_NONE_Click(sender As Object, e As EventArgs) Handles BTT_CHECK_ALL.Click, BTT_CHECK_NONE.Click, BTT_CHECK_INVERT.Click
+        Private Sub BTT_CHECK_ALL_NONE_Click(sender As Object, e As EventArgs) Handles BTT_CHECK_ALL.Click, BTT_CHECK_NONE.Click, BTT_CHECK_INVERT.Click, BTT_SELECT_ALL_T.Click, BTT_SELECT_NONE_T.Click, BTT_INVERT_T.Click
             Try
-                Dim checked As Boolean = sender Is BTT_CHECK_ALL
+                Dim checked As Boolean = sender Is BTT_CHECK_ALL Or sender Is BTT_SELECT_ALL_T
-                Dim isInvert As Boolean = sender Is BTT_CHECK_INVERT
+                Dim isInvert As Boolean = sender Is BTT_CHECK_INVERT Or sender Is BTT_INVERT_T
                 ControlInvokeFast(TP_DATA, Sub()
                                                With TP_DATA
                                                    If .Controls.Count > 0 Then
@@ -1408,8 +1521,10 @@ Namespace DownloadObjects
                 ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "Download subscription media")
             End Try
         End Sub
-        Private Sub FeedMedia_FeedAddWithRemove(ByVal Sender As FeedMedia, ByVal Feeds As IEnumerable(Of String), ByVal Media As UserMediaD, ByVal RemoveOperation As Boolean)
+        Private Sub FeedMedia_FeedRemoveCheckedMedia(ByVal Sender As FeedMedia, ByVal Media As UserMediaD, ByVal Names As IEnumerable(Of String),
-            FeedRemoveCheckedMedia({Media},, False, Feeds, RemoveOperation)
+                                                     ByVal IsOverriddenNames As Boolean, ByVal IsAddAndRemove As Boolean)
+            FeedRemoveCheckedMedia({Media}, CObj(If(IsOverriddenNames, Names.ListIfNothing, Nothing)), False,
+                                   CObj(If(IsOverriddenNames, Nothing, Names)), IsAddAndRemove)
         End Sub
 #End Region
 #Region "Delete / Remove"
@@ -1616,7 +1731,7 @@ Namespace DownloadObjects
                                                   AddHandler .MediaDownload, AddressOf FeedMedia_Download
                                                   AddHandler .MediaMove, AddressOf FeedMedia_MediaMove
                                                   AddHandler .MediaCopy, AddressOf FeedMedia_MediaCopy
-                                                  AddHandler .FeedAddWithRemove, AddressOf FeedMedia_FeedAddWithRemove
+                                                  AddHandler .FeedRemoveCheckedMedia, AddressOf FeedMedia_FeedRemoveCheckedMedia
                                               End With
                                               If de.Data.Type = UTypes.Text OrElse de.Data.PostTextFile.IsEmptyString Then Exit For
                                           Next

SCrawler/Download/Feed/FeedFilter.vb

@@ -18,24 +18,28 @@ Namespace DownloadObjects
         Private Const Name_Types As String = "Types"
         Private Const Name_Users As String = "Users"
         Private Const Name_Sites As String = "Sites"
+        Private Const Name_Labels As String = "Labels"
 #End Region
 #Region "Properties"
         Friend Property Name As String
         Friend ReadOnly Property Types As List(Of UTypes)
         Friend ReadOnly Property Users As List(Of UserInfo)
         Friend ReadOnly Property Sites As List(Of String)
+        Friend ReadOnly Property Labels As List(Of String)
 #End Region
 #Region "Initializers"
         Friend Sub New()
             Types = New List(Of UTypes)
             Users = New List(Of UserInfo)
             Sites = New List(Of String)
+            Labels = New List(Of String)
         End Sub
         Friend Sub New(ByVal e As EContainer)
             Me.New
             Name = e.Value(Name_Name)
             Types.ListAddList(e.Value(Name_Types).StringToList(Of Integer)(","))
             Sites.ListAddList(e.Value(Name_Sites).StringToList(Of String)("|"))
+            Labels.ListAddList(e.Value(Name_Labels).StringToList(Of String)("|"))
             If If(e(Name_Users)?.Count, 0) > 0 Then
                 Users.ListAddList(e(Name_Users), LAP.IgnoreICopier)
                 Users.RemoveAll(Function(u) Not u.File.Exists)
@@ -75,8 +79,9 @@ Namespace DownloadObjects
             ee.Add(New EContainer(Name_Name, Name))
             ee.Add(New EContainer(Name_Types, Types.ListToString(",")))
             ee.Add(New EContainer(Name_Sites, Sites.ListToString("|")))
+            ee.Add(New EContainer(Name_Labels, Labels.ListToString("|")))
             If Users.Count > 0 Then
-                ee.Add(New EContainer(Name_Users, String.Empty))
+                ee.Add(New EContainer(Name_Users, String.Empty) With {.AllowSameNames = True})
                 ee(Name_Users).AddRange(Users)
             End If
             Return ee
@@ -95,6 +100,8 @@ Namespace DownloadObjects
                 Users.ListAddList(.Users, LAP.NotContainsOnly)
                 Sites.Clear()
                 Sites.ListAddList(.Sites, LAP.NotContainsOnly)
+                Labels.Clear()
+                Labels.ListAddList(.Labels, LAP.NotContainsOnly)
             End With
             Return Me
         End Function
@@ -126,8 +133,15 @@ Namespace DownloadObjects
             End Get
             Set(ByVal f As FeedFilter)
                 _TEMP = f
-                If _TEMP Is Nothing And _CurrentFilterName.IsEmptyString And Not _CurrentFilterNameLast.IsEmptyString Then _
+                If _TEMP Is Nothing Then
+                    If Not _CurrentFilterNameLast.IsEmptyString Then
                         _CurrentFilterName = _CurrentFilterNameLast
+                    Else
+                        _CurrentFilterName = String.Empty
+                    End If
+                Else
+                    _CurrentFilterName = String.Empty
+                End If
             End Set
         End Property
         Friend ReadOnly Property Current(Optional ByVal Any As Boolean = True, Optional ByVal GetTemp As Boolean = True) As FeedFilter

SCrawler/Download/Feed/FeedFilterForm.Designer.vb

@@ -30,6 +30,8 @@ Namespace DownloadObjects
             Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(FeedFilterForm))
             Dim ActionButton2 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Dim ActionButton3 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton4 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim ActionButton5 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
             Me.TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
             Me.CH_T_ALL = New System.Windows.Forms.CheckBox()
             Me.CH_T_IMG = New System.Windows.Forms.CheckBox()
@@ -43,6 +45,7 @@ Namespace DownloadObjects
             Me.LIST_USERS = New System.Windows.Forms.CheckedListBox()
             Me.TXT_NAME = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             Me.TXT_SITE = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            Me.TXT_LABELS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
             CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
             TP_TYPES = New System.Windows.Forms.TableLayoutPanel()
             TP_USERS = New System.Windows.Forms.TableLayoutPanel()
@@ -55,6 +58,7 @@ Namespace DownloadObjects
             TP_USERS_2.SuspendLayout()
             CType(Me.TXT_NAME, System.ComponentModel.ISupportInitialize).BeginInit()
             CType(Me.TXT_SITE, System.ComponentModel.ISupportInitialize).BeginInit()
+            CType(Me.TXT_LABELS, System.ComponentModel.ISupportInitialize).BeginInit()
             Me.SuspendLayout()
             '
             'CONTAINER_MAIN
@@ -78,16 +82,18 @@ Namespace DownloadObjects
             Me.TP_MAIN.ColumnCount = 1
             Me.TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
             Me.TP_MAIN.Controls.Add(TP_TYPES, 0, 1)
-            Me.TP_MAIN.Controls.Add(TP_USERS, 0, 3)
+            Me.TP_MAIN.Controls.Add(TP_USERS, 0, 4)
             Me.TP_MAIN.Controls.Add(Me.TXT_NAME, 0, 0)
             Me.TP_MAIN.Controls.Add(Me.TXT_SITE, 0, 2)
+            Me.TP_MAIN.Controls.Add(Me.TXT_LABELS, 0, 3)
             Me.TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
             Me.TP_MAIN.Location = New System.Drawing.Point(0, 0)
             Me.TP_MAIN.Name = "TP_MAIN"
-            Me.TP_MAIN.RowCount = 4
+            Me.TP_MAIN.RowCount = 5
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
             Me.TP_MAIN.Size = New System.Drawing.Size(522, 336)
             Me.TP_MAIN.TabIndex = 0
@@ -178,13 +184,13 @@ Namespace DownloadObjects
             TP_USERS.Controls.Add(TP_USERS_2, 0, 0)
             TP_USERS.Controls.Add(Me.LIST_USERS, 0, 1)
             TP_USERS.Dock = System.Windows.Forms.DockStyle.Fill
-            TP_USERS.Location = New System.Drawing.Point(0, 81)
+            TP_USERS.Location = New System.Drawing.Point(0, 109)
             TP_USERS.Margin = New System.Windows.Forms.Padding(0)
             TP_USERS.Name = "TP_USERS"
             TP_USERS.RowCount = 2
             TP_USERS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_USERS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_USERS.Size = New System.Drawing.Size(522, 255)
+            TP_USERS.Size = New System.Drawing.Size(522, 227)
             TP_USERS.TabIndex = 3
             '
             'TP_USERS_2
@@ -206,7 +212,7 @@ Namespace DownloadObjects
             TP_USERS_2.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
             TP_USERS_2.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_USERS_2.Size = New System.Drawing.Size(522, 25)
-            TP_USERS_2.TabIndex = 0
+            TP_USERS_2.TabIndex = 4
             '
             'CH_U_USE
             '
@@ -258,8 +264,8 @@ Namespace DownloadObjects
             Me.LIST_USERS.FormattingEnabled = True
             Me.LIST_USERS.Location = New System.Drawing.Point(3, 28)
             Me.LIST_USERS.Name = "LIST_USERS"
-            Me.LIST_USERS.Size = New System.Drawing.Size(516, 224)
+            Me.LIST_USERS.Size = New System.Drawing.Size(516, 196)
-            Me.LIST_USERS.TabIndex = 1
+            Me.LIST_USERS.TabIndex = 5
             '
             'TXT_NAME
             '
@@ -298,6 +304,25 @@ Namespace DownloadObjects
             Me.TXT_SITE.TabIndex = 2
             Me.TXT_SITE.TextBoxReadOnly = True
             '
+            'TXT_LABELS
+            '
+            ActionButton4.BackgroundImage = CType(resources.GetObject("ActionButton4.BackgroundImage"), System.Drawing.Image)
+            ActionButton4.Name = "Edit"
+            ActionButton4.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Edit
+            ActionButton5.BackgroundImage = CType(resources.GetObject("ActionButton5.BackgroundImage"), System.Drawing.Image)
+            ActionButton5.Name = "Clear"
+            ActionButton5.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            Me.TXT_LABELS.Buttons.Add(ActionButton4)
+            Me.TXT_LABELS.Buttons.Add(ActionButton5)
+            Me.TXT_LABELS.CaptionText = "Labels"
+            Me.TXT_LABELS.CaptionWidth = 40.0R
+            Me.TXT_LABELS.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_LABELS.Location = New System.Drawing.Point(3, 84)
+            Me.TXT_LABELS.Name = "TXT_LABELS"
+            Me.TXT_LABELS.Size = New System.Drawing.Size(516, 22)
+            Me.TXT_LABELS.TabIndex = 3
+            Me.TXT_LABELS.TextBoxReadOnly = True
+            '
             'FeedFilterForm
             '
             Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
@@ -317,6 +342,7 @@ Namespace DownloadObjects
             TP_USERS_2.PerformLayout()
             CType(Me.TXT_NAME, System.ComponentModel.ISupportInitialize).EndInit()
             CType(Me.TXT_SITE, System.ComponentModel.ISupportInitialize).EndInit()
+            CType(Me.TXT_LABELS, System.ComponentModel.ISupportInitialize).EndInit()
             Me.ResumeLayout(False)
 
         End Sub
@@ -333,5 +359,6 @@ Namespace DownloadObjects
         Private WithEvents TXT_NAME As PersonalUtilities.Forms.Controls.TextBoxExtended
         Private WithEvents TP_MAIN As TableLayoutPanel
         Private WithEvents TXT_SITE As PersonalUtilities.Forms.Controls.TextBoxExtended
+        Private WithEvents TXT_LABELS As PersonalUtilities.Forms.Controls.TextBoxExtended
     End Class
 End Namespace

SCrawler/Download/Feed/FeedFilterForm.resx

@@ -133,7 +133,7 @@
   <data name="ActionButton1.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAHpJREFUOE+1kVEKgDAMQ3e2/e+MXqpn6W/HxM7SpkIVB4HxSMKiTUTaFwVQ1X25
+        wgAADsIBFShKgAAAAHpJREFUOE+1kVEKgDAMQ3e2/e+MXqpn6W/HxM7SpkIVB4HxSMKiTUTaFwVQ1X25
         DjMfSxskHBYsAxHJkjUjHgrUNMY4peaMPxb03rcZMVhgn2oDKAwn+L0aROH/Cny4NAGFSx8x+10ZDwV2
         gt+LOCxQsw1nPBS8VQBVTTzyhrdZSUm7AAAAAElFTkSuQmCC
 </value>
@@ -185,7 +185,7 @@
         RudN65tdA8sHzg46DF645Xrr8m2v29fvrLwzMBQydHc4cnjkLvvu5L2key/vZ9yff7DpIfph4SOpR+WP
         lR7X/aj3Y+uI5ciZUdfRvidBTx6Mscae/5T+04fx/Kfkp+UTqhONk2aTp6fcp24+W/1s/Hnq8/npgp+l
         f65+ofviu18cf+mbWTUz/pL/cuHX4lfyr468Xva6e9Z/9vGb5Dfzc4Vv5d8efcd41/s+7P3EfOYH7IeK
-        j3ofuz55f3q4kLyw8Bv3hPP74uYdwgAAAAlwSFlzAAALEwAACxMBAJqcGAAAAZlJREFUOE9j+P//PwOp
+        j3ofuz55f3q4kLyw8Bv3hPP74uYdwgAAAAlwSFlzAAALEQAACxEBf2RfkQAAAZlJREFUOE9j+P//PwOp
         eL2joz2MjSGJDztZWbGtcHTcuNrB4dx8W9tikgwQVxVnPXRo/9ObXZ3/7+bl/l/l7PxxkY1NJIZCbNhc
         U5Q9ozX9f/fS/v+vXr76f3XmzP9LPDxOMjAwqGIoRsc60iysF5ZFfd25acl/wxKX/w1T6n+uXbvm+9T+
         xiCQPIYGZKwhzsh0ZHbg189X2////3/gJ8gQkEtgmvEaIM/PwLJzgvfnd+ea/v//sf7/10ed/3qyTFA0
@@ -198,7 +198,72 @@
   <data name="ActionButton3.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>
         iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
-        xAAADsQBlSsOGwAAAHpJREFUOE+1kVEKgDAMQ3e2/e+MXqpn6W/HxM7SpkIVB4HxSMKiTUTaFwVQ1X25
+        wgAADsIBFShKgAAAAHpJREFUOE+1kVEKgDAMQ3e2/e+MXqpn6W/HxM7SpkIVB4HxSMKiTUTaFwVQ1X25
+        DjMfSxskHBYsAxHJkjUjHgrUNMY4peaMPxb03rcZMVhgn2oDKAwn+L0aROH/Cny4NAGFSx8x+10ZDwV2
+        gt+LOCxQsw1nPBS8VQBVTTzyhrdZSUm7AAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton4.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGOfPtRkwAAACBjSFJNAACH
+        DwAAjA8AAP1SAACBQAAAfXkAAOmLAAA85QAAGcxzPIV3AAAKOWlDQ1BQaG90b3Nob3AgSUNDIHByb2Zp
+        bGUAAEjHnZZ3VFTXFofPvXd6oc0w0hl6ky4wgPQuIB0EURhmBhjKAMMMTWyIqEBEEREBRZCggAGjoUis
+        iGIhKKhgD0gQUGIwiqioZEbWSnx5ee/l5ffHvd/aZ+9z99l7n7UuACRPHy4vBZYCIJkn4Ad6ONNXhUfQ
+        sf0ABniAAaYAMFnpqb5B7sFAJC83F3q6yAn8i94MAUj8vmXo6U+ng/9P0qxUvgAAyF/E5mxOOkvE+SJO
+        yhSkiu0zIqbGJIoZRomZL0pQxHJijlvkpZ99FtlRzOxkHlvE4pxT2clsMfeIeHuGkCNixEfEBRlcTqaI
+        b4tYM0mYzBXxW3FsMoeZDgCKJLYLOKx4EZuImMQPDnQR8XIAcKS4LzjmCxZwsgTiQ7mkpGbzuXHxArou
+        S49uam3NoHtyMpM4AoGhP5OVyOSz6S4pyalMXjYAi2f+LBlxbemiIluaWltaGpoZmX5RqP+6+Dcl7u0i
+        vQr43DOI1veH7a/8UuoAYMyKarPrD1vMfgA6tgIgd/8Pm+YhACRFfWu/8cV5aOJ5iRcIUm2MjTMzM424
+        HJaRuKC/6386/A198T0j8Xa/l4fuyollCpMEdHHdWClJKUI+PT2VyeLQDf88xP848K/zWBrIieXwOTxR
+        RKhoyri8OFG7eWyugJvCo3N5/6mJ/zDsT1qca5Eo9Z8ANcoISN2gAuTnPoCiEAESeVDc9d/75oMPBeKb
+        F6Y6sTj3nwX9+65wifiRzo37HOcSGExnCfkZi2viawnQgAAkARXIAxWgAXSBITADVsAWOAI3sAL4gWAQ
+        DtYCFogHyYAPMkEu2AwKQBHYBfaCSlAD6kEjaAEnQAc4DS6Ay+A6uAnugAdgBIyD52AGvAHzEARhITJE
+        geQhVUgLMoDMIAZkD7lBPlAgFA5FQ3EQDxJCudAWqAgqhSqhWqgR+hY6BV2ArkID0D1oFJqCfoXewwhM
+        gqmwMqwNG8MM2An2hoPhNXAcnAbnwPnwTrgCroOPwe3wBfg6fAcegZ/DswhAiAgNUUMMEQbigvghEUgs
+        wkc2IIVIOVKHtCBdSC9yCxlBppF3KAyKgqKjDFG2KE9UCIqFSkNtQBWjKlFHUe2oHtQt1ChqBvUJTUYr
+        oQ3QNmgv9Cp0HDoTXYAuRzeg29CX0HfQ4+g3GAyGhtHBWGE8MeGYBMw6TDHmAKYVcx4zgBnDzGKxWHms
+        AdYO64dlYgXYAux+7DHsOewgdhz7FkfEqeLMcO64CBwPl4crxzXhzuIGcRO4ebwUXgtvg/fDs/HZ+BJ8
+        Pb4LfwM/jp8nSBN0CHaEYEICYTOhgtBCuER4SHhFJBLVidbEACKXuIlYQTxOvEIcJb4jyZD0SS6kSJKQ
+        tJN0hHSedI/0ikwma5MdyRFkAXknuZF8kfyY/FaCImEk4SXBltgoUSXRLjEo8UISL6kl6SS5VjJHslzy
+        pOQNyWkpvJS2lIsUU2qDVJXUKalhqVlpirSptJ90snSxdJP0VelJGayMtoybDFsmX+awzEWZMQpC0aC4
+        UFiULZR6yiXKOBVD1aF6UROoRdRvqP3UGVkZ2WWyobJZslWyZ2RHaAhNm+ZFS6KV0E7QhmjvlygvcVrC
+        WbJjScuSwSVzcopyjnIcuUK5Vrk7cu/l6fJu8onyu+U75B8poBT0FQIUMhUOKlxSmFakKtoqshQLFU8o
+        3leClfSVApXWKR1W6lOaVVZR9lBOVd6vfFF5WoWm4qiSoFKmclZlSpWiaq/KVS1TPaf6jC5Ld6In0Svo
+        PfQZNSU1TzWhWq1av9q8uo56iHqeeqv6Iw2CBkMjVqNMo1tjRlNV01czV7NZ874WXouhFa+1T6tXa05b
+        RztMe5t2h/akjpyOl06OTrPOQ12yroNumm6d7m09jB5DL1HvgN5NfVjfQj9ev0r/hgFsYGnANThgMLAU
+        vdR6KW9p3dJhQ5Khk2GGYbPhqBHNyMcoz6jD6IWxpnGE8W7jXuNPJhYmSSb1Jg9MZUxXmOaZdpn+aqZv
+        xjKrMrttTjZ3N99o3mn+cpnBMs6yg8vuWlAsfC22WXRbfLS0suRbtlhOWWlaRVtVWw0zqAx/RjHjijXa
+        2tl6o/Vp63c2ljYCmxM2v9ga2ibaNtlOLtdZzllev3zMTt2OaVdrN2JPt4+2P2Q/4qDmwHSoc3jiqOHI
+        dmxwnHDSc0pwOub0wtnEme/c5jznYuOy3uW8K+Lq4Vro2u8m4xbiVun22F3dPc692X3Gw8Jjncd5T7Sn
+        t+duz2EvZS+WV6PXzAqrFetX9HiTvIO8K72f+Oj78H26fGHfFb57fB+u1FrJW9nhB/y8/Pb4PfLX8U/z
+        /z4AE+AfUBXwNNA0MDewN4gSFBXUFPQm2Dm4JPhBiG6IMKQ7VDI0MrQxdC7MNaw0bGSV8ar1q66HK4Rz
+        wzsjsBGhEQ0Rs6vdVu9dPR5pEVkQObRGZ03WmqtrFdYmrT0TJRnFjDoZjY4Oi26K/sD0Y9YxZ2O8Yqpj
+        ZlgurH2s52xHdhl7imPHKeVMxNrFlsZOxtnF7YmbineIL4+f5rpwK7kvEzwTahLmEv0SjyQuJIUltSbj
+        kqOTT/FkeIm8nhSVlKyUgVSD1ILUkTSbtL1pM3xvfkM6lL4mvVNAFf1M9Ql1hVuFoxn2GVUZbzNDM09m
+        SWfxsvqy9bN3ZE/kuOd8vQ61jrWuO1ctd3Pu6Hqn9bUboA0xG7o3amzM3zi+yWPT0c2EzYmbf8gzySvN
+        e70lbEtXvnL+pvyxrR5bmwskCvgFw9tst9VsR23nbu/fYb5j/45PhezCa0UmReVFH4pZxde+Mv2q4quF
+        nbE7+0ssSw7uwuzi7Rra7bD7aKl0aU7p2B7fPe1l9LLCstd7o/ZeLV9WXrOPsE+4b6TCp6Jzv+b+Xfs/
+        VMZX3qlyrmqtVqreUT13gH1g8KDjwZYa5ZqimveHuIfu1nrUttdp15UfxhzOOPy0PrS+92vG140NCg1F
+        DR+P8I6MHA082tNo1djYpNRU0gw3C5unjkUeu/mN6zedLYYtta201qLj4Ljw+LNvo78dOuF9ovsk42TL
+        d1rfVbdR2grbofbs9pmO+I6RzvDOgVMrTnV32Xa1fW/0/ZHTaqerzsieKTlLOJt/duFczrnZ86nnpy/E
+        XRjrjup+cHHVxds9AT39l7wvXbnsfvlir1PvuSt2V05ftbl66hrjWsd1y+vtfRZ9bT9Y/NDWb9nffsPq
+        RudN65tdA8sHzg46DF645Xrr8m2v29fvrLwzMBQydHc4cnjkLvvu5L2key/vZ9yff7DpIfph4SOpR+WP
+        lR7X/aj3Y+uI5ciZUdfRvidBTx6Mscae/5T+04fx/Kfkp+UTqhONk2aTp6fcp24+W/1s/Hnq8/npgp+l
+        f65+ofviu18cf+mbWTUz/pL/cuHX4lfyr468Xva6e9Z/9vGb5Dfzc4Vv5d8efcd41/s+7P3EfOYH7IeK
+        j3ofuz55f3q4kLyw8Bv3hPP74uYdwgAAAAlwSFlzAAALEAAACxABrSO9dQAAAZlJREFUOE9j+P//PwOp
+        eL2joz2MjSGJDztZWbGtcHTcuNrB4dx8W9tikgwQVxVnPXRo/9ObXZ3/7+bl/l/l7PxxkY1NJIZCbNhc
+        U5Q9ozX9f/fS/v+vXr76f3XmzP9LPDxOMjAwqGIoRsc60iysF5ZFfd25acl/wxKX/w1T6n+uXbvm+9T+
+        xiCQPIYGZKwhzsh0ZHbg189X2////3/gJ8gQkEtgmvEaIM/PwLJzgvfnd+ea/v//sf7/10ed/3qyTFA0
+        4zRAhoeBeXOn66c3J2r+//+64v+3B23/sWnGaoA0DwPbumbHT4/35v///23R/x/363FqxjBAWZSDeU6h
+        xdt727P+/38z8f/XO5V4NaMYICXAwbyuKeLW9Q0J//6/7vn/80YeQc0oBiyrdVh9c33B9//32159vZr2
+        hxjNcAPUZQTd766v+f3/4uz/n3Y1/19S6feSGM1gA1iZGPgXVQef/H+i7//xqWm/JlVG3CRWM9gAIwO9
+        xl1T8/83xVufE+fnqEBXQAgzKCmrXJGVkmhjYGDQR5ckBgMAtlm0SiCu/g8AAAAASUVORK5CYII=
+</value>
+  </data>
+  <data name="ActionButton5.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        wQAADsEBuJFr7QAAAHpJREFUOE+1kVEKgDAMQ3e2/e+MXqpn6W/HxM7SpkIVB4HxSMKiTUTaFwVQ1X25
         DjMfSxskHBYsAxHJkjUjHgrUNMY4peaMPxb03rcZMVhgn2oDKAwn+L0aROH/Cny4NAGFSx8x+10ZDwV2
         gt+LOCxQsw1nPBS8VQBVTTzyhrdZSUm7AAAAAElFTkSuQmCC
 </value>

SCrawler/Download/Feed/FeedFilterForm.vb

@@ -6,6 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
 Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Forms.Controls.Base
 Imports UTypes = SCrawler.API.Base.UserMedia.Types
@@ -105,6 +106,7 @@ Namespace DownloadObjects
                     End If
 
                     UpdateSitesText()
+                    UpdateLabelsText()
 
                     .EndLoaderOperations()
                     RefillList(True)
@@ -143,21 +145,29 @@ Namespace DownloadObjects
                 Return DirectCast(Obj, DataUserTmp).UserInfo
             End Function
         End Structure
+        Private SitePredicate As Predicate(Of IUserData) = Function(d) MyFilter.Sites.Count = 0 OrElse MyFilter.Sites.Contains(d.Site)
+        Private LabelPredicate As Predicate(Of IUserData) = Function(d) MyFilter.Labels.Count = 0 OrElse MyFilter.Labels.ListContains(d.Labels)
         Private Sub RefillList(Optional ByVal Init As Boolean = False)
             If Not MyDefs.Initializing Then
                 If Not Init Then ApplyFilter()
                 LIST_USERS.Items.Clear()
                 Dim dataUsers As New List(Of DataUserTmp)
                 If CH_U_SHOW_ALL.Checked Or MyData.Count = 0 Then
-                    dataUsers.ListAddList(Settings.UsersList)
+                    'dataUsers.ListAddList(Settings.UsersList)
-                Else
+                    If Settings.Users.Count > 0 Then dataUsers.ListAddList(From __d As IUserData In Settings.GetUsers(Function(____d) True)
-                    dataUsers.ListAddList(MyData.Select(Function(d) d.UserInfo), LAP.NotContainsOnly)
+                                                                           Where SitePredicate(__d) And LabelPredicate(__d)
+                                                                           Select DirectCast(__d, UserDataBase).User)
+                ElseIf MyData.Count > 0 Then
+                    'dataUsers.ListAddList(MyData.Select(Function(d) d.UserInfo), LAP.NotContainsOnly)
+                    dataUsers.ListAddList((From __d As IUserData In MyData.Select(Function(____d) ____d.User)
+                                           Where SitePredicate(__d) And LabelPredicate(__d)
+                                           Select DirectCast(__d, UserDataBase).User), LAP.NotContainsOnly)
                 End If
                 Dim i%
                 If dataUsers.Count > 0 Then
                     dataUsers.Sort()
                     With MyFilter
-                        If .Sites.Count > 0 Then dataUsers.RemoveAll(Function(d) Not .Sites.Contains(d.UserInfo.Site))
+                        'If .Sites.Count > 0 Then dataUsers.RemoveAll(Function(d) Not .Sites.Contains(d.UserInfo.Site))
                         If dataUsers.Count > 0 Then
                             dataUsers.ForEach(Sub(d) LIST_USERS.Items.Add(d, True))
                             If .Users.Count > 0 Then
@@ -243,10 +253,30 @@ Namespace DownloadObjects
             End Select
             If refill Then RefillList()
         End Sub
+        Private Sub TXT_LABELS_ActionOnButtonClick(ByVal Sender As Object, ByVal e As ActionButtonEventArgs) Handles TXT_LABELS.ActionOnButtonClick
+            Dim refill As Boolean = False
+            Select Case e.DefaultButton
+                Case ActionButton.DefaultButtons.Edit
+                    Using f As New LabelsForm(MyFilter.Labels, False)
+                        f.ShowDialog()
+                        If f.DialogResult = DialogResult.OK Then
+                            refill = Not MyFilter.Labels.ListEquals(f.LabelsList)
+                            MyFilter.Labels.ListAddList(f.LabelsList, LAP.NotContainsOnly, LAP.ClearBeforeAdd)
+                            UpdateLabelsText()
+                        End If
+                    End Using
+                Case ActionButton.DefaultButtons.Clear : refill = MyFilter.Labels.Count > 0 : MyFilter.Labels.Clear() : UpdateLabelsText()
+            End Select
+            If refill Then RefillList()
+        End Sub
         Private Sub UpdateSitesText()
             TXT_SITE.Clear()
             TXT_SITE.Text = MyFilter.Sites.ListToString
         End Sub
+        Private Sub UpdateLabelsText()
+            TXT_LABELS.Clear()
+            TXT_LABELS.Text = MyFilter.Labels.ListToString
+        End Sub
 #End Region
     End Class
 End Namespace

SCrawler/Download/Feed/FeedMedia.vb

@@ -12,6 +12,7 @@ Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Tools
 Imports SCrawler.API.Base
 Imports UserMediaD = SCrawler.DownloadObjects.TDownloader.UserMediaD
+Imports ImageRenderer2 = SCrawler.UserImage.ImageRenderer2
 Namespace DownloadObjects
     <ToolboxItem(False), DesignTimeVisible(False)>
     Public Class FeedMedia
@@ -20,7 +21,8 @@ Namespace DownloadObjects
         Friend Event MediaDeleted(ByVal Sender As Object)
         Friend Event MediaDeletedText(ByVal Sender As Object)
         Friend Event MediaDownload As EventHandler
-        Friend Event FeedAddWithRemove(ByVal Sender As FeedMedia, ByVal Feeds As IEnumerable(Of String), ByVal Media As UserMediaD, ByVal RemoveOperation As Boolean)
+        Friend Event FeedRemoveCheckedMedia(ByVal Sender As FeedMedia, ByVal Media As UserMediaD, ByVal Names As IEnumerable(Of String),
+                                            ByVal IsOverriddenNames As Boolean, ByVal IsAddAndRemove As Boolean)
         Friend Event MediaMove As MediaMoveCopyEventHandler
         Friend Event MediaCopy As MediaMoveCopyEventHandler
 #End Region
@@ -153,10 +155,12 @@ Namespace DownloadObjects
         End Function
 #End Region
 #Region "Initializers"
-        Private Sub FileCheckSpecialFolders(ByRef f As SFile, ByVal ExtractText As Boolean)
+        Private Sub FileCheckSpecialFolders(ByRef f As SFile, ByVal ExtractText As Boolean, ByVal OrigFile As SFile)
             If Not File.Exists Then
                 If ExtractText Then
                     f.Path = $"{File.Path.CSFilePS}{UserDataBase.PostTextSpecialFolderDefault}"
+                    If Not f.Exists Then f.Path = OrigFile.Path
+                    If Not f.Exists Then f.Path = $"{OrigFile.Path.CSFilePS}{UserDataBase.PostTextSpecialFolderDefault}"
                 ElseIf Media.Data.Type = UserMedia.Types.Video Then
                     f.Path = $"{File.Path.CSFilePS}Video"
                 End If
@@ -165,42 +169,6 @@ Namespace DownloadObjects
         Public Sub New()
             InitializeComponent()
         End Sub
-        Private Class ImageRenderer2 : Inherits ImageRenderer
-            Friend NativeFormat As String = Nothing
-            Friend ImgErr As Exception = Nothing
-            Friend Sub New(ByVal ImgPath As SFile, Optional ByVal e As ErrorsDescriber = Nothing)
-                MyBase.New()
-                Try
-                    If ImgPath.Exists(SFO.File, False) Then
-                        OriginalImageBytes = SFile.GetBytes(ImgPath, EDP.ThrowException)
-                        Try
-                            OriginalImage = GetImage(OriginalImageBytes)
-                        Catch exInternal As Exception
-                            HasError = True
-                            ImgErr = exInternal
-                            NativeFormat = GetTrueFormat(OriginalImageBytes, EDP.ReturnValue)
-                        End Try
-                    End If
-                    Address = ImgPath
-                Catch ex As Exception
-                    HasError = True
-                    NativeFormat = GetTrueFormat(OriginalImageBytes, EDP.ReturnValue)
-                    If Not e.Exists Then e = EDP.ThrowException
-                    ErrorsDescriber.Execute(e, ex, $"ImageRenderer2.New({ImgPath})")
-                End Try
-            End Sub
-            Friend Shared Function GetTrueFormat(ByVal Img() As Byte, Optional ByVal e As ErrorsDescriber = Nothing) As String
-                Try
-                    Using ms As New MemoryStream(Img, 0, Img.Length)
-                        Return System.Windows.Media.Imaging.BitmapDecoder.Create(ms, Windows.Media.Imaging.BitmapCreateOptions.PreservePixelFormat,
-                                                                                 Windows.Media.Imaging.BitmapCacheOption.OnLoad).Metadata.Format
-                    End Using
-                Catch ex As Exception
-                    If Not e.Exists Then e = EDP.ThrowException
-                    Return ErrorsDescriber.Execute(e, ex, "[ImageRenderer2.GetTrueFormat()]")
-                End Try
-            End Function
-        End Class
         Friend Sub New(ByVal Media As UserMediaD, ByVal Width As Integer, ByVal Height As Integer, ByVal IsSession As Boolean, ByVal ExtractText As Boolean)
             Try
                 InitializeComponent()
@@ -238,7 +206,17 @@ Namespace DownloadObjects
                         End With
                         If Not imgFile.Exists Then
                             Settings.Cache.Validate()
-                            If GetWebFile(Media.Data.URL, imgFile, EDP.None) AndAlso imgFile.Exists Then File = UserImage.ConvertWebp(imgFile, Nothing)
+                            If GetWebFile(Media.Data.URL, imgFile, EDP.None) AndAlso imgFile.Exists Then
+                                File = UserImage.ConvertWebp(imgFile, Nothing)
+                            ElseIf Not IsNothing(Media.Data.Object) AndAlso TypeOf Media.Data.Object Is IThumbList AndAlso
+                                   DirectCast(Media.Data.Object, IThumbList).Thumbs.ListExists Then
+                                With DirectCast(Media.Data.Object, IThumbList).Thumbs
+                                    For Each __tmpUrl$ In .Self
+                                        If GetWebFile(__tmpUrl, imgFile, EDP.None) AndAlso imgFile.Exists Then _
+                                           File = UserImage.ConvertWebp(imgFile, Nothing) : Exit For
+                                    Next
+                                End With
+                            End If
                         Else
                             File = imgFile
                         End If
@@ -248,18 +226,18 @@ Namespace DownloadObjects
                     LBL_TITLE.Dispose()
                     TP_MAIN.RowStyles(1).Height = 0
                     File = If(ExtractText, Media.Data.PostTextFile, Media.Data.File)
-                    FileCheckSpecialFolders(File, ExtractText)
+                    FileCheckSpecialFolders(File, ExtractText, Media.Data.File)
                 End If
 
                 If Not File.Exists And Not IsSubscription Then
                     If Not Media.Data.SpecialFolder.IsEmptyString Then
                         File = If(ExtractText, Media.Data.PostTextFile, Media.Data.File)
                         File.Path = $"{File.Path.CSFilePS}{Media.Data.SpecialFolder}".CSFileP
-                        FileCheckSpecialFolders(File, ExtractText)
+                        FileCheckSpecialFolders(File, ExtractText, Media.Data.File)
                     End If
                 End If
 
-                If File.Exists Then
+                If File.Exists Or IsSubscription Then
                     Information = $"Type: {IIf(ExtractText, UserMedia.Types.Text.ToString, Media.Data.Type.ToString)}"
                     Information.StringAppendLine($"File: {File.File}")
                     Information.StringAppendLine($"Address: {File}")
@@ -290,7 +268,7 @@ Namespace DownloadObjects
                             Dim webpConverted As Boolean = False
                             Dim isWebp As Boolean = False
                             tmpMediaFile = UserImage.ConvertWebp(tmpMediaFile, Nothing,,, webpConverted)
-                            If tmpMediaFile.IsEmptyString Then Throw New ArgumentNullException With {.HelpLink = 1}
+                            If tmpMediaFile.IsEmptyString And Not IsSubscription Then Throw New ArgumentNullException With {.HelpLink = 1}
                             Try
                                 For kConv As Byte = 0 To 1
                                     If kConv = 1 Then tmpMediaFile = UserImage.ConvertWebp(tmpMediaFile, Nothing, True, isWebp, webpConverted)
@@ -435,11 +413,11 @@ Namespace DownloadObjects
         End Function
         Private Sub Feed_SPEC_ADD_REMOVE(ByVal Source As ToolStripMenuItem, ByVal e As EventArgs)
             Dim f As FeedSpecial = Feed_SPEC_ADD_Impl(Source)
-            If Not f Is Nothing Then RaiseEvent FeedAddWithRemove(Me, {f.Name}, Media, False)
+            If Not f Is Nothing Then RaiseEvent FeedRemoveCheckedMedia(Me, Media, {f.Name}, False, True)
         End Sub
         Private Sub Feed_SPEC_REMOVE(ByVal Source As ToolStripMenuItem, ByVal e As EventArgs)
             Dim f As FeedSpecial = Source.Tag
-            If Not f Is Nothing AndAlso Not f.Disposed Then f.Remove(Media) : RaiseEvent FeedAddWithRemove(Me, {f.Name}, Media, True)
+            If Not f Is Nothing AndAlso Not f.Disposed Then f.Remove(Media) : RaiseEvent FeedRemoveCheckedMedia(Me, Media, {f.Name}, True, False)
         End Sub
 #End Region
 #Region "Dispose"
@@ -598,14 +576,14 @@ Namespace DownloadObjects
             With Settings.Feeds.Favorite
                 If Not .Contains(Media) Then .Add(Media)
                 BTT_FEED_ADD_FAV.ControlChangeColor(True, False)
-                If sender Is BTT_FEED_ADD_FAV_REMOVE Then RaiseEvent FeedAddWithRemove(Me, {FeedSpecial.FavoriteName}, Media, False)
+                If sender Is BTT_FEED_ADD_FAV_REMOVE Then RaiseEvent FeedRemoveCheckedMedia(Me, Media, {FeedSpecial.FavoriteName}, False, True)
             End With
         End Sub
         Private Sub BTT_FEED_ADD_SPEC_Click(sender As Object, e As EventArgs) Handles BTT_FEED_ADD_SPEC.Click, BTT_FEED_ADD_SPEC_REMOVE.Click
             With FeedSpecialCollection.ChooseFeeds(True)
                 If .ListExists Then
                     .ForEach(Sub(f) f.Add(Media))
-                    If sender Is BTT_FEED_ADD_SPEC_REMOVE Then RaiseEvent FeedAddWithRemove(Me, .Select(Function(f) f.Name), Media, False)
+                    If sender Is BTT_FEED_ADD_SPEC_REMOVE Then RaiseEvent FeedRemoveCheckedMedia(Me, Media, .Select(Function(f) f.Name), False, True)
                 End If
             End With
         End Sub
@@ -613,14 +591,14 @@ Namespace DownloadObjects
             With Settings.Feeds.Favorite
                 If .Contains(Media) Then .Remove(Media)
                 BTT_FEED_ADD_FAV.ControlChangeColor(True)
-                RaiseEvent FeedAddWithRemove(Me, {FeedSpecial.FavoriteName}, Media, True)
+                RaiseEvent FeedRemoveCheckedMedia(Me, Media, {FeedSpecial.FavoriteName}, True, False)
             End With
         End Sub
         Private Sub BTT_FEED_REMOVE_SPEC_Click(sender As Object, e As EventArgs) Handles BTT_FEED_REMOVE_SPEC.Click
             With FeedSpecialCollection.ChooseFeeds(False)
                 If .ListExists Then
                     .ForEach(Sub(f) f.Remove(Media))
-                    RaiseEvent FeedAddWithRemove(Me, .Select(Function(f) f.Name), Media, True)
+                    RaiseEvent FeedRemoveCheckedMedia(Me, Media, .Select(Function(f) f.Name), True, False)
                 End If
             End With
         End Sub

SCrawler/Editors/SiteEditorForm.vb

@@ -250,7 +250,14 @@ Namespace Editors
             MyDefs = New DefaultFormOptions(Me, Settings.Design)
             Host = h
             CookiesControlsInteraction = New List(Of PropertyValueHost)
-            If Not Host.Responser Is Nothing Then Cookies = Host.Responser.Cookies.Copy
+            If Not Host.Source Is Nothing AndAlso TypeOf Host.Source Is API.Base.SiteSettingsBase Then Cookies = DirectCast(Host.Source, API.Base.SiteSettingsBase).GetCookieKeeperInstance
+            If Not Host.Responser Is Nothing Then
+                If Cookies Is Nothing Then
+                    Cookies = Host.Responser.Cookies.Copy
+                Else
+                    Cookies.Copy(Host.Responser.Cookies)
+                End If
+            End If
             LabelControls = New List(Of Label)
             Host.BeginEdit()
         End Sub

SCrawler/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2026.1.17.0")>
+<Assembly: AssemblyVersion("2026.8.7.0")>
-<Assembly: AssemblyFileVersion("2026.1.17.0")>
+<Assembly: AssemblyFileVersion("2026.8.7.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/SCrawler.vbproj

@@ -194,6 +194,7 @@
     <Compile Include="API\Base\TokenBatch.vb" />
     <Compile Include="API\Base\YTDLP.vb" />
     <Compile Include="API\Bluesky\Declarations.vb" />
+    <Compile Include="API\Bluesky\EditorExchangeOptions.vb" />
     <Compile Include="API\Bluesky\M3U8.vb" />
     <Compile Include="API\Bluesky\SiteSettings.vb" />
     <Compile Include="API\Bluesky\UserData.vb" />

SCrawler/UserImage.vb

@@ -150,4 +150,45 @@ Friend Class UserImage : Inherits ImageRenderer
     Private Shared Function ConvertWebpTryImageMagick(ByVal InitFile As SFile, ByVal DestFile As SFile) As Boolean
         Return ImageRendererExt.ConvertWebp(InitFile, DestFile, EDP.SendToLog + EDP.ReturnValue)
     End Function
+    Friend Class ImageRenderer2 : Inherits ImageRenderer
+        Friend NativeFormat As String = Nothing
+        Friend ImgErr As Exception = Nothing
+        Friend ReadOnly Property IsWebP As Boolean
+            Get
+                Return NativeFormat.IfNullOrEmpty(ExtJpg).ToLower = ExtWebp
+            End Get
+        End Property
+        Friend Sub New(ByVal ImgPath As SFile, Optional ByVal e As ErrorsDescriber = Nothing)
+            MyBase.New()
+            Try
+                If ImgPath.Exists(SFO.File, False) Then
+                    OriginalImageBytes = SFile.GetBytes(ImgPath, EDP.ThrowException)
+                    Try
+                        OriginalImage = GetImage(OriginalImageBytes)
+                    Catch exInternal As Exception
+                        HasError = True
+                        ImgErr = exInternal
+                        NativeFormat = GetTrueFormat(OriginalImageBytes, EDP.ReturnValue)
+                    End Try
+                End If
+                Address = ImgPath
+            Catch ex As Exception
+                HasError = True
+                NativeFormat = GetTrueFormat(OriginalImageBytes, EDP.ReturnValue)
+                If Not e.Exists Then e = EDP.ThrowException
+                ErrorsDescriber.Execute(e, ex, $"ImageRenderer2.New({ImgPath})")
+            End Try
+        End Sub
+        Friend Shared Function GetTrueFormat(ByVal Img() As Byte, Optional ByVal e As ErrorsDescriber = Nothing) As String
+            Try
+                Using ms As New MemoryStream(Img, 0, Img.Length)
+                    Return System.Windows.Media.Imaging.BitmapDecoder.Create(ms, Windows.Media.Imaging.BitmapCreateOptions.PreservePixelFormat,
+                                                                             Windows.Media.Imaging.BitmapCacheOption.OnLoad).Metadata.Format
+                End Using
+            Catch ex As Exception
+                If Not e.Exists Then e = EDP.ThrowException
+                Return ErrorsDescriber.Execute(e, ex, "[ImageRenderer2.GetTrueFormat()]")
+            End Try
+        End Function
+    End Class
 End Class