2023.6.19.0

YT.Progress: make the playlists parsing progress more informative; change form display method
YT.YouTubeMediaContainerBase: fix sort algo
YT.Tray: add 'Add' button; add 'Ctrl+Click' on tray icon to add download
YT.Settings: add setting 'Download on click in tray: show form'
LPSG: some files didn't download (encoding)
Twitter: hide cache deletion errors
Mastogon: fixed bug in 'ReparseMissing' function
Reddit: downloaded gifs are static
XHamster: videos are not downloading or downloading incorrectly
Progress: fix bugs; minor improvements

Changelog.md

@@ -1,3 +1,44 @@
+# 2023.6.19.0
+
+*2023-06-19*
+
+- Added
+  - **OnlyFans**
+  - YouTube: make the playlists parsing progress more informative
+  - YouTube: add `Add` button to tray
+  - YouTube: add `Ctrl+Click` on tray icon to add download
+  - YouTube: add setting `Download on click in tray: show form`
+  - Minor improvements to progress bars
+  - Other improvements
+- Fixed
+  - YouTube: incorrect sorting algorithm
+  - LPSG: some files didn't download
+  - Reddit: downloaded gifs are static (Issue #141)
+  - xHamster: videos are not downloading or downloading incorrectly (Issue #144)
+  - Progress bar bugs
+  - Minor bugs
+
+# 2023.6.9.0
+
+*2023-06-09*
+
+- Fixed
+  - YouTube: opening paths to downloaded playlists and channels
+  - Twitter: make the algorithm faster
+  - Make progress more informative
+
+# 2023.6.8.0
+
+*2023-06-08*
+
+- Added
+  - YouTube: append artist name to music playlist output path
+  - YouTube: save thumbnail path for playlist and channel
+- Fixed
+  - YouTube: opening paths to downloaded playlists and channels
+  - Twitter: profile not fully downloaded
+  - Corrected form size for small monitors (Issue #136)
+
 # 2023.6.5.0
 
 *2023-06-05*

ProgramsComparison.md

@@ -117,7 +117,7 @@ https://github.com/RipMeApp/ripme
 | **Free options** | The program is completely free | The program is completely free, but site limits are not declared |
 | Operating Systems | Windows 10+ | Windows, MacOS, Linux |
 | Select want content type to download | Yes | Yes |
-| Suported sites | 15 internal and any site using plugins | 86+ sites (declared) |
+| Suported sites | 15+ internal and any site using plugins | 86+ sites (declared) |
 | Other sites support | **Yes** | No |
 | Still supported | **Yes** | **No (last release date May 4, 2021)** |
 

README.md

@@ -11,7 +11,7 @@
 :eu:
 :greece:
 
-A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, Reddit, Twitter, Mastodon, Instagram, TikTok, RedGifs, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
+A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, TikTok, RedGifs, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
 
 **If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
 <!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
@@ -31,6 +31,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
   - Reddit images, galleries of images, videos, saved posts;
   - Redgifs videos (https://www.redgifs.com/);
   - Twitter images and videos, saved (bookmarked) posts;
+  - OnlyFans images and videos, saved (bookmarked) posts;
   - Mastodon images and videos, saved (bookmarked) posts;
   - Instagram images and videos, tagged posts, stories, saved posts;
   - TikTok videos (*currently broken*; [limited](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok-limits));
@@ -65,6 +66,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **YouTube Music**
 - **Reddit**
 - **Twitter**
+- **OnlyFans**
 - **Mastodon**
 - **Instagram**
 - TikTok (*currently broken*; [limited](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok-limits))
@@ -122,6 +124,7 @@ The program parses user posts and compares file names with existing ones to remo
   - **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
   - [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
   - [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
+  - [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
   - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#Mastodon)
   - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
   - [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)

SCrawler.YouTube/Base/YouTubeSettings.vb

@@ -152,6 +152,9 @@ Namespace API.YouTube.Base
         <Browsable(True), GridVisible(False), XMLVN({"Defaults"}, False), Category("Defaults"), DisplayName("Confirm exit"),
             Description("Exit confirmation when closing the program.")>
         Public ReadOnly Property ExitConfirm As XMLValue(Of Boolean)
+        <Browsable(True), GridVisible(False), XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Download on click in tray: show form"),
+            Description("Show main window when download by clicking (Ctrl+Click) the tray icon. Default: false")>
+        Public ReadOnly Property ShowFormDownTrayClick As XMLValue(Of Boolean)
 #End Region
 #Region "Defaults Video"
         <Browsable(True), GridVisible, XMLVN({"DefaultsVideo"}, "MKV"), Category("Defaults Video"), DisplayName("Default format"),

SCrawler.YouTube/Controls/MusicPlaylistsForm.vb

@@ -81,6 +81,14 @@ Namespace API.YouTube.Controls
 
                 If Not .UserTitle.IsEmptyString Then
                     Text = .UserTitle
+                    If .ObjectType = Base.YouTubeMediaType.PlayList Then
+                        If Not .PlaylistTitle.IsEmptyString AndAlso Not .PlaylistTitle = .UserTitle Then
+                            Text &= $" - { .PlaylistTitle}"
+                        ElseIf Not .Title.IsEmptyString AndAlso Not .Title = .UserTitle Then
+                            Text &= $" - { .Title}"
+                        End If
+                    End If
+                    If Not TXT_OUTPUT_PATH.IsEmptyString AndAlso Not TXT_OUTPUT_PATH.Text.Contains(.UserTitle) Then TXT_OUTPUT_PATH.Text = $"{TXT_OUTPUT_PATH.Text.TrimEnd("\")}\{ .UserTitle}\"
                 ElseIf Not .PlaylistTitle.IsEmptyString Then
                     Text = .PlaylistTitle
                 End If

SCrawler.YouTube/Controls/ParsingProgressForm.Designer.vb

@@ -82,7 +82,6 @@ Namespace API.YouTube.Controls
             Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
             Me.StartPosition = System.Windows.Forms.FormStartPosition.CenterParent
             Me.Text = "Parsing progress"
-            Me.TopMost = True
             TP_MAIN.ResumeLayout(False)
             TP_MAIN.PerformLayout()
             Me.ResumeLayout(False)

SCrawler.YouTube/Controls/ParsingProgressForm.vb

@@ -17,9 +17,27 @@ Namespace API.YouTube.Controls
                 Return TokenSource.Token
             End Get
         End Property
-        Public Sub New()
+        Private ReadOnly CountMax As Integer
+        Private CountCurrent As Integer = 1
+        Friend Sub NextPlaylist()
+            CountCurrent += 1
+            MyProgress.InformationTemporary(True) = InfoStr
+            MyProgress.Information = InfoStr
+        End Sub
+        Private ReadOnly Property InfoStr As String
+            Get
+                Const MainMsg$ = "Data parsing in progress"
+                If CountMax > 1 Then
+                    Return $"{MainMsg} [{CountCurrent - 1}/{CountMax}]"
+                Else
+                    Return MainMsg
+                End If
+            End Get
+        End Property
+        Public Sub New(Optional ByVal _Count As Integer = 1)
             InitializeComponent()
-            MyProgress = New MyProgress(PR_MAIN, LBL_MAIN, "Data parsing in progress") With {.ResetProgressOnMaximumChanges = False}
+            CountMax = _Count
+            MyProgress = New MyProgress(PR_MAIN, LBL_MAIN, InfoStr) With {.ResetProgressOnMaximumChanges = False}
             TokenSource = New CancellationTokenSource
         End Sub
         Public Sub SetInitialValues(ByVal Count As Integer, ByVal Info As String)

SCrawler.YouTube/Downloader/MediaItem.vb

@@ -115,31 +115,36 @@ Namespace DownloadObjects.STDownloader
             Me.New
             Const d$ = " " & ChrW(183) & " "
             MyContainer = Container
-            MyContainer.Progress = MyProgress
+            With MyContainer
-            If MyContainer.HasElements Then FileOption = SFO.Path Else FileOption = SFO.File
+                .Progress = MyProgress
-            If Not MyContainer.SiteKey = YouTubeSiteKey Then
+                If .HasElements Then FileOption = SFO.Path Else FileOption = SFO.File
-                BTT_DOWN_AGAIN.Visible = False
+                If .DownloadState = Plugin.UserMediaStates.Downloaded AndAlso
-                SEP_DOWN_AGAIN.Visible = False
+                   (.ObjectType = Base.YouTubeMediaType.Channel Or .ObjectType = Base.YouTubeMediaType.PlayList) AndAlso FileOption = SFO.File AndAlso
-            End If
+                   Not .File.Exists AndAlso .File.Exists(SFO.Path, False) Then FileOption = SFO.Path
+                If Not .SiteKey = YouTubeSiteKey Then
+                    BTT_DOWN_AGAIN.Visible = False
+                    SEP_DOWN_AGAIN.Visible = False
+                End If
 
-            ICON_SITE.Image = MyContainer.SiteIcon
+                ICON_SITE.Image = .SiteIcon
-            LBL_TIME.Text = AConvert(Of String)(Container.Duration, TimeToStringProvider, String.Empty)
+                LBL_TIME.Text = AConvert(Of String)(.Duration, TimeToStringProvider, String.Empty)
-            LBL_TITLE.Text = Container.ToString(True)
+                LBL_TITLE.Text = .ToString(True)
-            If Not Container.SiteKey = YouTubeSiteKey And Container.ContentType = Plugin.UserMediaTypes.Picture Then
+                If Not .SiteKey = YouTubeSiteKey And .ContentType = Plugin.UserMediaTypes.Picture Then
-                LBL_INFO.Text = Container.File.Extension.StringToUpper
+                    LBL_INFO.Text = .File.Extension.StringToUpper
-            ElseIf Not Container.IsMusic Then
+                ElseIf Not .IsMusic Then
-                If Container.Height > 0 Then
+                    If .Height > 0 Then
-                    LBL_INFO.Text = $"{Container.File.Extension.StringToUpper}{d}{Container.Height}p"
+                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{ .Height}p"
+                    Else
+                        LBL_INFO.Text = .File.Extension.StringToUpper
+                    End If
                 Else
-                    LBL_INFO.Text = Container.File.Extension.StringToUpper
+                    If .Bitrate > 0 Then
+                        LBL_INFO.Text = $"{ .File.Extension.StringToUpper}{d}{ .Bitrate}k"
+                    Else
+                        LBL_INFO.Text = .File.Extension.StringToUpper
+                    End If
                 End If
-            Else
+            End With
-                If Container.Bitrate > 0 Then
-                    LBL_INFO.Text = $"{Container.File.Extension.StringToUpper}{d}{Container.Bitrate}k"
-                Else
-                    LBL_INFO.Text = Container.File.Extension.StringToUpper
-                End If
-            End If
             UpdateMediaIcon()
         End Sub
 #End Region
@@ -375,7 +380,7 @@ Namespace DownloadObjects.STDownloader
                         m.Show()
                     End If
                 Else
-                    If MyContainer.File.Exists(SFO.Path, False) Then MyContainer.File.Open(SFO.Path,, EDP.ShowMainMsg) Else m.Show()
+                    If MyContainer.File.Exists(SFO.Path, False) Then GlobalOpenPath(MyContainer.File, EDP.ShowMainMsg) Else m.Show()
                 End If
             End If
             OnDoubleClick(e)

SCrawler.YouTube/Downloader/VideoListForm.vb

@@ -237,6 +237,7 @@ Namespace DownloadObjects.STDownloader
             Dim pForm As ParsingProgressForm = Nothing
             Try
                 Dim useCookies As Boolean = MyYouTubeSettings.DefaultUseCookies
+                Dim sTag$ = If(Sender?.Tag, String.Empty)
                 Dim disableDown As Boolean = e.Shift
                 If e.Control Then useCookies = True
                 Dim useCookiesParse As Boolean? = Nothing
@@ -247,21 +248,28 @@ Namespace DownloadObjects.STDownloader
                 Dim GetDefault As Boolean = True
                 Dim GetShorts As Boolean = True
 
-                If Sender.Tag = "pls" Then
+                If sTag = "pls" Then
                     Using pf As New PlaylistArrayForm With {.DesignXML = DesignXML}
                         pf.ShowDialog()
                         If pf.DialogResult = DialogResult.OK Then
                             With pf.URLs
                                 If .Count > 0 Then
-                                    pForm = New ParsingProgressForm
+                                    pForm = New ParsingProgressForm(.Count)
-                                    pForm.Show()
+                                    pForm.Show(Me)
                                     pForm.SetInitialValues(.Count, "Parsing playlists...")
                                     Dim containers As New List(Of IYouTubeMediaContainer)
-                                    For Each u$ In .Self : containers.Add(YouTubeFunctions.Parse(u, useCookiesParse, pForm.Token, pForm.MyProgress, True, False)) : pForm.MyProgress.Perform() : Next
+                                    For Each u$ In .Self
+                                        containers.Add(YouTubeFunctions.Parse(u, useCookiesParse, pForm.Token, pForm.MyProgress, True, False))
+                                        pForm.NextPlaylist()
+                                        pForm.MyProgress.Perform()
+                                    Next
                                     pForm.Dispose()
                                     If containers.Count > 0 Then containers.ListDisposeRemoveAll(Function(cc) cc.HasError Or Not cc.Exists)
                                     If containers.Count > 0 Then
-                                        c = New Channel With {.UserTitle = IIf(pf.IsOneArtist, containers(0).UserTitle, "Playlists")}
+                                        c = New Channel With {
+                                            .UserTitle = IIf(pf.IsOneArtist, containers(0).UserTitle, "Playlists"),
+                                            .IsMusic = containers.Any(Function(cc) cc.IsMusic)
+                                        }
                                         c.Elements.AddRange(containers)
                                     End If
                                 End If
@@ -269,7 +277,7 @@ Namespace DownloadObjects.STDownloader
                         End If
                     End Using
                 Else
-                    Select Case CStr(Sender.Tag)
+                    Select Case sTag
                         Case "ans" : GetShorts = False
                         Case "as" : GetDefault = False : GetShorts = True
                     End Select
@@ -280,7 +288,7 @@ Namespace DownloadObjects.STDownloader
                 If Not c Is Nothing OrElse YouTubeFunctions.IsMyUrl(url) Then
                     If c Is Nothing Then
                         pForm = New ParsingProgressForm
-                        pForm.Show()
+                        pForm.Show(Me)
                         pForm.SetInitialValues(1, "Parsing data...")
                         c = YouTubeFunctions.Parse(url, useCookiesParse, pForm.Token, pForm.MyProgress, GetDefault, GetShorts)
                         pForm.Dispose()

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.6.5.0")>
+<Assembly: AssemblyVersion("2023.6.19.0")>
-<Assembly: AssemblyFileVersion("2023.6.5.0")>
+<Assembly: AssemblyFileVersion("2023.6.19.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/Objects/YouTubeMediaContainerBase.vb

@@ -22,17 +22,8 @@ Imports UMStates = SCrawler.Plugin.UserMediaStates
 Imports CollectionModes = PersonalUtilities.Functions.XML.Objects.IXMLValuesCollection.Modes
 Namespace API.YouTube.Objects
     Public Class ContainerDateComparer : Implements IComparer(Of IYouTubeMediaContainer)
-        Private ReadOnly NullDateValue As New Date
         Public Function Compare(ByVal x As IYouTubeMediaContainer, ByVal y As IYouTubeMediaContainer) As Integer Implements IComparer(Of IYouTubeMediaContainer).Compare
-            If x.DateDownloaded = NullDateValue And y.DateDownloaded = NullDateValue Then
+            Return x.DateCreated.CompareTo(y.DateCreated) * -1
-                Return x.DateCreated.CompareTo(y.DateCreated) * -1
-            ElseIf x.DateDownloaded = NullDateValue Then
-                Return -1
-            ElseIf y.DateDownloaded = NullDateValue Then
-                Return 1
-            Else
-                Return x.DateDownloaded.CompareTo(y.DateDownloaded) * -1
-            End If
         End Function
     End Class
     Public MustInherit Class YouTubeMediaContainerBase : Implements IYouTubeMediaContainer
@@ -1040,6 +1031,14 @@ Namespace API.YouTube.Objects
         End Sub
 #End Region
 #Region "Save"
+        Private Function GetThumbnails() As IEnumerable(Of SFile)
+            If HasElements Then
+                Return ListAddList(Of SFile)(New List(Of SFile)({ThumbnailFile}),
+                                             Elements.SelectMany(Function(ee As YouTubeMediaContainerBase) ee.GetThumbnails))
+            Else
+                Return {ThumbnailFile}
+            End If
+        End Function
         Public Overridable Sub Save() Implements IDownloadableMedia.Save
             Try
                 Dim fSettings As SFile = FileSettings
@@ -1068,6 +1067,11 @@ Namespace API.YouTube.Objects
                 Else
                     If CachePath.Exists(SFO.Path, False) Then CachePath.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.None)
                     CachePath = Nothing
+                    If ThumbnailFile.IsEmptyString And HasElements Then
+                        With ListAddList(Nothing, GetThumbnails, LAP.NotContainsOnly).ListWithRemove(Function(tf) tf.IsEmptyString)
+                            If .ListExists Then _ThumbnailFile = .FirstOrDefault(Function(tf) tf.Exists)
+                        End With
+                    End If
                 End If
 
                 Using x As New XmlFile With {.AllowSameNames = True}

SCrawler.YouTubeDownloader/MainFrame.Designer.vb

@@ -20,9 +20,12 @@ Partial Public Class MainFrame : Inherits SCrawler.DownloadObjects.STDownloader.
     Private Sub InitializeComponent()
         Me.components = New System.ComponentModel.Container()
         Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(MainFrame))
+        Dim CONTEXT_SEP_1 As System.Windows.Forms.ToolStripSeparator
         Me.TRAY_ICON = New System.Windows.Forms.NotifyIcon(Me.components)
         Me.TRAY_CONTEXT = New System.Windows.Forms.ContextMenuStrip(Me.components)
         Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem()
+        Me.CONTEXT_BTT_ADD = New PersonalUtilities.Forms.Controls.KeyClick.ToolStripMenuItemKeyClick()
+        CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator()
         Me.TRAY_CONTEXT.SuspendLayout()
         Me.SuspendLayout()
         '
@@ -32,13 +35,13 @@ Partial Public Class MainFrame : Inherits SCrawler.DownloadObjects.STDownloader.
         Me.TRAY_ICON.BalloonTipTitle = "YouTube Downloader"
         Me.TRAY_ICON.ContextMenuStrip = Me.TRAY_CONTEXT
         Me.TRAY_ICON.Icon = CType(resources.GetObject("TRAY_ICON.Icon"), System.Drawing.Icon)
-        Me.TRAY_ICON.Text = "YouTube Downloader"
+        Me.TRAY_ICON.Text = "YouTube Downloader" & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+Click to add download"
         '
         'TRAY_CONTEXT
         '
-        Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_TRAY_CLOSE})
+        Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.CONTEXT_BTT_ADD, CONTEXT_SEP_1, Me.BTT_TRAY_CLOSE})
         Me.TRAY_CONTEXT.Name = "ContextMenuStrip1"
-        Me.TRAY_CONTEXT.Size = New System.Drawing.Size(181, 48)
+        Me.TRAY_CONTEXT.Size = New System.Drawing.Size(181, 76)
         '
         'BTT_TRAY_CLOSE
         '
@@ -47,11 +50,24 @@ Partial Public Class MainFrame : Inherits SCrawler.DownloadObjects.STDownloader.
         Me.BTT_TRAY_CLOSE.Size = New System.Drawing.Size(180, 22)
         Me.BTT_TRAY_CLOSE.Text = "Close"
         '
+        'CONTEXT_BTT_ADD
+        '
+        Me.CONTEXT_BTT_ADD.Name = "CONTEXT_BTT_ADD"
+        Me.CONTEXT_BTT_ADD.Size = New System.Drawing.Size(180, 22)
+        Me.CONTEXT_BTT_ADD.Text = "Add"
+        Me.CONTEXT_BTT_ADD.Image = Global.PersonalUtilities.My.Resources.PlusPic_Green_24
+        '
+        'CONTEXT_SEP_1
+        '
+        CONTEXT_SEP_1.Name = "CONTEXT_SEP_1"
+        CONTEXT_SEP_1.Size = New System.Drawing.Size(177, 6)
+        '
         'MainFrame
         '
         Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
         Me.ClientSize = New System.Drawing.Size(1008, 729)
         Me.Name = "MainFrame"
+        Me.Text = "SCrawler: Happy LGBT Pride Month! :-)"
         Me.TRAY_CONTEXT.ResumeLayout(False)
         Me.ResumeLayout(False)
         Me.PerformLayout()
@@ -61,4 +77,5 @@ Partial Public Class MainFrame : Inherits SCrawler.DownloadObjects.STDownloader.
     Private WithEvents TRAY_ICON As NotifyIcon
     Private WithEvents TRAY_CONTEXT As ContextMenuStrip
     Private WithEvents BTT_TRAY_CLOSE As ToolStripMenuItem
+    Private WithEvents CONTEXT_BTT_ADD As PersonalUtilities.Forms.Controls.KeyClick.ToolStripMenuItemKeyClick
 End Class

SCrawler.YouTubeDownloader/MainFrame.resx

@@ -123,6 +123,9 @@
   <metadata name="TRAY_CONTEXT.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
     <value>425, 17</value>
   </metadata>
+  <metadata name="CONTEXT_SEP_1.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
+    <value>False</value>
+  </metadata>
   <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
   <data name="BTT_TRAY_CLOSE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
     <value>

SCrawler.YouTubeDownloader/MainFrame.vb

@@ -9,6 +9,7 @@
 Imports System.ComponentModel
 Imports SCrawler.API.YouTube
 Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Forms.Controls.KeyClick
 Public Class MainFrame
     Private WithEvents MyActivator As FormActivator
     Public Sub New()
@@ -66,6 +67,12 @@ CloseResume:
     Private Sub BTT_TRAY_CLOSE_Click(sender As Object, e As EventArgs) Handles BTT_TRAY_CLOSE.Click
         If CheckForClose(False) Then _IgnoreCloseConfirm = True : _IgnoreTrayOptions = True : Close()
     End Sub
+    Private Sub MyActivator_TrayIconClick(ByVal Sender As Object, ByVal e As KeyClickEventArgs) Handles MyActivator.TrayIconClick
+        If e.MouseButton = MouseButtons.Left And e.Control Then
+            BTT_ADD_KeyClick(Nothing, New KeyClickEventArgs)
+            e.Handled = Not MyYouTubeSettings.ShowFormDownTrayClick
+        End If
+    End Sub
     Private Function CheckForClose(ByVal _Ignore As Boolean) As Boolean
         If MyYouTubeSettings.ExitConfirm And Not _Ignore Then
             Return MsgBoxE({"Do you want to close the program?", "Closing the program"}, MsgBoxStyle.YesNo) = MsgBoxResult.Yes
@@ -77,6 +84,9 @@ CloseResume:
         MyBase.BTT_SETTINGS_Click(sender, e)
         TRAY_ICON.Visible = MyYouTubeSettings.CloseToTray
     End Sub
+    Protected Overrides Sub BTT_ADD_KeyClick(ByVal Sender As ToolStripMenuItemKeyClick, ByVal e As KeyClickEventArgs) Handles CONTEXT_BTT_ADD.KeyClick
+        MyBase.BTT_ADD_KeyClick(Sender, e)
+    End Sub
     Protected Overrides Sub MyJob_Started(ByVal Sender As Object, ByVal e As EventArgs)
         TRAY_ICON.Icon = My.Resources.ArrowDownIcon_Orange_24
     End Sub

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.6.5.0")>
+<Assembly: AssemblyVersion("2023.6.19.0")>
-<Assembly: AssemblyFileVersion("2023.6.5.0")>
+<Assembly: AssemblyFileVersion("2023.6.19.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/API/Base/UserDataBase.vb

@@ -1038,7 +1038,6 @@ BlockNullPicture:
                         _ContentList.Clear()
                         CreatedByChannel = False
                     End If
-                    If Not UserExists Then ReadyForDownload = False
                     UpdateUserInformation()
                     If _CollectionButtonsExists AndAlso _EnvirChanged Then UpdateButtonsColor()
                 ElseIf _ForceSaveUserInfo Then

SCrawler/API/BaseObjects/InternalSettingsForm.vb

@@ -154,7 +154,8 @@ Namespace API.Base
                     Dim tmpObj As Object
 
                     members = GetObjectMembers(MyObject, Function(m) (m.MemberType = MemberTypes.Field Or m.MemberType = MemberTypes.Property) AndAlso
-                                                                     Not m.GetCustomAttribute(Of PSettingAttribute) Is Nothing)
+                                                                     Not m.GetCustomAttribute(Of PSettingAttribute) Is Nothing,, True,
+                                                                     New FComparer(Of MemberInfo)(Function(mm1, mm2) mm1.Name = mm2.Name))
                     providersMembersSettings = GetObjectMembers(MySettingsInstance, providersPredicate)
                     providersMembersObj = GetObjectMembers(MyObject, providersPredicate)
 

SCrawler/API/LPSG/Declarations.vb

@@ -19,7 +19,10 @@ Namespace API.LPSG
         Friend ReadOnly Property NextPageRegex As RParams = RParams.DMS("<link rel=""next"" href=""(.+?/page-(\d+))""", 2)
         Private Const FileUrlRegexDefault As String = "([^/]+?)(jpg|jpeg|gif|png|webm)"
         Private ReadOnly InputFReplacer As New ErrorsDescriber(EDP.ReturnValue)
-        Private ReadOnly InputForbidRemover As Func(Of String, String) = Function(Input) If(Input.IsEmptyString, Input, Input.StringRemoveWinForbiddenSymbols(, InputFReplacer))
+        Private ReadOnly InputForbidRemover As Func(Of String, String) = Function(Input) If(Input.IsEmptyString,
+                                                                                            Input,
+                                                                                            Input.StringRemoveWinForbiddenSymbols(, InputFReplacer)).
+                                                                                            IfNullOrEmpty($"{Settings.Cache.NewFile.Name}.file")
         Private ReadOnly FileRegEx As RParams = RParams.DMS(FileUrlRegexDefault, 0, RegexReturn.ListByMatch, InputFReplacer)
 #Disable Warning IDE0060
         Friend Function FileRegExF(ByVal Input As String, ByVal Index As Integer) As String
@@ -28,7 +31,8 @@ Namespace API.LPSG
                 Dim l As List(Of String) = RegexReplace(Input, FileRegEx)
                 If l.ListExists(3) Then
                     Dim ext$ = l(2)
-                    Dim f$ = l(1).StringTrim("-", ".")
+                    Dim f$ = l(1).StringTrim("-", ".").StringRemoveWinForbiddenSymbols
+                    If f.IsEmptyString Then f = Settings.Cache.NewFile.Name
                     Input = $"{f}.{ext}"
                 End If
             End If

SCrawler/API/Mastodon/EditorExchangeOptions.vb

@@ -0,0 +1,22 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.Plugin.Attributes
+Namespace API.Mastodon
+    Friend Class EditorExchangeOptions : Inherits Twitter.EditorExchangeOptions
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelMedia As Boolean
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelProfile As Boolean
+        <PSetting(Address:=SettingAddress.None)> Friend Overrides Property DownloadModelSearch As Boolean
+        Friend Sub New(ByVal s As SiteSettings)
+            MyBase.New(s)
+        End Sub
+        Friend Sub New(ByVal u As UserData)
+            MyBase.New(u)
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Mastodon/SiteSettings.vb

@@ -139,9 +139,9 @@ Namespace API.Mastodon
 #End Region
 #Region "UserOptions"
         Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
-            If Options Is Nothing OrElse (Not TypeOf Options Is Twitter.EditorExchangeOptions OrElse
+            If Options Is Nothing OrElse (Not TypeOf Options Is EditorExchangeOptions OrElse
-                                          Not DirectCast(Options, Twitter.EditorExchangeOptions).SiteKey = MastodonSiteKey) Then _
+                                          Not DirectCast(Options, EditorExchangeOptions).SiteKey = MastodonSiteKey) Then _
-               Options = New Twitter.EditorExchangeOptions(Me) With {.SiteKey = MastodonSiteKey}
+               Options = New EditorExchangeOptions(Me) With {.SiteKey = MastodonSiteKey}
             If OpenForm Then
                 Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
             End If

SCrawler/API/Mastodon/UserData.vb

@@ -261,8 +261,9 @@ Namespace API.Mastodon
                                     If Not j Is Nothing Then
                                         PostDate = String.Empty
                                         If j.Contains("created_at") Then PostDate = j("created_at").Value Else PostDate = String.Empty
-                                        ObtainMedia(j, m.Post.ID, PostDate, UStates.Missing)
+                                        ObtainMedia(j, m.Post.ID, PostDate, m.URL_BASE)
                                         rList.Add(i)
+                                        j.Dispose()
                                     End If
                                 End If
                             End If

SCrawler/API/OnlyFans/Declarations.vb

@@ -0,0 +1,15 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.OnlyFans
+    Friend Module Declarations
+        Friend ReadOnly DateProvider As New ADateTime("O")
+        Friend ReadOnly RegExPostID As RParams = RParams.DM("(?<=onlyfans\.com/)(\d+)", 0, EDP.ReturnValue)
+    End Module
+End Namespace

SCrawler/API/OnlyFans/SiteSettings.vb

@@ -0,0 +1,165 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports SCrawler.API.Base
+Imports SCrawler.Plugin
+Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Cookies
+Imports PersonalUtilities.Functions.RegularExpressions
+Namespace API.OnlyFans
+    <Manifest("AndyProgram_OnlyFans"), SavedPosts, SeparatedTasks(1)>
+    Friend Class SiteSettings : Inherits SiteSettingsBase
+#Region "Icon"
+        Friend Overrides ReadOnly Property Icon As Icon
+            Get
+                Return My.Resources.SiteResources.OnlyFansIcon_32
+            End Get
+        End Property
+        Friend Overrides ReadOnly Property Image As Image
+            Get
+                Return My.Resources.SiteResources.OnlyFansPic_32
+            End Get
+        End Property
+#End Region
+#Region "Declarations"
+        Private Const HeaderBrowser As String = "sec-ch-ua"
+        Private Const HeaderUserID As String = "User-Id"
+        Private Const HeaderXBC As String = "X-Bc"
+        Private Const HeaderAppToken As String = "App-Token"
+        <PropertyOption(ControlText:=HeaderUserID, AllowNull:=False)>
+        Friend ReadOnly Property HH_USER_ID As PropertyValue
+        <PropertyOption(ControlText:=HeaderXBC, AllowNull:=False)>
+        Private ReadOnly Property HH_X_BC As PropertyValue
+        <PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False)>
+        Private ReadOnly Property HH_APP_TOKEN As PropertyValue
+        <PropertyOption(ControlText:=HeaderBrowser, AllowNull:=False)>
+        Private ReadOnly Property HH_BROWSER As PropertyValue
+        <PropertyOption(AllowNull:=False)>
+        Private ReadOnly Property UserAgent As PropertyValue
+        Private Sub UpdateHeader(ByVal PropertyName As String, ByVal Value As String)
+            Dim hName$ = String.Empty
+            Dim isUserAgent As Boolean = False
+            Select Case PropertyName
+                Case NameOf(HH_USER_ID) : hName = HeaderUserID
+                Case NameOf(HH_X_BC) : hName = HeaderXBC
+                Case NameOf(HH_APP_TOKEN) : hName = HeaderAppToken
+                Case NameOf(HH_BROWSER) : hName = HeaderBrowser
+                Case NameOf(UserAgent) : isUserAgent = True
+            End Select
+            If Not hName.IsEmptyString Then
+                Responser.Headers.Add(hName, Value)
+            ElseIf isUserAgent Then
+                Responser.UserAgent = Value
+            End If
+        End Sub
+        <PXML("LastDateUpdated")> Private ReadOnly Property LastDateUpdated_XML As PropertyValue
+        Friend Property LastDateUpdated As Date
+            Get
+                Return LastDateUpdated_XML.Value
+            End Get
+            Set(ByVal d As Date)
+                LastDateUpdated_XML.Value = d
+            End Set
+        End Property
+        <PropertyOption(ControlText:="Use old authorization rules",
+                        ControlToolTip:="Use old dynamic rules (from 'DATAHOARDERS') or new ones (from 'DIGITALCRIMINALS')." & vbCr &
+                                        "Change this value only if you know what you are doing."), PXML>
+        Friend ReadOnly Property UseOldAuthRules As PropertyValue
+        <PropertyOption(ControlText:="Dynamic rules update", ControlToolTip:="'Dynamic rules' update interval (minutes). Default: 1440", LeftOffset:=110), PXML>
+        Friend ReadOnly Property DynamicRulesUpdateInterval As PropertyValue
+        <Provider(NameOf(DynamicRulesUpdateInterval), FieldsChecker:=True)>
+        Private ReadOnly Property DynamicRulesUpdateIntervalProvider As IFormatProvider
+        <PropertyOption(ControlText:="Dynamic rules",
+                        ControlToolTip:="Overwrite 'Dynamic rules' with this URL" & vbCr &
+                                        "Change this value only if you know what you are doing."), PXML>
+        Friend ReadOnly Property DynamicRules As PropertyValue
+#End Region
+#Region "Initializer"
+        Friend Sub New()
+            MyBase.New("OnlyFans", ".onlyfans.com")
+
+            With Responser
+                .Accept = "application/json, text/plain, */*"
+                .AutomaticDecompression = Net.DecompressionMethods.GZip
+                .CookiesExtractMode = Responser.CookiesExtractModes.Any
+                .CookiesExtractedAutoSave = False
+                .CookiesUpdateMode = CookieKeeper.UpdateModes.Disabled
+                .Cookies.ChangedAllowInternalDrop = False
+                .Cookies.Changed = False
+                With .Headers
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaPlatform))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecChUaMobile))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchDest))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchMode))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.SecFetchSite))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.DHT))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.Authority, "onlyfans.com"))
+                    .Add(HttpHeaderCollection.GetSpecialHeader(MyHeaderTypes.AcceptEncoding))
+                    HH_USER_ID = New PropertyValue(.Value(HeaderUserID), GetType(String), Sub(v) UpdateHeader(NameOf(HH_USER_ID), v))
+                    HH_X_BC = New PropertyValue(.Value(HeaderXBC), GetType(String), Sub(v) UpdateHeader(NameOf(HH_X_BC), v))
+                    HH_APP_TOKEN = New PropertyValue(.Value(HeaderAppToken), GetType(String), Sub(v) UpdateHeader(NameOf(HH_APP_TOKEN), v))
+                    HH_BROWSER = New PropertyValue(.Value(HeaderBrowser), GetType(String), Sub(v) UpdateHeader(NameOf(HH_BROWSER), v))
+                End With
+                UserAgent = New PropertyValue(IIf(.UserAgentExists, .UserAgent, String.Empty), GetType(String), Sub(v) UpdateHeader(NameOf(UserAgent), v))
+            End With
+
+            LastDateUpdated_XML = New PropertyValue(Now.AddYears(-1), GetType(Date))
+            UseOldAuthRules = New PropertyValue(False)
+            DynamicRulesUpdateInterval = New PropertyValue(60 * 24)
+            DynamicRulesUpdateIntervalProvider = New FieldsCheckerProviderSimple(Function(v) IIf(AConvert(Of Integer)(v, 0) > 0, v, Nothing),
+                                                                                 "The value of [{0}] field must be greater than 0")
+            DynamicRules = New PropertyValue(String.Empty, GetType(String))
+            UserRegex = RParams.DMS("onlyfans.com/(\w+)", 1, EDP.ReturnValue)
+            UrlPatternUser = "https://onlyfans.com/{0}"
+            ImageVideoContains = "onlyfans.com"
+        End Sub
+#End Region
+#Region "GetInstance"
+        Friend Overrides Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider
+            Return New UserData
+        End Function
+#End Region
+#Region "Update"
+        Friend Overrides Sub Update()
+            If _SiteEditorFormOpened Then Responser.Cookies.Changed = False
+            MyBase.Update()
+        End Sub
+#End Region
+#Region "Download"
+        Friend Overrides Function BaseAuthExists() As Boolean
+            Return Responser.CookiesExists And {HH_USER_ID, HH_X_BC, HH_APP_TOKEN, HH_BROWSER, UserAgent}.All(Function(v) ACheck(v.Value))
+        End Function
+        Friend Overrides Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean
+            Return BaseAuthExists() And Not SessionAborted
+        End Function
+        Friend Property SessionAborted As Boolean = False
+        Friend Overrides Sub AfterDownload(ByVal User As Object, ByVal What As ISiteSettings.Download)
+            Responser.Cookies.Update(DirectCast(User, UserData).CCookie)
+        End Sub
+        Friend Overrides Sub DownloadDone(ByVal What As ISiteSettings.Download)
+            MyBase.DownloadDone(What)
+            SessionAborted = False
+            If Responser.Cookies.Changed Then Responser.SaveCookies() : Responser.Cookies.Changed = False
+        End Sub
+#End Region
+#Region "GetUserUrl, GetUserPostUrl"
+        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
+            Return String.Format(UrlPatternUser, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
+        End Function
+        Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
+            If Not Media.Post.ID.IsEmptyString Then
+                Return String.Format("https://onlyfans.com/{0}/{1}", Media.Post.ID, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
+            Else
+                Return String.Empty
+            End If
+        End Function
+#End Region
+    End Class
+End Namespace

SCrawler/API/OnlyFans/UserData.vb

@@ -0,0 +1,362 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Threading
+Imports SCrawler.API.Base
+Imports SCrawler.API.YouTube.Objects
+Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Clients.EventArguments
+Imports PersonalUtilities.Tools.Web.Cookies
+Imports PersonalUtilities.Tools.Web.Documents.JSON
+Imports UTypes = SCrawler.API.Base.UserMedia.Types
+Imports UStates = SCrawler.API.Base.UserMedia.States
+Namespace API.OnlyFans
+    Friend Class UserData : Inherits UserDataBase
+#Region "Declarations"
+        Friend Property CCookie As CookieKeeper = Nothing
+        Private Const HeaderSign As String = "Sign"
+        Private Const HeaderTime As String = "Time"
+        Private ReadOnly Property MySettings As SiteSettings
+            Get
+                Return HOST.Source
+            End Get
+        End Property
+        Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+        End Sub
+#End Region
+#Region "Download functions"
+        Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
+            If Not CCookie Is Nothing Then CCookie.Dispose()
+            CCookie = Responser.Cookies.Copy
+            Responser.Cookies.Clear()
+            AddHandler Responser.ResponseReceived, AddressOf OnResponseReceived
+            UpdateCookieHeader()
+            DownloadData(IIf(IsSavedPosts, 0, String.Empty), Token)
+        End Sub
+        Private Sub OnResponseReceived(ByVal Sender As Object, ByVal e As WebDataResponse)
+            If e.CookiesExists Then
+                CCookie.Update(e.Cookies, CookieKeeper.UpdateModes.ReplaceByNameAll,, EDP.ReturnValue)
+                UpdateCookieHeader()
+            End If
+        End Sub
+        Private Sub UpdateCookieHeader()
+            Responser.Headers.Add("Cookie", CCookie.ToString(False))
+        End Sub
+        Private Const BaseUrlPattern As String = "https://onlyfans.com{0}"
+        Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
+
+            Dim url$ = String.Empty
+            Dim _complete As Boolean = True
+            Do
+                Try
+                    Dim tmpCursor$ = String.Empty
+                    Dim hasMore As Boolean = False
+                    Dim path$ = String.Empty
+                    Dim postDate$, postID$
+                    Dim n As EContainer
+                    Dim mediaList As List(Of UserMedia)
+                    Dim mediaResult As Boolean
+
+                    If IsSavedPosts Then
+                        path = $"/api2/v2/posts/bookmarks/all/?format=infinite&limit=10&offset={Cursor}"
+                    Else
+                        If ID.IsEmptyString Then GetUserID()
+                        If ID.IsEmptyString Then Throw New ArgumentNullException("ID", "Unable to get user ID")
+
+                        path = $"/api2/v2/users/{ID}/posts/medias?limit=50&order=publish_date_desc&skip_users=all&format=infinite&counters=1"
+                        If Not Cursor.IsEmptyString Then path &= $"&counters=0&beforePublishTime={Cursor}" Else path &= "&counters=1"
+                    End If
+
+                    If UpdateSignature(path) Then
+                        url = String.Format(BaseUrlPattern, path)
+                        ThrowAny(Token)
+
+                        Dim r$ = Responser.GetResponse(url)
+                        If Not r.IsEmptyString Then
+                            Using j As EContainer = JsonDocument.Parse(r)
+                                If j.ListExists Then
+                                    If IsSavedPosts Then
+                                        hasMore = j.Value("hasMore").FromXML(Of Boolean)(False)
+                                    Else
+                                        tmpCursor = j.Value("tailMarker")
+                                        hasMore = Not tmpCursor.IsEmptyString
+                                    End If
+                                    With j("list")
+                                        If .ListExists Then
+                                            ProgressPre.ChangeMax(.Count)
+                                            For Each n In .Self
+                                                ProgressPre.Perform()
+                                                postID = n.Value("id")
+                                                postDate = n.Value("postedAt")
+
+                                                If Not _TempPostsList.Contains(postID) Then
+                                                    _TempPostsList.Add(postID)
+                                                Else
+                                                    Exit Sub
+                                                End If
+
+                                                Select Case MyBase.CheckDatesLimit(postDate, DateProvider)
+                                                    Case DateResult.Skip : Continue For
+                                                    Case DateResult.Exit : Exit Sub
+                                                End Select
+
+                                                mediaResult = False
+                                                mediaList = TryCreateMedia(n, postID, postDate, mediaResult)
+                                                If mediaResult Then _TempMediaList.ListAddList(mediaList, LNC)
+                                            Next
+                                        Else
+                                            hasMore = False
+                                        End If
+                                    End With
+                                End If
+                            End Using
+                        End If
+                    End If
+
+                    If hasMore Then
+                        If IsSavedPosts Then tmpCursor = CInt(Cursor.IfNullOrEmpty(0)) + 10
+                        DownloadData(tmpCursor, Token)
+                    End If
+                Catch ex As Exception
+                    If ProcessException(ex, Token, $"data downloading error [{url}]") = 2 Then _complete = False
+                End Try
+            Loop While Not _complete
+        End Sub
+        Private Function TryCreateMedia(ByVal n As EContainer, ByVal PostID As String, Optional ByVal PostDate As String = Nothing,
+                                        Optional ByRef Result As Boolean = False) As List(Of UserMedia)
+            Dim postUrl$, ext$
+            Dim t As UTypes
+            Dim mList As New List(Of UserMedia)
+            Result = False
+            With n("media")
+                If .ListExists Then
+                    For Each m In .Self
+                        postUrl = m.Value({"source"}, "source").IfNullOrEmpty(m.Value("full"))
+                        Select Case m.Value("type")
+                            Case "photo" : t = UTypes.Picture : ext = "jpg"
+                            Case "video" : t = UTypes.Video : ext = "mp4"
+                            Case Else : t = UTypes.Undefined : ext = String.Empty
+                        End Select
+                        If Not t = UTypes.Undefined And Not postUrl.IsEmptyString Then
+                            Dim media As New UserMedia(postUrl, t) With {
+                                        .Post = New UserPost(PostID, AConvert(Of Date)(PostDate, DateProvider, Nothing))}
+                            media.File.Extension = ext
+                            Result = True
+                            mList.Add(media)
+                        End If
+                    Next
+                End If
+            End With
+            Return mList
+        End Function
+        Private Sub GetUserID()
+            Dim path$ = $"/api2/v2/users/{Name}"
+            Dim url$ = String.Format(BaseUrlPattern, path)
+            Try
+                If ID.IsEmptyString AndAlso UpdateSignature(path) Then
+                    Dim r$ = Responser.GetResponse(url)
+                    If Not r.IsEmptyString Then
+                        Using j As EContainer = JsonDocument.Parse(r)
+                            If j.ListExists Then
+                                ID = j.Value("id")
+                                If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
+                                UserSiteNameUpdate(j.Value("name"))
+                                UserDescriptionUpdate(j.Value("about"))
+                                Dim a As Action(Of String) = Sub(ByVal address As String)
+                                                                 If Not address.IsEmptyString Then
+                                                                     Dim f As SFile = address
+                                                                     f.Separator = "\"
+                                                                     f.Path = DownloadContentDefault_GetRootDir()
+                                                                     If Not f.Exists Then GetWebFile(address, f, EDP.None)
+                                                                 End If
+                                                             End Sub
+                                a.Invoke(j.Value("avatar"))
+                                a.Invoke(j.Value("header"))
+                            End If
+                        End Using
+                    End If
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Nothing, $"user info parsing error [{url}]")
+            End Try
+        End Sub
+        Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
+            Const PathPattern$ = "/api2/v2/posts/{0}?skip_users=all"
+            Dim rList As New List(Of Integer)
+            Dim URL$ = String.Empty
+            Try
+                If ContentMissingExists Then
+                    Dim m As UserMedia
+                    Dim mList As List(Of UserMedia)
+                    Dim mediaResult As Boolean
+                    Dim r$, path$, postDate$
+                    Dim j As EContainer
+                    ProgressPre.ChangeMax(_ContentList.Count)
+                    For i% = 0 To _ContentList.Count - 1
+                        ProgressPre.Perform()
+                        If _ContentList(i).State = UStates.Missing Then
+                            m = _ContentList(i)
+                            If Not m.Post.ID.IsEmptyString Then
+                                ThrowAny(Token)
+                                path = String.Format(PathPattern, m.Post.ID)
+                                If UpdateSignature(path) Then
+                                    URL = String.Format(BaseUrlPattern, path)
+                                    r = Responser.GetResponse(URL,, EDP.ReturnValue)
+                                    If Not r.IsEmptyString Then
+                                        j = JsonDocument.Parse(r)
+                                        If Not j Is Nothing Then
+                                            postDate = j.Value("postedAt")
+                                            mediaResult = False
+                                            mList = TryCreateMedia(j, m.Post.ID, postDate, mediaResult)
+                                            If mediaResult Then
+                                                _TempMediaList.ListAddList(mList, LNC)
+                                                rList.Add(i)
+                                                mList.Clear()
+                                            End If
+                                            j.Dispose()
+                                        End If
+                                    End If
+                                End If
+                            End If
+                        End If
+                    Next
+                End If
+            Catch ex As Exception
+                ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
+            Finally
+                If rList.Count > 0 Then
+                    For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next
+                    rList.Clear()
+                End If
+            End Try
+        End Sub
+#End Region
+#Region "DownloadSingleObject"
+        Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
+            Dim postID$ = RegexReplace(Data.URL, RegExPostID)
+            If Not postID.IsEmptyString Then _ContentList.Add(New UserMedia With {.Post = postID, .State = UStates.Missing}) : ReparseMissing(Token)
+        End Sub
+#End Region
+#Region "Auth"
+        Private ReadOnly Property AuthFile As SFile
+            Get
+                Dim f As SFile = MySettings.Responser.File
+                f.Name &= "_Auth"
+                f.Extension = "json"
+                Return f
+            End Get
+        End Property
+        Private Function UpdateSignature(ByVal Path As String, Optional ByVal ForceUpdateAuth As Boolean = False) As Boolean
+            Try
+                If UpdateAuthFile(ForceUpdateAuth) Then
+                    Const nullMsg$ = "The auth parameter is null"
+                    Dim j As EContainer = JsonDocument.Parse(AuthFile.GetText)
+                    Dim pattern$ = j.Value("format")
+                    If pattern.IsEmptyString Then Throw New ArgumentNullException("format", nullMsg)
+                    pattern = pattern.Replace("{}", "{0}").Replace("{:x}", "{1:x}")
+
+                    Dim li%() = j("checksum_indexes").Select(Function(e) CInt(e(0).Value)).ToArray
+
+                    If Not li.ListExists Then Throw New ArgumentNullException("checksum_indexes", nullMsg)
+                    If j.Value("static_param").IsEmptyString Then Throw New ArgumentNullException("static_param", nullMsg)
+                    If j.Value("checksum_constant").IsEmptyString Then Throw New ArgumentNullException("checksum_constant", nullMsg)
+
+                    Dim t$ = ADateTime.ConvertToUnix64(Now.ToUniversalTime).ToString
+                    Dim h$ = String.Join(vbLf, j.Value("static_param"), t, Path, MySettings.HH_USER_ID.Value.ToString)
+
+                    Dim hash$ = GetHashSha1(h)
+                    Dim hashBytes() As Byte = System.Text.Encoding.ASCII.GetBytes(hash)
+                    Dim hashSum% = li.Sum(Function(i) hashBytes(i)) + CInt(j.Value("checksum_constant"))
+                    Dim sign$ = String.Format(pattern, hash, Math.Abs(hashSum))
+
+                    '#If DEBUG Then
+                    'Debug.WriteLine(sign)
+                    'Debug.WriteLine(t)
+                    '#End If
+
+                    Responser.Headers.Add(HeaderSign, sign)
+                    Responser.Headers.Add(HeaderTime, t)
+
+                    j.Dispose()
+                    Return True
+                Else
+                    Return False
+                End If
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"{ToStringForLog()}: UpdateSignature", False)
+            End Try
+        End Function
+        Private Function UpdateAuthFile(ByVal Force As Boolean) As Boolean
+            Const urlOld$ = "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json"
+            Const urlNew$ = "https://raw.githubusercontent.com/DIGITALCRIMINALS/dynamic-rules/main/onlyfans.json"
+            Try
+                If MySettings.LastDateUpdated.AddMinutes(CInt(MySettings.DynamicRulesUpdateInterval.Value)) < Now Or Not AuthFile.Exists Or Force Then
+                    Dim r$ = GetWebString(If(ACheck(Of String)(MySettings.DynamicRules.Value),
+                                             CStr(MySettings.DynamicRules.Value),
+                                             IIf(MySettings.UseOldAuthRules.Value, urlOld, urlNew)),, EDP.ReturnValue)
+                    If Not r.IsEmptyString Then
+                        Using j As EContainer = JsonDocument.Parse(r, EDP.ReturnValue)
+                            If j.ListExists Then
+                                If Not j.Value("format").IsEmptyString And j("checksum_indexes").ListExists And
+                                   Not j.Value("static_param").IsEmptyString And Not j.Value("checksum_constant").IsEmptyString Then _
+                                   TextSaver.SaveTextToFile(r, AuthFile, True, False, EDP.ThrowException) : MySettings.LastDateUpdated = Now
+                            End If
+                        End Using
+                    End If
+                End If
+                Return AuthFile.Exists
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendToLog + EDP.ReturnValue, ex, $"{ToStringForLog()}: UpdateAuthFile", False)
+            End Try
+        End Function
+        Private Function GetHashSha1(ByVal Input As String) As String
+            Dim s As New Security.Cryptography.SHA1CryptoServiceProvider
+            Dim inputBytes() As Byte = System.Text.Encoding.UTF8.GetBytes(Input)
+            Dim hashBytes() As Byte = s.ComputeHash(inputBytes)
+            s.Dispose()
+            Dim result As String = String.Empty
+            For Each b As Byte In hashBytes : result &= b.ToString("x2") : Next
+            Return result
+        End Function
+#End Region
+#Region "DownloadContent"
+        Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
+            DownloadContentDefault(Token)
+        End Sub
+#End Region
+#Region "DownloadingException"
+        Private _DownloadingException_AuthFileUpdate As Boolean = False
+        Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
+                                                          Optional ByVal EObj As Object = Nothing) As Integer
+            If Responser.StatusCode = Net.HttpStatusCode.BadRequest Then
+                If Not _DownloadingException_AuthFileUpdate AndAlso UpdateAuthFile(True) Then
+                    _DownloadingException_AuthFileUpdate = True
+                    Return 2
+                Else
+                    MySettings.SessionAborted = True
+                    MyMainLOG = $"{ToStringForLog()}: OnlyFans credentials expired"
+                    Return 1
+                End If
+            ElseIf Responser.StatusCode = Net.HttpStatusCode.NotFound Then
+                UserExists = False
+                Return 1
+            Else
+                Return 0
+            End If
+        End Function
+#End Region
+#Region "IDisposable Support"
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            If Not disposedValue And disposing Then CCookie.DisposeIfReady(False) : CCookie = Nothing
+            MyBase.Dispose(disposing)
+        End Sub
+#End Region
+    End Class
+End Namespace

SCrawler/API/Reddit/UserData.vb

@@ -585,12 +585,32 @@ Namespace API.Reddit
                             End If
                         End If
                         If Not added And e.Contains("preview") Then
-                            tmpUrl = If(e.ItemF({"preview", "images", eCount, "source", "url"})?.Value, String.Empty)
+                            With e.ItemF({"preview", "images", eCount})
-                            If Not tmpUrl.IsEmptyString Then
+                                If .ListExists Then
-                                _TempMediaList.ListAddValue(MediaFromData(UTypes.Picture, tmpUrl, PostID, PostDate, UserID), LNC)
+                                    tmpType = UTypes.Undefined
-                                _TotalPostsDownloaded += 1
+                                    tmpUrl = String.Empty
-                                added = True
+                                    Dim sv$ = .Value({"source"}, "url")
-                            End If
+                                    If Not sv.IsEmptyString AndAlso sv.Contains(".gif") Then
+                                        tmpUrl = .Value({"variants", "gif", "source"}, "url")
+                                        If Not tmpUrl.IsEmptyString Then tmpType = UTypes.GIF
+                                    End If
+                                    If tmpUrl.IsEmptyString Then
+                                        tmpUrl = .Value({"variants", "mp4", "source"}, "url")
+                                        If Not tmpUrl.IsEmptyString Then tmpType = UTypes.Video
+                                    End If
+                                    If tmpUrl.IsEmptyString Then
+                                        tmpUrl = .Value({"source"}, "url")
+                                        If Not tmpUrl.IsEmptyString Then tmpType = UTypes.Picture
+                                    End If
+                                    If Not tmpUrl.IsEmptyString And Not tmpType = UTypes.Undefined Then
+                                        Dim m As UserMedia = MediaFromData(tmpType, tmpUrl, PostID, PostDate, UserID)
+                                        If tmpType = UTypes.Video Then m.File.Extension = "mp4"
+                                        _TempMediaList.ListAddValue(m, LNC)
+                                        _TotalPostsDownloaded += 1
+                                        added = True
+                                    End If
+                                End If
+                            End With
                         End If
                     End If
                 End If

SCrawler/API/Twitter/EditorExchangeOptions.vb

@@ -7,6 +7,7 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports SCrawler.Plugin.Attributes
+Imports DModels = SCrawler.API.Twitter.UserData.DownloadModels
 Namespace API.Twitter
     Friend Class EditorExchangeOptions
         Private Const DefaultOffset As Integer = 100
@@ -23,6 +24,18 @@ Namespace API.Twitter
                   ToolTip:="Existing files will be checked for duplicates and duplicates removed." & vbCr &
                            "Works only on the first activation 'Use MD5 comparison'.", LeftOffset:=DefaultOffset)>
         Friend Property RemoveExistingDuplicates As Boolean = False
+        <PSetting(Address:=SettingAddress.User,
+                  Caption:="Download model 'Media'",
+                  ToolTip:="Download the data using the 'https://twitter.com/UserName/media' command.", LeftOffset:=DefaultOffset)>
+        Friend Overridable Property DownloadModelMedia As Boolean = False
+        <PSetting(Address:=SettingAddress.User,
+                  Caption:="Download model 'Profile'",
+                  ToolTip:="Download the data using the 'https://twitter.com/UserName' command.", LeftOffset:=DefaultOffset)>
+        Friend Overridable Property DownloadModelProfile As Boolean = False
+        <PSetting(Address:=SettingAddress.User,
+                  Caption:="Download model 'Search'",
+                  ToolTip:="Download the data using the 'https://twitter.com/search?q=from:UserName+include:nativeretweets' command.", LeftOffset:=DefaultOffset)>
+        Friend Overridable Property DownloadModelSearch As Boolean = False
         Private ReadOnly Property MySettings As Object
         Friend Sub New(ByVal s As SiteSettings)
             GifsDownload = s.GifsDownload.Value
@@ -44,6 +57,14 @@ Namespace API.Twitter
             GifsPrefix = u.GifsPrefix
             UseMD5Comparison = u.UseMD5Comparison
             RemoveExistingDuplicates = u.RemoveExistingDuplicates
+            If Not TypeOf u Is Mastodon.UserData Then
+                Dim dm As DModels() = EnumExtract(Of DModels)(u.DownloadModel)
+                If dm.ListExists Then
+                    DownloadModelMedia = dm.Contains(DModels.Media)
+                    DownloadModelProfile = dm.Contains(DModels.Profile)
+                    DownloadModelSearch = dm.Contains(DModels.Search)
+                End If
+            End If
             MySettings = u.HOST.Source
         End Sub
     End Class

SCrawler/API/Twitter/UserData.vb

@@ -18,11 +18,21 @@ Imports UTypes = SCrawler.API.Base.UserMedia.Types
 Namespace API.Twitter
     Friend Class UserData : Inherits UserDataBase
 #Region "XML names"
+        Private Const Name_FirstDownloadComplete As String = "FirstDownloadComplete"
+        Private Const Name_DownloadModel As String = "DownloadModel"
         Private Const Name_GifsDownload As String = "GifsDownload"
         Private Const Name_GifsSpecialFolder As String = "GifsSpecialFolder"
         Private Const Name_GifsPrefix As String = "GifsPrefix"
 #End Region
 #Region "Declarations"
+        Friend Enum DownloadModels As Integer
+            Undefined = 0
+            Media = 1
+            Profile = 2
+            Search = 5
+        End Enum
+        Private FirstDownloadComplete As Boolean = False
+        Friend Property DownloadModel As DownloadModels = DownloadModels.Undefined
         Friend Property GifsDownload As Boolean = True
         Friend Property GifsSpecialFolder As String = String.Empty
         Friend Property GifsPrefix As String = String.Empty
@@ -53,6 +63,10 @@ Namespace API.Twitter
                     GifsPrefix = .GifsPrefix
                     UseMD5Comparison = .UseMD5Comparison
                     RemoveExistingDuplicates = .RemoveExistingDuplicates
+                    DownloadModel = DownloadModels.Undefined
+                    If .DownloadModelMedia Then DownloadModel += DownloadModels.Media
+                    If .DownloadModelProfile Then DownloadModel += DownloadModels.Profile
+                    If .DownloadModelSearch Then DownloadModel += DownloadModels.Search
                 End With
             End If
         End Sub
@@ -62,25 +76,48 @@ Namespace API.Twitter
             _DataNames = New List(Of String)
         End Sub
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
-            If Loading Then
+            With Container
-                GifsDownload = Container.Value(Name_GifsDownload).FromXML(Of Boolean)(True)
+                If Loading Then
-                GifsSpecialFolder = Container.Value(Name_GifsSpecialFolder)
+                    If .Contains(Name_FirstDownloadComplete) Then
-                If Not Container.Contains(Name_GifsPrefix) Then
+                        FirstDownloadComplete = .Value(Name_FirstDownloadComplete).FromXML(Of Boolean)(False)
-                    GifsPrefix = "GIF_"
+                        DownloadModel = .Value(Name_DownloadModel).FromXML(Of Integer)(DownloadModels.Undefined)
+                    Else
+                        FirstDownloadComplete = DownloadedVideos(True) + DownloadedPictures(True) > 0
+                        If .Contains(Name_DownloadModel) Then
+                            DownloadModel = .Value(Name_DownloadModel).FromXML(Of Integer)(DownloadModels.Undefined)
+                        Else
+                            If FirstDownloadComplete Then
+                                If ParseUserMediaOnly Then
+                                    DownloadModel = DownloadModels.Media
+                                Else
+                                    DownloadModel = DownloadModels.Media + DownloadModels.Profile + DownloadModels.Search
+                                End If
+                            Else
+                                DownloadModel = DownloadModels.Undefined
+                            End If
+                        End If
+                    End If
+                    GifsDownload = .Value(Name_GifsDownload).FromXML(Of Boolean)(True)
+                    GifsSpecialFolder = .Value(Name_GifsSpecialFolder)
+                    If Not .Contains(Name_GifsPrefix) Then
+                        GifsPrefix = "GIF_"
+                    Else
+                        GifsPrefix = .Value(Name_GifsPrefix)
+                    End If
+                    UseMD5Comparison = .Value(Name_UseMD5Comparison).FromXML(Of Boolean)(False)
+                    RemoveExistingDuplicates = .Value(Name_RemoveExistingDuplicates).FromXML(Of Boolean)(False)
+                    StartMD5Checked = .Value(Name_StartMD5Checked).FromXML(Of Boolean)(False)
                 Else
-                    GifsPrefix = Container.Value(Name_GifsPrefix)
+                    .Add(Name_FirstDownloadComplete, FirstDownloadComplete.BoolToInteger)
+                    .Add(Name_DownloadModel, CInt(DownloadModel))
+                    .Add(Name_GifsDownload, GifsDownload.BoolToInteger)
+                    .Add(Name_GifsSpecialFolder, GifsSpecialFolder)
+                    .Add(Name_GifsPrefix, GifsPrefix)
+                    .Add(Name_UseMD5Comparison, UseMD5Comparison.BoolToInteger)
+                    .Add(Name_RemoveExistingDuplicates, RemoveExistingDuplicates.BoolToInteger)
+                    .Add(Name_StartMD5Checked, StartMD5Checked.BoolToInteger)
                 End If
-                UseMD5Comparison = Container.Value(Name_UseMD5Comparison).FromXML(Of Boolean)(False)
+            End With
-                RemoveExistingDuplicates = Container.Value(Name_RemoveExistingDuplicates).FromXML(Of Boolean)(False)
-                StartMD5Checked = Container.Value(Name_StartMD5Checked).FromXML(Of Boolean)(False)
-            Else
-                Container.Add(Name_GifsDownload, GifsDownload.BoolToInteger)
-                Container.Add(Name_GifsSpecialFolder, GifsSpecialFolder)
-                Container.Add(Name_GifsPrefix, GifsPrefix)
-                Container.Add(Name_UseMD5Comparison, UseMD5Comparison.BoolToInteger)
-                Container.Add(Name_RemoveExistingDuplicates, RemoveExistingDuplicates.BoolToInteger)
-                Container.Add(Name_StartMD5Checked, StartMD5Checked.BoolToInteger)
-            End If
         End Sub
 #End Region
 #Region "Download functions"
@@ -97,118 +134,191 @@ Namespace API.Twitter
             Dim URL$ = String.Empty
             Dim tCache As CacheKeeper = Nothing
             Try
+                Const entry$ = "entry"
                 Dim PostID$ = String.Empty
                 Dim PostDate$, tmpUserId$
-                Dim j As EContainer
+                Dim i%
-                Dim nn As EContainer
+                Dim dirIndx% = -1
-                Dim NewPostDetected As Boolean = False
+                Dim timelineNode As Predicate(Of EContainer) = Function(ee) ee.Value("type").StringToLower = "timelineaddentries"
-                Dim ExistsDetected As Boolean = False
+                Dim pinNode As Predicate(Of EContainer) = Function(ee) ee.Value("type").StringToLower = "timelinepinentry"
+                Dim entriesNode As Predicate(Of EContainer) = Function(ee) ee.Name = "entries" Or ee.Name = entry
+                Dim sourceIdPredicate As Predicate(Of EContainer) = Function(ee) ee.Name = "source_user_id_str" Or ee.Name = "source_user_id"
+                Dim p As Predicate(Of EContainer)
+                Dim pIndx%
+                Dim isOneNode As Boolean, isPins As Boolean, ExistsDetected As Boolean, userInfoParsed As Boolean = False
+                Dim j As EContainer, rootNode As EContainer, tmpNode As EContainer, nn As EContainer = Nothing
 
-                tCache = New CacheKeeper($"{DownloadContentDefault_GetRootDir()}\_tCache\")
+                Dim __parseContainer As Func(Of EContainer, Boolean) =
+                    Function(ByVal ee As EContainer) As Boolean
+                        If dirIndx <= 1 Then
+                            nn = ee({"content", "itemContent", "tweet_results", "result", "legacy"})
+                        Else
+                            nn = ee
+                        End If
+
+                        If Not nn.ListExists Then nn = ee({"content", "itemContent", "tweet_results", "result", "tweet", "legacy"})
+                        If nn.ListExists Then
+                            PostID = nn.Value("id_str").IfNullOrEmpty(nn.Value("id"))
+
+                            'Date Pattern:
+                            'Sat Jan 01 01:10:15 +0000 2000
+                            If nn.Contains("created_at") Then PostDate = nn("created_at").Value Else PostDate = String.Empty
+                            Select Case CheckDatesLimit(PostDate, Declarations.DateProvider)
+                                Case DateResult.Skip, DateResult.Exit : Return False
+                            End Select
+
+                            If Not _TempPostsList.Contains(PostID) Then
+                                _TempPostsList.Add(PostID)
+                            ElseIf isPins Then
+                                Return False
+                            Else
+                                ExistsDetected = True
+                                Return False
+                            End If
+
+                            tmpUserId = nn({"retweeted_status_result", "result", "legacy", "user_id_str"}).XmlIfNothingValue
+
+                            If tmpUserId.IsEmptyString Then tmpUserId = nn.ItemF({"extended_entities", "media", 0, sourceIdPredicate}).XmlIfNothingValue.
+                                                                                IfNullOrEmpty(nn.Value("user_id")).IfNullOrEmpty(nn.Value("user_id_str")).IfNullOrEmpty("/")
+
+                            If Not ParseUserMediaOnly OrElse (Not ID.IsEmptyString AndAlso tmpUserId = ID) Then ObtainMedia(nn, PostID, PostDate)
+                        End If
+                        Return True
+                    End Function
+
+                tCache = New CacheKeeper($"{DownloadContentDefault_GetRootDir()}\_tCache\") With {
+                    .CacheDeleteError = New ErrorsDescriber(EDP.None) With {.Action = Sub(ee, eex, msg, obj) Settings.Cache.AddPath(tCache)}}
                 If tCache.RootDirectory.Exists(SFO.Path, False) Then tCache.RootDirectory.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.ReturnValue)
                 tCache.Validate()
-                Dim f As SFile = GetTimelineFromGalleryDL(tCache.RootDirectory, Token)
+
-                If Not f.IsEmptyString Then
+                Dim dirs As List(Of SFile) = GetTimelineFromGalleryDL(tCache, Token)
-                    ThrowAny(Token)
+                If dirs.ListExists Then
-                    Dim timelineFiles As List(Of SFile) = SFile.GetFiles(f, "*.txt",, EDP.ReturnValue)
+                    For Each dir As SFile In dirs
-                    If timelineFiles.ListExists Then
+                        dirIndx += 1
-                        Dim i%
+
-                        ResetFileNameProvider(Math.Max(timelineFiles.Count.ToString.Length, 2))
+                        ExistsDetected = False
-                        'rename files
+
-                        For i = 0 To timelineFiles.Count - 1 : timelineFiles(i) = RenameGdlFile(timelineFiles(i), i) : Next
+                        If Not dir.IsEmptyString Then
-                        'parse files
+                            ThrowAny(Token)
-                        For i = 0 To timelineFiles.Count - 1
+                            Dim timelineFiles As List(Of SFile) = SFile.GetFiles(dir, "*.txt",, EDP.ReturnValue)
-                            j = JsonDocument.Parse(timelineFiles(i).GetText)
+                            If timelineFiles.ListExists Then
-                            If Not j Is Nothing Then
+                                ResetFileNameProvider(Math.Max(timelineFiles.Count.ToString.Length, 2))
-                                If i = 0 Then
+                                'rename files
-                                    Dim resValue$ = j.Value({"data", "user", "result"}, "__typename").StringTrim.StringToLower
+                                For i = 0 To timelineFiles.Count - 1 : timelineFiles(i) = RenameGdlFile(timelineFiles(i), i) : Next
-                                    If resValue.IsEmptyString Then
+                                'parse files
-                                        UserExists = False
+                                For i = 0 To timelineFiles.Count - 1
-                                        j.Dispose()
+                                    j = JsonDocument.Parse(timelineFiles(i).GetText)
-                                        Exit Sub
+                                    If Not j Is Nothing Then
-                                    ElseIf resValue = "userunavailable" Then
+                                        If i = 0 Then
-                                        UserSuspended = True
+                                            If Not userInfoParsed Then
-                                        j.Dispose()
+                                                userInfoParsed = True
-                                        Exit Sub
+                                                Dim resValue$ = j.Value({"data", "user", "result"}, "__typename").StringTrim.StringToLower
-                                    Else
+                                                If resValue.IsEmptyString Then
-                                        With j({"data", "user", "result"})
+                                                    UserExists = False
-                                            If .ListExists Then
+                                                    j.Dispose()
-                                                If ID.IsEmptyString Then
+                                                    Exit Sub
-                                                    ID = .Value("rest_id")
+                                                ElseIf resValue = "userunavailable" Then
-                                                    If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
+                                                    UserSuspended = True
-                                                End If
+                                                    j.Dispose()
-                                                With .Item({"legacy"})
+                                                    Exit Sub
-                                                    If .ListExists Then
+                                                Else
-                                                        If .Value("screen_name").StringToLower = Name.ToLower Then
+                                                    With j({"data", "user", "result"})
-                                                            UserSiteNameUpdate(.Value("name"))
+                                                        If .ListExists Then
-                                                            UserDescriptionUpdate(.Value("description"))
+                                                            If ID.IsEmptyString Then
-                                                            Dim __getImage As Action(Of String) = Sub(ByVal img As String)
+                                                                ID = .Value("rest_id")
-                                                                                                      If Not img.IsEmptyString Then
+                                                                If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
-                                                                                                          Dim __imgFile As SFile = UrlFile(img, True)
-                                                                                                          If Not __imgFile.Name.IsEmptyString Then
-                                                                                                              If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
-                                                                                                              __imgFile.Path = MyFile.CutPath.Path
-                                                                                                              If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
-                                                                                                              If __imgFile.Exists Then IconBannerDownloaded = True
-                                                                                                          End If
-                                                                                                      End If
-                                                                                                  End Sub
-                                                            Dim icon$ = .Value("profile_image_url_https")
-                                                            If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty)
-                                                            If DownloadIconBanner Then
-                                                                __getImage.Invoke(.Value("profile_banner_url"))
-                                                                __getImage.Invoke(icon)
                                                             End If
+                                                            With .Item({"legacy"})
+                                                                If .ListExists Then
+                                                                    If .Value("screen_name").StringToLower = Name.ToLower Then
+                                                                        UserSiteNameUpdate(.Value("name"))
+                                                                        UserDescriptionUpdate(.Value("description"))
+                                                                        Dim __getImage As Action(Of String) = Sub(ByVal img As String)
+                                                                                                                  If Not img.IsEmptyString Then
+                                                                                                                      Dim __imgFile As SFile = UrlFile(img, True)
+                                                                                                                      If Not __imgFile.Name.IsEmptyString Then
+                                                                                                                          If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
+                                                                                                                          __imgFile.Path = MyFile.CutPath.Path
+                                                                                                                          If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
+                                                                                                                          If __imgFile.Exists Then IconBannerDownloaded = True
+                                                                                                                      End If
+                                                                                                                  End If
+                                                                                                              End Sub
+                                                                        Dim icon$ = .Value("profile_image_url_https")
+                                                                        If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty)
+                                                                        If DownloadIconBanner Then
+                                                                            __getImage.Invoke(.Value("profile_banner_url"))
+                                                                            __getImage.Invoke(icon)
+                                                                        End If
+                                                                    End If
+                                                                End If
+                                                            End With
                                                         End If
-                                                    End If
+                                                    End With
-                                                End With
+                                                End If
                                             End If
-                                        End With
+                                        Else
-                                    End If
+                                            For pIndx = 0 To IIf(dirIndx < 2, 1, 0)
-                                Else
+                                                Select Case dirIndx
-                                    With j({"globalObjects", "tweets"})
+                                                    Case 0, 1
-                                        If .ListExists Then
+                                                        rootNode = j({"data", "user", "result", "timeline_v2", "timeline", "instructions"})
-                                            ProgressPre.ChangeMax(.Count)
+                                                        If rootNode.ListExists Then
-                                            For Each nn In .Self
+                                                            p = If(pIndx = 0, pinNode, timelineNode)
-                                                ProgressPre.Perform()
+                                                            isPins = pIndx = 0
-                                                If nn.Count > 0 Then
+                                                            rootNode = rootNode.Find(p, False)
-                                                    PostID = nn.Value("id")
+                                                            If rootNode.ListExists Then rootNode = rootNode.Find(entriesNode, False)
+                                                        End If
+                                                    Case Else
+                                                        isPins = False
+                                                        rootNode = j({"globalObjects", "tweets"})
+                                                End Select
 
-                                                    'Date Pattern:
+                                                If rootNode.ListExists Then
-                                                    'Sat Jan 01 01:10:15 +0000 2000
+                                                    With rootNode
-                                                    If nn.Contains("created_at") Then PostDate = nn("created_at").Value Else PostDate = String.Empty
+                                                        isOneNode = dirIndx < 2 AndAlso .Name = entry
-                                                    Select Case CheckDatesLimit(PostDate, Declarations.DateProvider)
+                                                        ProgressPre.ChangeMax(If(isOneNode, 1, .Count))
-                                                        Case DateResult.Skip : Continue For
+                                                        If isOneNode Then
-                                                        Case DateResult.Exit : Exit Sub
+                                                            ProgressPre.Perform()
-                                                    End Select
+                                                            If Not __parseContainer(.Self) Then Exit For
-
+                                                        Else
-                                                    If Not _TempPostsList.Contains(PostID) Then
+                                                            For Each tmpNode In .Self
-                                                        NewPostDetected = True
+                                                                ProgressPre.Perform()
-                                                        _TempPostsList.Add(PostID)
+                                                                If Not __parseContainer(tmpNode) Then Exit For
-                                                    Else
+                                                            Next
-                                                        ExistsDetected = True
+                                                        End If
-                                                        Continue For
+                                                    End With
-                                                    End If
-
-                                                    tmpUserId = nn.ItemF({"extended_entities", "media", 0, "source_user_id"}).
-                                                                XmlIfNothingValue.IfNullOrEmpty(nn.Value("user_id")).IfNullOrEmpty("/")
-
-                                                    If Not ParseUserMediaOnly OrElse (Not ID.IsEmptyString AndAlso tmpUserId = ID) Then _
-                                                       ObtainMedia(nn, PostID, PostDate)
                                                 End If
                                             Next
+
+                                            'TODO: Twitter: is this line needed?
+                                            If ExistsDetected And i = 1 Then Exit For Else ExistsDetected = False
                                         End If
-                                    End With
+                                        j.Dispose()
-                                End If
+                                    End If
-                                j.Dispose()
+                                Next
+                                timelineFiles.Clear()
                             End If
-                        Next
+                        End If
+                    Next
+                    dirs.Clear()
+                End If
+                ThrowAny(Token)
+                If Not FirstDownloadComplete Then
+                    _ForceSaveUserInfo = True
+                    If DownloadModel = DownloadModels.Undefined Then
+                        If ParseUserMediaOnly Then
+                            DownloadModel = DownloadModels.Media
+                        Else
+                            DownloadModel = DownloadModels.Media + DownloadModels.Profile + DownloadModels.Search
+                        End If
                     End If
                 End If
+                FirstDownloadComplete = True
             Catch ex As Exception
                 ProcessException(ex, Token, $"data downloading error [{URL}]")
             Finally
                 If Not tCache Is Nothing Then tCache.Dispose()
+                If _TempPostsList.Count > 0 Then _TempPostsList.Sort()
             End Try
         End Sub
         Private Sub DownloadData_SavedPosts(ByVal Token As CancellationToken)
@@ -249,8 +359,10 @@ Namespace API.Twitter
 #End Region
 #Region "Obtain media"
         Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown)
-            Dim s As EContainer = e.ItemF({"extended_entities", "media"})
+            Dim s As EContainer = e({"extended_entities", "media"})
-            If s Is Nothing OrElse s.Count = 0 Then s = e.ItemF({"retweeted_status", "extended_entities", "media"})
+            If If(s?.Count, 0) = 0 Then s = e({"retweeted_status", "extended_entities", "media"})
+            If If(s?.Count, 0) = 0 Then s = e({"retweeted_status_result", "result", "legacy", "extended_entities", "media"})
+
             If If(s?.Count, 0) > 0 Then
                 Dim mUrl$
                 For Each m As EContainer In s
@@ -353,7 +465,7 @@ Namespace API.Twitter
             Friend Sub New(ByVal Dir As SFile, ByVal _Token As CancellationToken)
                 MyBase.New
                 Commands.Clear()
-                ChangeDirectory(Dir)
+                If Not Dir.IsEmptyString Then ChangeDirectory(Dir)
                 Token = _Token
             End Sub
             Protected Overrides Async Function Validate(ByVal Value As String) As Task
@@ -384,7 +496,7 @@ Namespace API.Twitter
                         End If
                         command &= URL
                         '#If DEBUG Then
-                        '                        Debug.WriteLine(command)
+                        'Debug.WriteLine(command)
                         '#End If
                         batch.Execute(command)
                     End Using
@@ -395,27 +507,59 @@ Namespace API.Twitter
                 Return ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{ToStringForLog()}: GetDataFromGalleryDL({command})")
             End Try
         End Function
-        Private Function GetTimelineFromGalleryDL(ByVal Cache As CacheKeeper, ByVal Token As CancellationToken) As SFile
+        Private Function GetTimelineFromGalleryDL(ByVal Cache As CacheKeeper, ByVal Token As CancellationToken) As List(Of SFile)
             Dim command$ = String.Empty
             Try
-                Dim conf As SFile = $"{Cache.NewPath.PathWithSeparator}TwitterGdlConfig.conf"
+                Dim confCache As CacheKeeper = Cache.NewInstance(Of BatchFileExchanger)
+                Dim conf As SFile = $"{confCache.RootDirectory.PathWithSeparator}TwitterGdlConfig.conf"
                 Dim confText$ = "{""extractor"":{""cookies"": """ & MySettings.CookiesNetscapeFile.ToString.Replace("\", "/") &
-                                """,""cookies-update"": false,""twitter"":{""cards"": false,""conversations"": false,""pinned"": false,""quoted"": false,""replies"": true,""retweets"": true,""strategy"": null,""text-tweets"": false,""twitpic"": false,""unique"": true,""users"": ""timeline"",""videos"": true}}}"
+                                """,""cookies-update"": false,""twitter"":{""cards"": false,""conversations"": true,""pinned"": false,""quoted"": false,""replies"": true,""retweets"": true,""strategy"": null,""text-tweets"": false,""twitpic"": false,""unique"": true,""users"": ""timeline"",""videos"": true}}}"
                 If conf.Exists(SFO.Path, True, EDP.ThrowException) Then TextSaver.SaveTextToFile(confText, conf)
                 If Not conf.Exists Then Throw New IO.FileNotFoundException("Can't find Twitter GDL config file", conf)
 
-                command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
+                Dim outList As New List(Of SFile)
-                command &= GdlGetIdFilterString()
+                Dim rootDir As CacheKeeper = Cache.NewInstance
-                command &= $"https://twitter.com/search?q=from:{Name}+include:nativeretweets"
+                Dim dir As SFile
-                Dim dir As SFile = Cache.NewPath
+                Dim dm As List(Of DownloadModels) = EnumExtract(Of DownloadModels)(DownloadModel).ListIfNothing
-                dir.Exists(SFO.Path, True, EDP.ThrowException)
+                Dim process As Boolean
-                '#If DEBUG Then
+                Dim bProcess As Boolean = DownloadModel = DownloadModels.Undefined Or Not FirstDownloadComplete
-                '                Debug.WriteLine(command)
+
-                '#End If
+                Using tgdl As New TwitterGDL(Nothing, Token) With {
-                Using tgdl As New TwitterGDL(dir, Token) With {.TempPostsList = _TempPostsList} : tgdl.Execute(command) : End Using
+                    .TempPostsList = _TempPostsList,
-                Return dir
+                    .AutoClear = True,
+                    .AutoReset = True,
+                    .CommandPermanent = $"chcp {BatchExecutor.UnicodeEncoding}",
+                    .FileExchanger = confCache
+                }
+                    tgdl.FileExchanger.DeleteCacheOnDispose = False
+                    tgdl.FileExchanger.DeleteRootOnDispose = False
+                    For i As Byte = 0 To 2
+                        dir = rootDir.NewPath
+                        dir.Exists(SFO.Path, True, EDP.ThrowException)
+                        outList.Add(dir)
+                        tgdl.ChangeDirectory(dir)
+                        command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
+                        command &= GdlGetIdFilterString()
+                        Select Case i
+                            Case 0 : command &= $"https://twitter.com/{Name}/media" : process = bProcess Or dm.Contains(DownloadModels.Media)
+                            Case 1 : command &= $"https://twitter.com/{Name}" : process = bProcess Or dm.Contains(DownloadModels.Profile)
+                            Case 2 : command &= $"https://twitter.com/search?q=from:{Name}+include:nativeretweets" : process = bProcess Or dm.Contains(DownloadModels.Search)
+                            Case Else : process = False
+                        End Select
+                        '#If DEBUG Then
+                        'Debug.WriteLine(command)
+                        '#End If
+                        ThrowAny(Token)
+                        If process Then tgdl.Execute(command)
+                        ThrowAny(Token)
+                    Next
+                End Using
+                dm.Clear()
+
+                Return outList
             Catch ex As Exception
-                Return ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{ToStringForLog()}: GetTimelineFromGalleryDL({command})")
+                ProcessException(ex, Token, $"{ToStringForLog()}: GetTimelineFromGalleryDL({command})")
+                Return Nothing
             End Try
         End Function
         Private Function GdlGetIdFilterString() As String

SCrawler/API/Xhamster/UserData.vb

@@ -10,6 +10,7 @@ Imports System.Threading
 Imports SCrawler.API.Base
 Imports SCrawler.API.YouTube.Objects
 Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Documents.JSON
@@ -287,8 +288,11 @@ Namespace API.Xhamster
             End Try
         End Function
         Private Overloads Function GetM3U8(ByRef m As UserMedia, ByVal j As EContainer) As Boolean
-            Dim url$ = j.Value({"xplayerSettings", "sources", "hls"}, "url")
+            Dim node As EContainer = j({"xplayerSettings", "sources", "hls"})
-            If Not url.IsEmptyString Then m.URL = url : m.Type = UTypes.m3u8 : Return True
+            If node.ListExists Then
+                Dim url$ = node.GetNode({New NodeParams("url", True, True, True, True, 2)})
+                If Not url.IsEmptyString Then m.URL = url : m.Type = UTypes.m3u8 : Return True
+            End If
             Return False
         End Function
 #End Region

SCrawler/Download/ActiveDownloadingProgress.vb

@@ -62,8 +62,7 @@ Namespace DownloadObjects
                                                   .RowStyles.Add(New RowStyle(SizeType.Absolute, RowHeight))
                                                   .RowCount += 1
                                                   JobsList.Add(New DownloadProgress(j))
-                                                  AddHandler JobsList.Last.ProgressMaximumChanged, AddressOf Jobs_ProgressMaximumChanged
+                                                  AddHandler JobsList.Last.ProgressChanged, AddressOf Jobs_ProgressChanged
-                                                  AddHandler JobsList.Last.ProgressMaximum0Changed, AddressOf Jobs_ProgressMaximum0Changed
                                                   .Controls.Add(JobsList.Last.Get, 0, .RowStyles.Count - 1)
                                               End With
                                           Next
@@ -84,16 +83,18 @@ Namespace DownloadObjects
                               End Sub
             If TP_MAIN.InvokeRequired Then TP_MAIN.Invoke(a) Else a.Invoke
         End Sub
-        Private Sub Jobs_ProgressMaximumChanged()
+        Private Sub Jobs_ProgressChanged(ByVal Main As Boolean, ByVal IsMaxValue As Boolean, ByVal IsDone As Boolean)
             If JobsList.Count > 0 And Not DisableProgressChange Then
-                MainProgress.Maximum = JobsList.Sum(Function(j) CLng(j.Job.Progress.Maximum))
+                If Main Then
-                MainProgress.Value = Math.Max(JobsList.Sum(Function(j) CLng(j.Job.Progress.Value)) - 1, 0)
+                    MainProgress.Maximum = JobsList.Sum(Function(j) CLng(j.Job.Progress.Maximum))
-                If MainProgress.Value > 0 Then MainProgress.Perform()
+                    MainProgress.Value = Math.Max(JobsList.Sum(Function(j) CLng(j.Job.Progress.Value)) - 1, 0)
+                    If MainProgress.Value > 0 Then MainProgress.Perform()
+                Else
+                    MainProgress.Maximum0 = JobsList.Sum(Function(j) CLng(DirectCast(j.Job.Progress, MyProgressExt).Maximum0))
+                    MainProgress.Value0 = Math.Max(JobsList.Sum(Function(j) CLng(DirectCast(j.Job.Progress, MyProgressExt).Value0)) - 1, 0)
+                    If MainProgress.Value0 > 0 Then MainProgress.Perform0()
+                End If
             End If
         End Sub
-        Private Sub Jobs_ProgressMaximum0Changed()
-            If JobsList.Count > 0 And Not DisableProgressChange Then _
-               MainProgress.Maximum0 = JobsList.Sum(Function(j) CLng(DirectCast(j.Job.Progress, MyProgressExt).Maximum0))
-        End Sub
     End Class
 End Namespace

SCrawler/Download/DownloadProgress.vb

@@ -13,8 +13,7 @@ Namespace DownloadObjects
     Friend Class DownloadProgress : Implements IDisposable
 #Region "Events"
         Friend Event DownloadDone As NotificationEventHandler
-        Friend Event ProgressMaximumChanged()
+        Friend Event ProgressChanged(ByVal Main As Boolean, ByVal IsMaxValue As Boolean, ByVal IsDone As Boolean)
-        Friend Event ProgressMaximum0Changed()
 #End Region
 #Region "Declarations"
 #Region "Controls"
@@ -24,6 +23,7 @@ Namespace DownloadObjects
         Private WithEvents BTT_STOP As Button
         Private WithEvents BTT_OPEN As Button
         Private ReadOnly PR_MAIN As ProgressBar
+        Private ReadOnly PR_PRE As ProgressBar
         Private ReadOnly LBL_INFO As Label
         Private ReadOnly Icon As PictureBox
 #End Region
@@ -39,6 +39,7 @@ Namespace DownloadObjects
             TP_MAIN.ColumnCount = 1
             TP_CONTROLS = New TableLayoutPanel With {.Margin = New Padding(0), .Dock = DockStyle.Fill}
             PR_MAIN = New ProgressBar With {.Dock = DockStyle.Fill}
+            PR_PRE = New ProgressBar With {.Dock = DockStyle.Fill}
             LBL_INFO = New Label With {.Text = String.Empty, .Dock = DockStyle.Fill}
             Icon = New PictureBox With {
                 .SizeMode = PictureBoxSizeMode.Zoom,
@@ -66,7 +67,8 @@ Namespace DownloadObjects
                 With TP_CONTROLS
                     .ColumnStyles.Add(New ColumnStyle(SizeType.Absolute, 30))
                     .ColumnStyles.Add(New ColumnStyle(SizeType.Absolute, 30))
-                    .ColumnStyles.Add(New ColumnStyle(SizeType.Absolute, 150))
+                    .ColumnStyles.Add(New ColumnStyle(SizeType.Absolute, 75))
+                    .ColumnStyles.Add(New ColumnStyle(SizeType.Absolute, 75)) '150
                     .ColumnStyles.Add(New ColumnStyle(SizeType.Percent, 100))
                     .ColumnCount = .ColumnStyles.Count
                     .RowStyles.Add(New RowStyle(SizeType.Percent, 100))
@@ -74,8 +76,9 @@ Namespace DownloadObjects
                     With .Controls
                         If Not img Is Nothing Then .Add(Icon, 0, 0)
                         .Add(BTT_STOP, 1, 0)
-                        .Add(PR_MAIN, 2, 0)
+                        .Add(PR_PRE, 2, 0)
-                        .Add(LBL_INFO, 3, 0)
+                        .Add(PR_MAIN, 3, 0)
+                        .Add(LBL_INFO, 4, 0)
                     End With
                 End With
                 TP_MAIN.Controls.Add(TP_CONTROLS, 0, 0)
@@ -90,7 +93,8 @@ Namespace DownloadObjects
                         .Add(New ColumnStyle(SizeType.Absolute, 30))
                         .Add(New ColumnStyle(SizeType.Absolute, 30))
                         .Add(New ColumnStyle(SizeType.Absolute, 30))
-                        .Add(New ColumnStyle(SizeType.Percent, 100))
+                        .Add(New ColumnStyle(SizeType.Percent, 50))
+                        .Add(New ColumnStyle(SizeType.Percent, 50)) '100
                     End With
                     .ColumnCount = .ColumnStyles.Count
                     .RowStyles.Add(New RowStyle(SizeType.Percent, 50))
@@ -100,7 +104,8 @@ Namespace DownloadObjects
                         .Add(BTT_START, 1, 0)
                         .Add(BTT_STOP, 2, 0)
                         .Add(BTT_OPEN, 3, 0)
-                        .Add(PR_MAIN, 4, 0)
+                        .Add(PR_PRE, 4, 0)
+                        .Add(PR_MAIN, 5, 0)
                     End With
                 End With
                 With TP_MAIN
@@ -115,12 +120,14 @@ Namespace DownloadObjects
             End If
 
             With Job
-                .Progress = New MyProgressExt(PR_MAIN, LBL_INFO) With {.ResetProgressOnMaximumChanges = False}
+                .Progress = New MyProgressExt(PR_MAIN, PR_PRE, LBL_INFO) With {.ResetProgressOnMaximumChanges = False}
                 With DirectCast(.Progress, MyProgressExt)
                     AddHandler .ProgressChanged, AddressOf JobProgress_ProgressChanged
                     AddHandler .MaximumChanged, AddressOf JobProgress_MaximumChanged
                     AddHandler .Maximum0Changed, AddressOf JobProgress_Maximum0Changed
                     AddHandler .Progress0Changed, AddressOf JobProgress_Progress0Changed
+                    AddHandler .ProgressCompleted, AddressOf JobProgress_Done
+                    AddHandler .Progress0Completed, AddressOf JobProgress_Done0
                 End With
             End With
 
@@ -184,22 +191,22 @@ Namespace DownloadObjects
 #End Region
 #Region "Progress, Jobs count"
         Private Sub JobProgress_MaximumChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            RaiseEvent ProgressMaximumChanged()
+            If Not Job.Type = Download.SavedPosts Then RaiseEvent ProgressChanged(True, True, False)
         End Sub
         Private Sub JobProgress_Maximum0Changed(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            RaiseEvent ProgressMaximum0Changed()
+            If Not Job.Type = Download.SavedPosts Then RaiseEvent ProgressChanged(False, True, False)
         End Sub
         Private Sub JobProgress_ProgressChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            If Not Job.Type = Download.SavedPosts Then
+            If Not Job.Type = Download.SavedPosts Then MainProgress.Perform()
-                MainProgress.Value = DirectCast(Sender, MyProgressExt).Value
-                MainProgress.Perform(0)
-            End If
         End Sub
         Private Sub JobProgress_Progress0Changed(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            If Not Job.Type = Download.SavedPosts Then
+            If Not Job.Type = Download.SavedPosts Then MainProgress.Perform0()
-                MainProgress.Value0 = DirectCast(Sender, MyProgressExt).Value0
+        End Sub
-                MainProgress.Perform0(0)
+        Private Sub JobProgress_Done(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            End If
+            If Not Job.Type = Download.SavedPosts Then RaiseEvent ProgressChanged(True, False, True)
+        End Sub
+        Private Sub JobProgress_Done0(ByVal Sender As Object, ByVal e As ProgressEventArgs)
+            If Not Job.Type = Download.SavedPosts Then RaiseEvent ProgressChanged(False, False, True)
         End Sub
 #End Region
 #Region "IDisposable Support"

SCrawler/Editors/SiteEditorForm.Designer.vb

@@ -54,7 +54,7 @@ Namespace Editors
             'CONTAINER_MAIN.ContentPanel
             '
             CONTAINER_MAIN.ContentPanel.Controls.Add(Me.TP_MAIN)
-            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(544, 218)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(544, 243)
             CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
             CONTAINER_MAIN.LeftToolStripPanelVisible = False
             CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
@@ -84,7 +84,7 @@ Namespace Editors
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            Me.TP_MAIN.Size = New System.Drawing.Size(544, 218)
+            Me.TP_MAIN.Size = New System.Drawing.Size(544, 243)
             Me.TP_MAIN.TabIndex = 0
             '
             'TXT_PATH
@@ -135,7 +135,7 @@ Namespace Editors
             Me.TP_SITE_PROPS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             Me.TP_SITE_PROPS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             Me.TP_SITE_PROPS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            Me.TP_SITE_PROPS.Size = New System.Drawing.Size(538, 78)
+            Me.TP_SITE_PROPS.Size = New System.Drawing.Size(538, 103)
             Me.TP_SITE_PROPS.TabIndex = 5
             '
             'TXT_PATH_SAVED_POSTS

SCrawler/Editors/SiteEditorForm.vb

@@ -90,6 +90,11 @@ Namespace Editors
                         End If
 
                         If .PropList.Count > 0 Then
+                            With TP_SITE_PROPS
+                                With .RowStyles : .RemoveAt(.Count - 1) : End With
+                                .RowCount -= 1
+                            End With
+
                             Dim laAdded As Boolean = False
                             Dim loAdded As Boolean = False
                             Dim pArr() As Boolean
@@ -134,6 +139,19 @@ Namespace Editors
                         CH_GET_USER_MEDIA_ONLY.Padding = New PaddingE(CH_GET_USER_MEDIA_ONLY.Padding) With {.Left = offset}
                         If c > 0 Or h <> 0 Then
                             Dim ss As New Size(Size.Width, Size.Height + h + c)
+                            Dim minScrh% = Screen.AllScreens.Min(Function(scr) scr.WorkingArea.Height)
+
+                            If ss.Height >= minScrh - 20 Then
+                                ss.Height = minScrh - 40
+                                With TP_SITE_PROPS
+                                    .AutoScroll = True
+                                    Dim p As Padding = .Padding
+                                    p.Right = 3
+                                    .Padding = p
+                                    .PerformLayout()
+                                End With
+                            End If
+
                             MinimumSize = ss
                             Size = ss
                             MaximumSize = ss

SCrawler/MainFrame.Designer.vb

@@ -92,6 +92,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_DONATE = New System.Windows.Forms.ToolStripButton()
         Me.Toolbar_BOTTOM = New System.Windows.Forms.StatusStrip()
         Me.BTT_PR_INFO = New System.Windows.Forms.ToolStripStatusLabel()
+        Me.PR_PRE = New System.Windows.Forms.ToolStripProgressBar()
         Me.PR_MAIN = New System.Windows.Forms.ToolStripProgressBar()
         Me.LBL_JOBS_COUNT = New System.Windows.Forms.ToolStripStatusLabel()
         Me.LBL_STATUS = New System.Windows.Forms.ToolStripStatusLabel()
@@ -122,10 +123,10 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_TRAY_SILENT_MODE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_FEED_SHOW = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_CHANNELS = New System.Windows.Forms.ToolStripMenuItem()
+        Me.BTT_TRAY_DOWNLOADER = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_SHOW_HIDE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_CLOSE_NO_SCRIPT = New System.Windows.Forms.ToolStripMenuItem()
-        Me.BTT_TRAY_DOWNLOADER = New System.Windows.Forms.ToolStripMenuItem()
         SEP_1 = New System.Windows.Forms.ToolStripSeparator()
         SEP_2 = New System.Windows.Forms.ToolStripSeparator()
         CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator()
@@ -633,7 +634,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         '
         'Toolbar_BOTTOM
         '
-        Me.Toolbar_BOTTOM.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_PR_INFO, Me.PR_MAIN, Me.LBL_JOBS_COUNT, Me.LBL_STATUS})
+        Me.Toolbar_BOTTOM.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_PR_INFO, Me.PR_PRE, Me.PR_MAIN, Me.LBL_JOBS_COUNT, Me.LBL_STATUS})
         Me.Toolbar_BOTTOM.Location = New System.Drawing.Point(0, 439)
         Me.Toolbar_BOTTOM.Name = "Toolbar_BOTTOM"
         Me.Toolbar_BOTTOM.Size = New System.Drawing.Size(934, 22)
@@ -647,6 +648,12 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_PR_INFO.Padding = New System.Windows.Forms.Padding(0, 0, 3, 0)
         Me.BTT_PR_INFO.Size = New System.Drawing.Size(19, 17)
         '
+        'PR_PRE
+        '
+        Me.PR_PRE.Name = "PR_PRE"
+        Me.PR_PRE.Size = New System.Drawing.Size(100, 16)
+        Me.PR_PRE.Visible = False
+        '
         'PR_MAIN
         '
         Me.PR_MAIN.Name = "PR_MAIN"
@@ -865,6 +872,13 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_TRAY_CHANNELS.Size = New System.Drawing.Size(170, 22)
         Me.BTT_TRAY_CHANNELS.Text = "Channels"
         '
+        'BTT_TRAY_DOWNLOADER
+        '
+        Me.BTT_TRAY_DOWNLOADER.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
+        Me.BTT_TRAY_DOWNLOADER.Name = "BTT_TRAY_DOWNLOADER"
+        Me.BTT_TRAY_DOWNLOADER.Size = New System.Drawing.Size(170, 22)
+        Me.BTT_TRAY_DOWNLOADER.Text = "Downloader"
+        '
         'BTT_TRAY_SHOW_HIDE
         '
         Me.BTT_TRAY_SHOW_HIDE.Image = Global.SCrawler.My.Resources.Resources.ApplicationPic_16
@@ -893,13 +907,6 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_TRAY_CLOSE_NO_SCRIPT.ToolTipText = "Close the program without executing the script"
         Me.BTT_TRAY_CLOSE_NO_SCRIPT.Visible = False
         '
-        'BTT_TRAY_DOWNLOADER
-        '
-        Me.BTT_TRAY_DOWNLOADER.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
-        Me.BTT_TRAY_DOWNLOADER.Name = "BTT_TRAY_DOWNLOADER"
-        Me.BTT_TRAY_DOWNLOADER.Size = New System.Drawing.Size(170, 22)
-        Me.BTT_TRAY_DOWNLOADER.Text = "Downloader"
-        '
         'MainFrame
         '
         Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
@@ -1005,4 +1012,5 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
     Friend WithEvents MENU_DOWN_ALL As ToolStripDropDownButton
     Private WithEvents BTT_TRAY_CHANNELS As ToolStripMenuItem
     Private WithEvents BTT_TRAY_DOWNLOADER As ToolStripMenuItem
+    Private WithEvents PR_PRE As ToolStripProgressBar
 End Class

SCrawler/MainFrame.vb

@@ -58,7 +58,7 @@ Public Class MainFrame
         YouTube.MyCache = Settings.Cache
         YouTube.MyYouTubeSettings = New YouTube.YTSettings_Internal
         UpdateYouTubeSettings()
-        MainProgress = New MyProgressExt(Toolbar_BOTTOM, PR_MAIN, LBL_STATUS, "Downloading profiles' data") With {
+        MainProgress = New MyProgressExt(Toolbar_BOTTOM, PR_MAIN, PR_PRE, LBL_STATUS, "Downloading profiles' data") With {
             .ResetProgressOnMaximumChanges = False, .Visible = False}
         Downloader = New TDownloader
         InfoForm = New DownloadedInfoForm

SCrawler/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.6.5.0")>
+<Assembly: AssemblyVersion("2023.6.19.0")>
-<Assembly: AssemblyFileVersion("2023.6.5.0")>
+<Assembly: AssemblyFileVersion("2023.6.19.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/MyProgressExt.vb

@@ -21,41 +21,23 @@ Friend Class PreProgress : Implements IDisposable
             ProgressExists = True
         End If
     End Sub
-    Private _Maximum As Integer = 0
     Friend Sub ChangeMax(ByVal Value As Integer, Optional ByVal Add As Boolean = True)
         If Ready Then
             If Add Then
-                _Maximum += Value
                 If Value > 0 Then Progress.Maximum0 += Value
             Else
-                _Maximum = Value
                 Progress.Maximum0 = Value
             End If
         End If
     End Sub
-    Private CumulVal As Integer = 0
     Friend Sub Perform(Optional ByVal Value As Integer = 1)
-        If Ready Then
+        If Ready Then Progress.Perform0(Value)
-            CumulVal += Value
-            Progress.Perform0(Value)
-        End If
     End Sub
     Friend Sub Reset()
-        _Maximum = 0
+        If Ready Then Progress.Reset0()
-        CumulVal = 0
     End Sub
     Friend Sub Done()
-        If Ready Then
+        If Ready Then Progress.Done0()
-            Dim v# = _Maximum - CumulVal
-            If v > 0 Then
-                With Progress
-                    If v + .Value0 > .Maximum0 Then v = .Maximum0 - .Value0
-                    If v < 0 Then v = 0
-                    .Perform0(v)
-                    Reset()
-                End With
-            End If
-        End If
     End Sub
 #Region "IDisposable Support"
     Private disposedValue As Boolean = False
@@ -85,14 +67,7 @@ Friend Class MyProgressExt : Inherits MyProgress
             _Progress0ChangedEventHandlers.Remove(h)
         End RemoveHandler
         RaiseEvent(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            If _Progress0ChangedEventHandlers.Count > 0 Then
+            InvokeHandlers(_Progress0ChangedEventHandlers, Sender, e)
-                Try
-                    For i% = 0 To _Progress0ChangedEventHandlers.Count - 1
-                        Try : _Progress0ChangedEventHandlers(i).Invoke(Sender, e) : Catch : End Try
-                    Next
-                Catch
-                End Try
-            End If
         End RaiseEvent
     End Event
     Private ReadOnly _Maximum0ChangedEventHandlers As List(Of EventHandler(Of ProgressEventArgs))
@@ -104,72 +79,99 @@ Friend Class MyProgressExt : Inherits MyProgress
             _Maximum0ChangedEventHandlers.Remove(h)
         End RemoveHandler
         RaiseEvent(ByVal Sender As Object, ByVal e As ProgressEventArgs)
-            If _Maximum0ChangedEventHandlers.Count > 0 Then
+            InvokeHandlers(_Maximum0ChangedEventHandlers, Sender, e)
-                Try
-                    For i% = 0 To _Maximum0ChangedEventHandlers.Count - 1
-                        Try : _Maximum0ChangedEventHandlers(i).Invoke(Sender, e) : Catch : End Try
-                    Next
-                Catch
-                End Try
-            End If
         End RaiseEvent
     End Event
+    Private ReadOnly _Progress0CompletedEventHandlers As List(Of EventHandler(Of ProgressEventArgs))
+    Friend Custom Event Progress0Completed As EventHandler(Of ProgressEventArgs)
+        AddHandler(ByVal h As EventHandler(Of ProgressEventArgs))
+            If Not _Progress0CompletedEventHandlers.Contains(h) Then _Progress0CompletedEventHandlers.Add(h)
+        End AddHandler
+        RemoveHandler(ByVal h As EventHandler(Of ProgressEventArgs))
+            _Progress0CompletedEventHandlers.Remove(h)
+        End RemoveHandler
+        RaiseEvent(ByVal Sender As Object, ByVal e As ProgressEventArgs)
+            InvokeHandlers(_Progress0CompletedEventHandlers, Sender, e)
+        End RaiseEvent
+    End Event
+    Private WithEvents PR_PRE As MyProgress
+    Private Sub PR_PRE_ProgressChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs) Handles PR_PRE.ProgressChanged
+        RaiseEvent Progress0Changed(Sender, e)
+    End Sub
+    Private Sub PR_PRE_MaximumChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs) Handles PR_PRE.MaximumChanged
+        RaiseEvent Maximum0Changed(Sender, e)
+    End Sub
+    Private Sub PR_PRE_ProgressCompleted(ByVal Sender As Object, ByVal e As ProgressEventArgs) Handles PR_PRE.ProgressCompleted
+        RaiseEvent Progress0Completed(Sender, e)
+    End Sub
     Friend Sub New()
         _Progress0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
         _Maximum0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
+        _Progress0CompletedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
     End Sub
-    Friend Sub New(ByRef StatusStrip As StatusStrip, ByRef ProgressBar As ToolStripProgressBar, ByRef Label As ToolStripStatusLabel,
+    Friend Sub New(ByRef StatusStrip As StatusStrip, ByRef ProgressBar As ToolStripProgressBar, ByRef ProgressBarPre As ToolStripProgressBar, ByRef Label As ToolStripStatusLabel,
                    Optional ByVal Information As String = Nothing)
         MyBase.New(StatusStrip, ProgressBar, Label, Information)
+        PR_PRE = New MyProgress(StatusStrip, ProgressBarPre, Nothing) With {.PerformMod = 10, .ResetProgressOnMaximumChanges = False}
         _Progress0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
         _Maximum0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
+        _Progress0CompletedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
     End Sub
-    Friend Sub New(ByRef ProgressBar As ProgressBar, ByRef Label As Label, Optional ByVal Information As String = Nothing)
+    Friend Sub New(ByRef ProgressBar As ProgressBar, ByRef ProgressBarPre As ProgressBar, ByRef Label As Label, Optional ByVal Information As String = Nothing)
         MyBase.New(ProgressBar, Label, Information)
+        PR_PRE = New MyProgress(ProgressBarPre, Nothing) With {.PerformMod = 10, .ResetProgressOnMaximumChanges = False}
         _Progress0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
         _Maximum0ChangedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
+        _Progress0CompletedEventHandlers = New List(Of EventHandler(Of ProgressEventArgs))
     End Sub
-    Private _Maximum0 As Double = 0
     Friend Property Maximum0 As Double
         Get
-            Return _Maximum0
+            Return PR_PRE.Maximum
         End Get
         Set(ByVal v As Double)
-            Dim b As Boolean = Not _Maximum0 = v
+            PR_PRE.Maximum = v
-            _Maximum0 = v
+        End Set
-            If ResetProgressOnMaximumChanges Then Value0 = 0
+    End Property
-            If b Then RaiseEvent Maximum0Changed(Me, Nothing)
+    Friend Property Value0 As Double
+        Get
+            Return PR_PRE.Value
+        End Get
+        Set(ByVal v As Double)
+            PR_PRE.Value = v
         End Set
     End Property
-    Friend Property Value0 As Double = 0
     Friend Sub Perform0(Optional ByVal Value As Double = 1)
-        Value0 += Value
+        PR_PRE.Perform(Value)
-        If Perform(0, 10, False, False) Then RaiseEvent Progress0Changed(Me, Nothing)
     End Sub
-    Public Overloads Overrides Sub Perform(Optional ByVal Value As Double = 1)
-        If Perform(Value, PerformMod, True, True) Then OnProgressChanged()
-    End Sub
-    Public Overloads Function Perform(ByVal Value As Double, ByVal pm As Integer, ByVal SetText As Boolean, ByVal InvokeProgressChangeHandler As Boolean) As Boolean
-        Me.Value += Value
-        If Me.Value < 0 Then Me.Value = 0
-        Dim v# = Me.Value + Value0
-        Dim m# = Maximum + Maximum0
-        If pm = 0 OrElse (v Mod pm) = 0 OrElse v = m Then PerformImpl(GetPercentage(v, m), SetText, InvokeProgressChangeHandler) : Return True
-        Return False
-    End Function
     Public Overrides Sub Done()
-        Value0 = Maximum0
+        PR_PRE.Done()
         MyBase.Done()
     End Sub
-    Public Overrides Sub Reset()
+    Friend Sub Done0()
-        MyBase.Reset()
+        PR_PRE.Done()
-        Value0 = 0
-        Maximum0 = 0
     End Sub
+    Public Overrides Sub Reset()
+        PR_PRE.Reset()
+        MyBase.Reset()
+    End Sub
+    Friend Sub Reset0()
+        PR_PRE.Reset()
+    End Sub
+    Public Overrides Property Visible(Optional ByVal ProgressBar As Boolean = True, Optional ByVal Label As Boolean = True) As Boolean
+        Get
+            Return MyBase.Visible(ProgressBar, Label)
+        End Get
+        Set(ByVal _Visible As Boolean)
+            MyBase.Visible(ProgressBar, Label) = _Visible
+            PR_PRE.Visible(ProgressBar, Label) = _Visible
+        End Set
+    End Property
     Protected Overrides Sub Dispose(ByVal disposing As Boolean)
         If Not disposedValue And disposing Then
             _Progress0ChangedEventHandlers.Clear()
             _Maximum0ChangedEventHandlers.Clear()
+            _Progress0CompletedEventHandlers.Clear()
+            PR_PRE.Dispose()
         End If
         MyBase.Dispose(disposing)
     End Sub

SCrawler/PluginsEnvironment/Attributes/Attributes.vb

@@ -8,7 +8,7 @@
 ' but WITHOUT ANY WARRANTY
 Imports System.Runtime.CompilerServices
 Namespace Plugin.Attributes
-    Public Enum SettingAddress : Both : Settings : User : End Enum
+    Public Enum SettingAddress : Both : Settings : User : None : End Enum
     Public Class PSettingAttribute : Inherits Attribute
         Public Number As Integer = 0
         Friend ReadOnly Name As String

SCrawler/PluginsEnvironment/Hosts/PluginHost.vb

@@ -88,7 +88,8 @@ Namespace Plugin.Hosts
                     New PluginHost(New API.Xhamster.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(New API.XVIDEOS.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids),
                     New PluginHost(New API.ThisVid.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids),
-                    New PluginHost(New API.PathPlugin.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids)}
+                    New PluginHost(New API.PathPlugin.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids),
+                    New PluginHost(New API.OnlyFans.SiteSettings, _XML, GlobalPath, _Temp, _Imgs, _Vids)}
         End Function
         Friend Shared Function GetPluginsHosts(ByRef _XML As XmlFile, ByVal GlobalPath As SFile,
                                                ByRef _Temp As XMLValue(Of Boolean), ByRef _Imgs As XMLValue(Of Boolean),

SCrawler/SCrawler.vbproj

@@ -178,6 +178,7 @@
     <Compile Include="API\LPSG\UserData.vb" />
     <Compile Include="API\Mastodon\Credentials.vb" />
     <Compile Include="API\Mastodon\Declarations.vb" />
+    <Compile Include="API\Mastodon\EditorExchangeOptions.vb" />
     <Compile Include="API\Mastodon\MastodonDomains.vb" />
     <Compile Include="API\Mastodon\SettingsForm.Designer.vb">
       <DependentUpon>SettingsForm.vb</DependentUpon>
@@ -187,6 +188,9 @@
     </Compile>
     <Compile Include="API\Mastodon\SiteSettings.vb" />
     <Compile Include="API\Mastodon\UserData.vb" />
+    <Compile Include="API\OnlyFans\Declarations.vb" />
+    <Compile Include="API\OnlyFans\SiteSettings.vb" />
+    <Compile Include="API\OnlyFans\UserData.vb" />
     <Compile Include="API\PathPlugin\Declarations.vb" />
     <Compile Include="API\PathPlugin\SiteSettings.vb" />
     <Compile Include="API\PathPlugin\UserData.vb" />
@@ -660,6 +664,12 @@
   <ItemGroup>
     <None Include="Content\Icons\SiteIcons\MastodonIcon_48.ico" />
   </ItemGroup>
+  <ItemGroup>
+    <None Include="Content\Pictures\SitePictures\OnlyFansPic_32.png" />
+  </ItemGroup>
+  <ItemGroup>
+    <None Include="Content\Icons\SiteIcons\OnlyFansIcon_32.ico" />
+  </ItemGroup>
   <Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
   <Import Project="..\packages\VideoLAN.LibVLC.Windows.3.0.17.4\build\VideoLAN.LibVLC.Windows.targets" Condition="Exists('..\packages\VideoLAN.LibVLC.Windows.3.0.17.4\build\VideoLAN.LibVLC.Windows.targets')" />
   <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">

SCrawler/SiteResources.Designer.vb

@@ -124,6 +124,26 @@ Namespace My.Resources
             End Get
         End Property
         
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
+        '''</summary>
+        Friend Shared ReadOnly Property OnlyFansIcon_32() As System.Drawing.Icon
+            Get
+                Dim obj As Object = ResourceManager.GetObject("OnlyFansIcon_32", resourceCulture)
+                Return CType(obj,System.Drawing.Icon)
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Bitmap.
+        '''</summary>
+        Friend Shared ReadOnly Property OnlyFansPic_32() As System.Drawing.Bitmap
+            Get
+                Dim obj As Object = ResourceManager.GetObject("OnlyFansPic_32", resourceCulture)
+                Return CType(obj,System.Drawing.Bitmap)
+            End Get
+        End Property
+        
         '''<summary>
         '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
         '''</summary>

SCrawler/SiteResources.resx

@@ -136,6 +136,12 @@
   <data name="MastodonPic_48" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>Content\Pictures\SitePictures\MastodonPic_48.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>
+  <data name="OnlyFansIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>Content\Icons\SiteIcons\OnlyFansIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
+  <data name="OnlyFansPic_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>Content\Pictures\SitePictures\OnlyFansPic_32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
   <data name="PinterestIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>Content\Icons\SiteIcons\PinterestIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>