2023.10.1.0

JustForFans: some profiles won't download

Changelog.md

@@ -1,3 +1,18 @@
+# 2023.10.1.0
+
+*2023-10-01*
+
+- Added
+  - **Threads.net**
+  - YouTube: add URL standardization
+- Fixed
+  - UserEditor: disable updating labels if they haven't changed
+  - Collections: incorrect updating of colors and labels when adding a new user
+  - RedGifs: incorrect handling of error 410
+  - Mastodon: hide error 503
+  - JustForFans: some profiles won't download
+  - Minor bugs
+
 # 2023.9.21.0
 
 *2023-09-21*

README.md

@@ -11,7 +11,7 @@
 :eu:
 :greece:
 
-A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
+A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
 
 **If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
 <!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
@@ -37,6 +37,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
   - JustForFans images and videos, saved (bookmarked) posts;
   - Mastodon images and videos, saved (bookmarked) posts;
   - Instagram images and videos, tagged posts, stories, saved posts;
+  - Threads images and videos;
   - TikTok videos;
   - Pinterest boards, users, saved posts;
   - Imgur images, galleries and videos;
@@ -73,6 +74,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **OnlyFans**
 - **Mastodon**
 - **Instagram**
+- **Threads**
 - JustForFans
 - TikTok
 - RedGifs
@@ -124,6 +126,7 @@ First, the program downloads the full profile. After the program downloads only
   - [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
   - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
   - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
+  - [Threads](https://github.com/AAndyProgram/SCrawler/wiki/Settings#threads)
   - [JustForFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#justforfans)
   - [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
   - [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)

SCrawler.PluginProvider/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.8.6.0")>
+<Assembly: AssemblyVersion("2023.10.1.0")>
-<Assembly: AssemblyFileVersion("2023.8.6.0")>
+<Assembly: AssemblyFileVersion("2023.10.1.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.9.20.0")>
+<Assembly: AssemblyVersion("2023.10.1.0")>
-<Assembly: AssemblyFileVersion("2023.9.20.0")>
+<Assembly: AssemblyFileVersion("2023.10.1.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.9.20.0")>
+<Assembly: AssemblyVersion("2023.10.1.0")>
-<Assembly: AssemblyFileVersion("2023.9.20.0")>
+<Assembly: AssemblyFileVersion("2023.10.1.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/API/JustForFans/UserData.vb

@@ -238,6 +238,8 @@ Namespace API.JustForFans
 
                                 _DownloadedPostsCount += 1
                                 _TempMediaList.ListAddList(post.GetUserMedia(FileSerialInstance), LNC)
+
+                                If _Limit > 0 And _DownloadedPostsCount >= _Limit Then Exit For
                             End If
                         Next
                     End If
@@ -250,11 +252,12 @@ Namespace API.JustForFans
         End Sub
         Private Sub GetUserID()
             Try
-                Dim r$, hash$, new_id$
+                Dim r$, hash$, new_id$, profilePic$
                 If ID.IsEmptyString Then
                     r = Responser.GetResponse($"https://justfor.fans/{Name}")
                     If Not r.IsEmptyString Then
                         hash = RegexReplace(r, RegexUser)
+                        profilePic = RegexReplace(r, RParams.DMS("<img class=.mainProfilePic..+?src=""([^""]+)", 1, EDP.ReturnValue))
                         If Not hash.IsEmptyString Then
                             r = Responser.GetResponse($"https://justfor.fans/ajax/getAssetCount.php?User={Name}&Ver={hash}")
                             If Not r.IsEmptyString Then
@@ -262,8 +265,14 @@ Namespace API.JustForFans
                                     If j.ListExists Then
                                         new_id = j.Value("UserID")
                                         If Not new_id.IsEmptyString Then
-                                            new_id = RegexReplace(new_id, RParams.DM("\D", 0, RegexReturn.Replace, CType(Function(input$) String.Empty, Func(Of String, String))))
+                                            new_id = RegexReplace(new_id, RParams.DM("\D", -1, RegexReturn.Replace,
-                                            If Not new_id.IsEmptyString Then ID = new_id : _ForceSaveUserInfo = True
+                                                                                     CType(Function(input$) String.Empty, Func(Of String, String)),
+                                                                                     String.Empty, EDP.ReturnValue))
+                                            If Not new_id.IsEmptyString Then
+                                                ID = new_id
+                                                _ForceSaveUserInfo = True
+                                                If Not profilePic.IsEmptyString Then GetWebFile(profilePic, $"{DownloadContentDefault_GetRootDir.CSFilePS}ProfilePic.jpg", EDP.None)
+                                            End If
                                         End If
                                     End If
                                 End Using

SCrawler/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.9.21.0")>
+<Assembly: AssemblyVersion("2023.10.1.0")>
-<Assembly: AssemblyFileVersion("2023.9.21.0")>
+<Assembly: AssemblyFileVersion("2023.10.1.0")>
 <Assembly: NeutralResourcesLanguage("en")>