mirror of
https://github.com/AAndyProgram/SCrawler.git
synced 2026-03-16 16:52:17 +00:00
2023.8.6.0
Plugins.Attributes: add 'DependentFields' attribute Plugins.IPluginContentProvider: add 'Options' and 'IsSubscription' properties Plugins.ISiteSettings: add 'SubscriptionsAllowed' property Plugins.ExchangeOptions: add 'Options' field Plugins.Attributes.PropertyUpdater: replace 'Dependencies' with 'Arguments' YT: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties; add the ability to store download locations; add 'DownloadLocation' and 'DownloadLocationsCollection' objects YT.IDownloaderSettings: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties YT.Downloader: fixed bug with re-saving elements when loading a video list; fixed bug when files were not deleted when clicking on the delete button; fixed a bug that caused the video to redownload; download job removes elements at wrong indexes; added skipping of downloaded elements in the job; fixed a bug, pending option did not change after download complete YT.YouTubeMediaContainerBase: add '_MediaStateOnLoad' field and 'NeedToSave' function; update the 'Save' function to prevent saving a file when a download is complete and the file has already been saved; update code for new yt-dlp version Fixed cache deletion errors Add user queue Add global locations API.Base.SiteSettingsBase: implement 'SubscriptionsAllowed' property; remove request headers with null values on save; add '_AllowUserAgentUpdate' parameter API.Base.Structures: add 'SiteModes' enum API.Base.UserDataBase: add 'Erase' button; implement 'Options' and 'IsSubscription' properties; add 'SpecialLabels' property; update 'LVIKey'; update 'FitToAddParams' function; add 'EraseData' function; user colors; Not UserExists notification, UserQueue support API.Base: add 'DeclaredNames' API.Instagram: remove default values for headers; disable updating UserAgent from global; check for a new username for non-existent users API.Mastodon: bypass new inherited twitter options; update names and headers API.OnlyFans: make 'HH_BROWSER' property nullable; remove 'HH_BROWSER' from required; fix username bug (dots); handling of 504 and 429 errors; add 'DownloadHighlights' and 'DownloadChatMedia' options; add 'UserExchangeOptions'; fixed incorrect error handler API.PathPlugin: fixed incorrect detection of path existence API.Pinterest: add 'SpecialLabels' API.PornHub: add new video regex; remove old regex; added 'DownloadUploaded', 'DownloadTagged', 'DownloadPrivate' and 'DownloadFavorite' properties to 'SiteSettings', 'UserData' and 'UserExchangeOptions'; update regex to define user; added downloading search queries; update 'GetUserUrl' function; hide unnecessary 'RegexFieldsTextBecameNullException' errors; add subscriptions API.Reddit: add 'SpecialLabels'; add bearer token and its refresh interval; add OAuth; add additional options API.RedGifs: add 'DependentFields' for 'Token' API.ThisVid: add 'DownloadFavourite' option; add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; updating cookies issue API.TikTok: rewrite algorithms API.Twitter: add 'UseAppropriateModel', 'UseNewEndPointSearch', 'UseNewEndPointProfiles', 'AbortOnLimit', 'DownloadAlreadyParsed', 'MediaModelAllowNonUserTweets' properties; remove old commented code; remove 'TwitterPic_400' and replace with 'TwitterIcon_32.ToBitmap'; add 'DownloadModelForceApply' user option; update environment to GDL 1.25.8; fixed gifs downloading; fix typo in 'ReparseMissing'; update names API.UserDataBind: prevent adding site-specific labels when adding to a collection API.Xhamster: add downloading search queries, tags, categories; add 'SpecialLabels'; add additional nodes for channels; add subscriptions API.XVIDEOS: add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; changed users creation method; add subscriptions API.YouTube: add subscriptions AutoDownloader: add new group subscription options; update predicates; fixed excluded labels and sites in default mode; update notifications; add an additional skip options, add 'Force start' option DownloadedInfoForm: add subscriptions; fixed size/location bug; hide unnecessary error (refill) Feed: add subscriptions; update filters; add 'Ctrl+G' shortcut FeedMedia: add subscriptions; fixed 'webm' bug; add title for subscription media; add site icon to post; user colors; always using 'FriendlyName' instead of 'UserName' if it exists DownloadGroup, GroupDefaults, GroupParameters: add subscription and 'UsersCount' options MissingPostsForm: add 'BTT_DELETE_ALL' VideoDownloaderForm, DownloaderUrlForm, DownloaderUrlsArrForm: add download locations support VideoDownloaderForm: add subscriptions support GlobalSettingsForm: add new properties UserCreatorForm: add subscriptions; add 'Options' support (of 'ExchangeOptions'); user colors ListImagesLoader: add subscription colors; user colors MainFrame: add subscriptions; add filters by subscription and user; update predicates NuGet: update 'LibVLCSharp', 'LibVLCSharp.WinForms', 'VideoLAN.LibVLC.Windows' DownloadableMediaHost: update 'Save' function PropertyValueHost: fix 'CaptionWidth' bug; add 'Dependents' SettingsHost: add 'Dependents' UserDataHost: add 'Options' and 'IsSubscription' properties SettingsCLS: implement new 'IDownloaderSettings' properties; add 'CacheSnapshots'; add 'DownloadLocations'; add new properties UserInfo, UserFinder: add subscriptions UserSearchForm: fixed search by name bug
This commit is contained in:
Andy
@@ -14,7 +14,7 @@ Imports PersonalUtilities.Tools.Web.Clients
|
||||
Imports PersonalUtilities.Tools.Web.Cookies
|
||||
Imports PersonalUtilities.Functions.RegularExpressions
|
||||
Namespace API.OnlyFans
|
||||
<Manifest("AndyProgram_OnlyFans"), SavedPosts, SeparatedTasks(1)>
|
||||
<Manifest("AndyProgram_OnlyFans"), SavedPosts, SpecialForm(False), SeparatedTasks(1)>
|
||||
Friend Class SiteSettings : Inherits SiteSettingsBase
|
||||
#Region "Icon"
|
||||
Friend Overrides ReadOnly Property Icon As Icon
|
||||
@@ -29,6 +29,13 @@ Namespace API.OnlyFans
|
||||
End Property
|
||||
#End Region
|
||||
#Region "Declarations"
|
||||
#Region "Options"
|
||||
<PropertyOption(ControlText:="Download highlights", ControlToolTip:="Download profile highlights if they exists"), PXML>
|
||||
Friend Property DownloadHighlights As PropertyValue
|
||||
<PropertyOption(ControlText:="Download chat", ControlToolTip:="Download unlocked chat media"), PXML>
|
||||
Friend Property DownloadChatMedia As PropertyValue
|
||||
#End Region
|
||||
#Region "Headers"
|
||||
Private Const HeaderBrowser As String = "sec-ch-ua"
|
||||
Private Const HeaderUserID As String = "User-Id"
|
||||
Private Const HeaderXBC As String = "X-Bc"
|
||||
@@ -39,7 +46,7 @@ Namespace API.OnlyFans
|
||||
Private ReadOnly Property HH_X_BC As PropertyValue
|
||||
<PropertyOption(ControlText:=HeaderAppToken, AllowNull:=False)>
|
||||
Private ReadOnly Property HH_APP_TOKEN As PropertyValue
|
||||
<PropertyOption(ControlText:=HeaderBrowser, AllowNull:=False)>
|
||||
<PropertyOption(ControlText:=HeaderBrowser, ControlToolTip:="Can be null", AllowNull:=True)>
|
||||
Private ReadOnly Property HH_BROWSER As PropertyValue
|
||||
<PropertyOption(AllowNull:=False)>
|
||||
Private ReadOnly Property UserAgent As PropertyValue
|
||||
@@ -59,6 +66,8 @@ Namespace API.OnlyFans
|
||||
Responser.UserAgent = Value
|
||||
End If
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Rules"
|
||||
<PXML("LastDateUpdated")> Private ReadOnly Property LastDateUpdated_XML As PropertyValue
|
||||
Friend Property LastDateUpdated As Date
|
||||
Get
|
||||
@@ -81,6 +90,7 @@ Namespace API.OnlyFans
|
||||
"Change this value only if you know what you are doing."), PXML>
|
||||
Friend ReadOnly Property DynamicRules As PropertyValue
|
||||
#End Region
|
||||
#End Region
|
||||
#Region "Initializer"
|
||||
Friend Sub New()
|
||||
MyBase.New("OnlyFans", ".onlyfans.com")
|
||||
@@ -110,13 +120,16 @@ Namespace API.OnlyFans
|
||||
UserAgent = New PropertyValue(IIf(.UserAgentExists, .UserAgent, String.Empty), GetType(String), Sub(v) UpdateHeader(NameOf(UserAgent), v))
|
||||
End With
|
||||
|
||||
DownloadHighlights = New PropertyValue(True)
|
||||
DownloadChatMedia = New PropertyValue(True)
|
||||
|
||||
LastDateUpdated_XML = New PropertyValue(Now.AddYears(-1), GetType(Date))
|
||||
UseOldAuthRules = New PropertyValue(False)
|
||||
DynamicRulesUpdateInterval = New PropertyValue(60 * 24)
|
||||
DynamicRulesUpdateIntervalProvider = New FieldsCheckerProviderSimple(Function(v) IIf(AConvert(Of Integer)(v, 0) > 0, v, Nothing),
|
||||
"The value of [{0}] field must be greater than 0")
|
||||
DynamicRules = New PropertyValue(String.Empty, GetType(String))
|
||||
UserRegex = RParams.DMS("onlyfans.com/(\w+)", 1, EDP.ReturnValue)
|
||||
UserRegex = RParams.DMS("onlyfans.com/([\w\._]+)", 1, EDP.ReturnValue)
|
||||
UrlPatternUser = "https://onlyfans.com/{0}"
|
||||
ImageVideoContains = "onlyfans.com"
|
||||
End Sub
|
||||
@@ -134,7 +147,7 @@ Namespace API.OnlyFans
|
||||
#End Region
|
||||
#Region "Download"
|
||||
Friend Overrides Function BaseAuthExists() As Boolean
|
||||
Return Responser.CookiesExists And {HH_USER_ID, HH_X_BC, HH_APP_TOKEN, HH_BROWSER, UserAgent}.All(Function(v) ACheck(v.Value))
|
||||
Return Responser.CookiesExists And {HH_USER_ID, HH_X_BC, HH_APP_TOKEN, UserAgent}.All(Function(v) ACheck(v.Value))
|
||||
End Function
|
||||
Friend Overrides Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean
|
||||
Return BaseAuthExists() And Not SessionAborted
|
||||
@@ -149,17 +162,36 @@ Namespace API.OnlyFans
|
||||
If Responser.Cookies.Changed Then Responser.SaveCookies() : Responser.Cookies.Changed = False
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "GetUserUrl, GetUserPostUrl"
|
||||
#Region "GetUserUrl, GetUserPostUrl, UserOptions"
|
||||
Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider) As String
|
||||
Return String.Format(UrlPatternUser, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
|
||||
End Function
|
||||
Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
|
||||
If Not Media.Post.ID.IsEmptyString Then
|
||||
Return String.Format("https://onlyfans.com/{0}/{1}", Media.Post.ID, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
|
||||
Dim post$() = Media.Post.ID.Split("_")
|
||||
Dim p$ = String.Empty
|
||||
If post.ListExists Then
|
||||
If post(0) = UserData.A_MESSAGE Then
|
||||
If Not User.ID.IsEmptyString Then Return $"https://onlyfans.com/my/chats/chat/{User.ID}/"
|
||||
ElseIf Not post(0) = UserData.A_HIGHLIGHT Then
|
||||
p = post(0)
|
||||
End If
|
||||
End If
|
||||
If p.IsEmptyString Then
|
||||
Return GetUserUrl(User)
|
||||
Else
|
||||
Return String.Format("https://onlyfans.com/{0}/{1}", p, If(User.ID.IsEmptyString, User.Name, $"u{User.ID}"))
|
||||
End If
|
||||
Else
|
||||
Return String.Empty
|
||||
End If
|
||||
End Function
|
||||
Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
|
||||
If Options Is Nothing OrElse Not TypeOf Options Is UserExchangeOptions Then Options = New UserExchangeOptions(Me)
|
||||
If OpenForm Then
|
||||
Using f As New InternalSettingsForm(Options, Me, False) : f.ShowDialog() : End Using
|
||||
End If
|
||||
End Sub
|
||||
#End Region
|
||||
End Class
|
||||
End Namespace
|
||||
@@ -19,26 +19,68 @@ Imports UTypes = SCrawler.API.Base.UserMedia.Types
|
||||
Imports UStates = SCrawler.API.Base.UserMedia.States
|
||||
Namespace API.OnlyFans
|
||||
Friend Class UserData : Inherits UserDataBase
|
||||
#Region "XML names"
|
||||
Private Const Name_MediaDownloadHighlights As String = "DownloadHighlights"
|
||||
Private Const Name_MediaDownloadChatMedia As String = "DownloadChatMedia"
|
||||
#End Region
|
||||
#Region "Declarations"
|
||||
Friend Property CCookie As CookieKeeper = Nothing
|
||||
Private Const HeaderSign As String = "Sign"
|
||||
Private Const HeaderTime As String = "Time"
|
||||
Private ReadOnly HighlightsList As List(Of String)
|
||||
Friend Property MediaDownloadHighlights As Boolean = True
|
||||
Friend Property MediaDownloadChatMedia As Boolean = True
|
||||
Private ReadOnly Property MySettings As SiteSettings
|
||||
Get
|
||||
Return HOST.Source
|
||||
End Get
|
||||
End Property
|
||||
#End Region
|
||||
#Region "Load"
|
||||
Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
|
||||
With Container
|
||||
If Loading Then
|
||||
MediaDownloadHighlights = .Value(Name_MediaDownloadHighlights).FromXML(Of Boolean)(True)
|
||||
MediaDownloadChatMedia = .Value(Name_MediaDownloadChatMedia).FromXML(Of Boolean)(True)
|
||||
Else
|
||||
.Add(Name_MediaDownloadHighlights, MediaDownloadHighlights.BoolToInteger)
|
||||
.Add(Name_MediaDownloadChatMedia, MediaDownloadChatMedia.BoolToInteger)
|
||||
End If
|
||||
End With
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Exchange"
|
||||
Friend Overrides Function ExchangeOptionsGet() As Object
|
||||
Return New UserExchangeOptions(Me)
|
||||
End Function
|
||||
Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
|
||||
If Not Obj Is Nothing AndAlso TypeOf Obj Is UserExchangeOptions Then
|
||||
With DirectCast(Obj, UserExchangeOptions)
|
||||
MediaDownloadHighlights = .DownloadHighlights
|
||||
MediaDownloadChatMedia = .DownloadChatMedia
|
||||
End With
|
||||
End If
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Initializer"
|
||||
Friend Sub New()
|
||||
HighlightsList = New List(Of String)
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Download functions"
|
||||
Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
|
||||
If Not CCookie Is Nothing Then CCookie.Dispose()
|
||||
CCookie = Responser.Cookies.Copy
|
||||
Responser.Cookies.Clear()
|
||||
AddHandler Responser.ResponseReceived, AddressOf OnResponseReceived
|
||||
UpdateCookieHeader()
|
||||
DownloadData(IIf(IsSavedPosts, 0, String.Empty), Token)
|
||||
If Not MySettings.SessionAborted Then
|
||||
If Not CCookie Is Nothing Then CCookie.Dispose()
|
||||
CCookie = Responser.Cookies.Copy
|
||||
Responser.Cookies.Clear()
|
||||
AddHandler Responser.ResponseReceived, AddressOf OnResponseReceived
|
||||
UpdateCookieHeader()
|
||||
DownloadTimeline(IIf(IsSavedPosts, 0, String.Empty), Token)
|
||||
If Not IsSavedPosts Then
|
||||
If MediaDownloadHighlights Then DownloadHighlights(Token)
|
||||
If MediaDownloadChatMedia Then DownloadChatMedia(0, Token)
|
||||
End If
|
||||
End If
|
||||
End Sub
|
||||
Private Sub OnResponseReceived(ByVal Sender As Object, ByVal e As WebDataResponse)
|
||||
If e.CookiesExists Then
|
||||
@@ -49,9 +91,11 @@ Namespace API.OnlyFans
|
||||
Private Sub UpdateCookieHeader()
|
||||
Responser.Headers.Add("Cookie", CCookie.ToString(False))
|
||||
End Sub
|
||||
Friend Const A_HIGHLIGHT As String = "HL"
|
||||
Friend Const A_MESSAGE As String = "MSG"
|
||||
Private Const BaseUrlPattern As String = "https://onlyfans.com{0}"
|
||||
Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken)
|
||||
|
||||
#Region "Download timeline"
|
||||
Private Overloads Sub DownloadTimeline(ByVal Cursor As String, ByVal Token As CancellationToken)
|
||||
Dim url$ = String.Empty
|
||||
Dim _complete As Boolean = True
|
||||
Do
|
||||
@@ -122,15 +166,148 @@ Namespace API.OnlyFans
|
||||
|
||||
If hasMore Then
|
||||
If IsSavedPosts Then tmpCursor = CInt(Cursor.IfNullOrEmpty(0)) + 10
|
||||
DownloadData(tmpCursor, Token)
|
||||
DownloadTimeline(tmpCursor, Token)
|
||||
End If
|
||||
Catch ex As Exception
|
||||
If ProcessException(ex, Token, $"data downloading error [{url}]") = 2 Then _complete = False
|
||||
_complete = Not ProcessException(ex, Token, $"data downloading error [{url}]") = 2
|
||||
End Try
|
||||
Loop While Not _complete
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Download highlights"
|
||||
Private Overloads Sub DownloadHighlights(ByVal Token As CancellationToken)
|
||||
HighlightsList.Clear()
|
||||
DownloadHighlights(0, Token)
|
||||
If HighlightsList.Count > 0 Then HighlightsList.ForEach(Sub(hl) DownloadHighlightMedia(hl, Token))
|
||||
End Sub
|
||||
Private Overloads Sub DownloadHighlights(ByVal Cursor As Integer, ByVal Token As CancellationToken)
|
||||
Dim url$ = String.Empty
|
||||
Dim _complete As Boolean = True
|
||||
Do
|
||||
Try
|
||||
Dim hasMore As Boolean = False
|
||||
Dim path$ = $"/api2/v2/users/{ID}/stories/highlights?limit=5&offset={Cursor}"
|
||||
If UpdateSignature(path) Then
|
||||
url = String.Format(BaseUrlPattern, path)
|
||||
ThrowAny(Token)
|
||||
Dim r$ = Responser.GetResponse(url)
|
||||
If Not r.IsEmptyString Then
|
||||
Using j As EContainer = JsonDocument.Parse(r)
|
||||
If j.ListExists Then
|
||||
hasMore = j.Value("hasMore").FromXML(Of Boolean)(False)
|
||||
With j("list")
|
||||
If .ListExists Then
|
||||
HighlightsList.AddRange(.Select(Function(e) e.Value("id")))
|
||||
Else
|
||||
hasMore = False
|
||||
End If
|
||||
End With
|
||||
End If
|
||||
End Using
|
||||
End If
|
||||
End If
|
||||
If hasMore Then DownloadHighlights(Cursor + 5, Token)
|
||||
Catch ex As Exception
|
||||
_complete = Not ProcessException(ex, Token, $"highlights downloading error [{url}]") = 2
|
||||
End Try
|
||||
Loop While Not _complete
|
||||
End Sub
|
||||
Private Sub DownloadHighlightMedia(ByVal HLID As String, ByVal Token As CancellationToken)
|
||||
Dim url$ = String.Empty
|
||||
Dim _complete As Boolean = True
|
||||
Do
|
||||
Try
|
||||
Dim specFolder$, postID$, postDate$
|
||||
Dim media As List(Of UserMedia)
|
||||
Dim result As Boolean
|
||||
Dim path$ = $"/api2/v2/stories/highlights/{HLID}"
|
||||
If UpdateSignature(path) Then
|
||||
url = String.Format(BaseUrlPattern, path)
|
||||
ThrowAny(Token)
|
||||
Dim r$ = Responser.GetResponse(url)
|
||||
If Not r.IsEmptyString Then
|
||||
Using j As EContainer = JsonDocument.Parse(r)
|
||||
If j.ListExists Then
|
||||
specFolder = j.Value("title").StringRemoveWinForbiddenSymbols.IfNullOrEmpty(HLID)
|
||||
specFolder &= "*"
|
||||
With j("stories")
|
||||
If .ListExists Then
|
||||
ProgressPre.ChangeMax(.Count)
|
||||
For Each m As EContainer In .Self
|
||||
ProgressPre.Perform()
|
||||
postID = $"{A_HIGHLIGHT}_{HLID}_{m.Value("id")}"
|
||||
postDate = m.Value("createdAt")
|
||||
If Not _TempPostsList.Contains(postID) Then
|
||||
_TempPostsList.Add(postID)
|
||||
Else
|
||||
Exit Sub
|
||||
End If
|
||||
result = False
|
||||
media = TryCreateMedia(m, postID, postDate, result, True, specFolder)
|
||||
If result Then _TempMediaList.ListAddList(media, LNC)
|
||||
Next
|
||||
End If
|
||||
End With
|
||||
End If
|
||||
End Using
|
||||
End If
|
||||
End If
|
||||
Catch ex As Exception
|
||||
_complete = Not ProcessException(ex, Token, $"highlights downloading error [{url}]") = 2
|
||||
End Try
|
||||
Loop While Not _complete
|
||||
End Sub
|
||||
#End Region
|
||||
#Region "Download chat media"
|
||||
Private Sub DownloadChatMedia(ByVal Cursor As Integer, ByVal Token As CancellationToken)
|
||||
Dim url$ = String.Empty
|
||||
Dim _complete As Boolean = True
|
||||
Do
|
||||
Try
|
||||
Dim hasMore As Boolean = False
|
||||
Dim postID$, postDate$
|
||||
Dim media As List(Of UserMedia)
|
||||
Dim result As Boolean
|
||||
Dim path$ = $"/api2/v2/chats/{ID}/media/?opened=1&limit=20&skip_users=all"
|
||||
If Cursor > 0 Then path &= $"&offset={Cursor}"
|
||||
If UpdateSignature(path) Then
|
||||
url = String.Format(BaseUrlPattern, path)
|
||||
ThrowAny(Token)
|
||||
Dim r$ = Responser.GetResponse(url)
|
||||
If Not r.IsEmptyString Then
|
||||
Using j As EContainer = JsonDocument.Parse(r)
|
||||
If j.ListExists Then
|
||||
hasMore = j.Value("hasMore").FromXML(Of Boolean)(False)
|
||||
With j("list")
|
||||
If .ListExists Then
|
||||
For Each m As EContainer In .Self
|
||||
postID = $"{A_MESSAGE}_{m.Value("id")}"
|
||||
postDate = m.Value("createdAt")
|
||||
If Not _TempPostsList.Contains(postID) Then
|
||||
_TempPostsList.Add(postID)
|
||||
Else
|
||||
Exit Sub
|
||||
End If
|
||||
result = False
|
||||
media = TryCreateMedia(m, postID, postDate, result,, "Chats*")
|
||||
If result Then _TempMediaList.ListAddList(media, LNC)
|
||||
Next
|
||||
End If
|
||||
End With
|
||||
End If
|
||||
End Using
|
||||
End If
|
||||
End If
|
||||
If hasMore Then DownloadChatMedia(Cursor + 20, Token)
|
||||
Catch ex As Exception
|
||||
_complete = Not ProcessException(ex, Token, $"chats downloading error [{url}]") = 2
|
||||
End Try
|
||||
Loop While Not _complete
|
||||
End Sub
|
||||
#End Region
|
||||
Private Function TryCreateMedia(ByVal n As EContainer, ByVal PostID As String, Optional ByVal PostDate As String = Nothing,
|
||||
Optional ByRef Result As Boolean = False) As List(Of UserMedia)
|
||||
Optional ByRef Result As Boolean = False, Optional ByVal IsHL As Boolean = False,
|
||||
Optional ByVal SpecFolder As String = Nothing) As List(Of UserMedia)
|
||||
Dim postUrl$, ext$
|
||||
Dim t As UTypes
|
||||
Dim mList As New List(Of UserMedia)
|
||||
@@ -138,7 +315,11 @@ Namespace API.OnlyFans
|
||||
With n("media")
|
||||
If .ListExists Then
|
||||
For Each m In .Self
|
||||
postUrl = m.Value({"source"}, "source").IfNullOrEmpty(m.Value("full"))
|
||||
If IsHL Then
|
||||
postUrl = m.Value({"files", "source"}, "url")
|
||||
Else
|
||||
postUrl = m.Value({"source"}, "source").IfNullOrEmpty(m.Value("full"))
|
||||
End If
|
||||
Select Case m.Value("type")
|
||||
Case "photo" : t = UTypes.Picture : ext = "jpg"
|
||||
Case "video" : t = UTypes.Video : ext = "mp4"
|
||||
@@ -146,7 +327,9 @@ Namespace API.OnlyFans
|
||||
End Select
|
||||
If Not t = UTypes.Undefined And Not postUrl.IsEmptyString Then
|
||||
Dim media As New UserMedia(postUrl, t) With {
|
||||
.Post = New UserPost(PostID, AConvert(Of Date)(PostDate, DateProvider, Nothing))}
|
||||
.Post = New UserPost(PostID, AConvert(Of Date)(PostDate, DateProvider, Nothing)),
|
||||
.SpecialFolder = SpecFolder
|
||||
}
|
||||
media.File.Extension = ext
|
||||
Result = True
|
||||
mList.Add(media)
|
||||
@@ -157,6 +340,7 @@ Namespace API.OnlyFans
|
||||
Return mList
|
||||
End Function
|
||||
Private Sub GetUserID()
|
||||
Const brTag$ = "<br />"
|
||||
Dim path$ = $"/api2/v2/users/{Name}"
|
||||
Dim url$ = String.Format(BaseUrlPattern, path)
|
||||
Try
|
||||
@@ -168,7 +352,9 @@ Namespace API.OnlyFans
|
||||
ID = j.Value("id")
|
||||
If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
|
||||
UserSiteNameUpdate(j.Value("name"))
|
||||
UserDescriptionUpdate(j.Value("about"))
|
||||
Dim descr$ = j.Value("about")
|
||||
If Not descr.IsEmptyString Then descr = descr.Replace(brTag, String.Empty)
|
||||
UserDescriptionUpdate(descr)
|
||||
Dim a As Action(Of String) = Sub(ByVal address As String)
|
||||
If Not address.IsEmptyString Then
|
||||
Dim f As SFile = address
|
||||
@@ -232,7 +418,7 @@ Namespace API.OnlyFans
|
||||
ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
|
||||
Finally
|
||||
If rList.Count > 0 Then
|
||||
For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next
|
||||
For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next
|
||||
rList.Clear()
|
||||
End If
|
||||
End Try
|
||||
@@ -347,6 +533,10 @@ Namespace API.OnlyFans
|
||||
ElseIf Responser.StatusCode = Net.HttpStatusCode.NotFound Then
|
||||
UserExists = False
|
||||
Return 1
|
||||
ElseIf Responser.StatusCode = Net.HttpStatusCode.GatewayTimeout Or Responser.StatusCode = 429 Then
|
||||
If Responser.StatusCode = 429 Then MyMainLOG = $"[429] OnlyFans too many requests ({ToStringForLog()})"
|
||||
MySettings.SessionAborted = True
|
||||
Return 1
|
||||
Else
|
||||
Return 0
|
||||
End If
|
||||
@@ -354,7 +544,7 @@ Namespace API.OnlyFans
|
||||
#End Region
|
||||
#Region "IDisposable Support"
|
||||
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
|
||||
If Not disposedValue And disposing Then CCookie.DisposeIfReady(False) : CCookie = Nothing
|
||||
If Not disposedValue And disposing Then CCookie.DisposeIfReady(False) : CCookie = Nothing : HighlightsList.Clear()
|
||||
MyBase.Dispose(disposing)
|
||||
End Sub
|
||||
#End Region
|
||||
|
||||
28
SCrawler/API/OnlyFans/UserExchangeOptions.vb
Normal file
28
SCrawler/API/OnlyFans/UserExchangeOptions.vb
Normal file
@@ -0,0 +1,28 @@
|
||||
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
|
||||
' This program is free software: you can redistribute it and/or modify
|
||||
' it under the terms of the GNU General Public License as published by
|
||||
' the Free Software Foundation, either version 3 of the License, or
|
||||
' (at your option) any later version.
|
||||
'
|
||||
' This program is distributed in the hope that it will be useful,
|
||||
' but WITHOUT ANY WARRANTY
|
||||
Imports SCrawler.Plugin.Attributes
|
||||
Namespace API.OnlyFans
|
||||
Friend Class UserExchangeOptions
|
||||
<PSetting(NameOf(SiteSettings.DownloadHighlights), NameOf(MySettings))>
|
||||
Friend Property DownloadHighlights As Boolean
|
||||
<PSetting(NameOf(SiteSettings.DownloadChatMedia), NameOf(MySettings))>
|
||||
Friend Property DownloadChatMedia As Boolean
|
||||
Private ReadOnly MySettings As SiteSettings
|
||||
Friend Sub New(ByVal u As UserData)
|
||||
DownloadHighlights = u.MediaDownloadHighlights
|
||||
DownloadChatMedia = u.MediaDownloadChatMedia
|
||||
MySettings = u.HOST.Source
|
||||
End Sub
|
||||
Friend Sub New(ByVal s As SiteSettings)
|
||||
DownloadHighlights = s.DownloadHighlights.Value
|
||||
DownloadChatMedia = s.DownloadChatMedia.Value
|
||||
MySettings = s
|
||||
End Sub
|
||||
End Class
|
||||
End Namespace
|
||||
Reference in New Issue
Block a user