Accounting A-1: Ingest-Fundament (unabhaengiger Ledger, idempotent, read-only)
Neues Modul PolyTrader.Modules.Accounting (IPolyTraderModule, acc_-Praefix, nur Core-Referenz, KEIN Handel). Konzept: docs/konzepte/KONZEPT-Modul-Accounting.md, Phase A-1. Buchungsgrundlage ausschliesslich aus unabhaengigen Polymarket-/On-Chain-Abrufen (nie unsere Trading-DB), append-only, prueffaehig: - Modelle: LedgerEntry (+ LedgerEventType), IngestRun (mit Balance-Anker), RawSnapshot, RawActivity/RawTransfer (normalisierte Eingaben, entkoppeln pure Logik von der API-Feldbenennung). - AccountingClassifier (Logic/, pur+getestet): Activity->Buchungssatz (Typ/Vorzeichen: BUY=Cash raus inkl. Fee, SELL=Cash rein minus Fee, Redeem/Reward +, Split/Merge/Conversion geldneutral), stabiler Idempotency-Key; Transfer-Klassifikation trennt intern (System-Contract-Whitelist) von externen Deposits/Withdrawals. SumNet fuer den Balance-Anker-Abgleich. - AccountingDbContext (acc_ledger append-only + Unique-Index Idempotency, acc_ingest_runs, acc_raw; Autoincrement-PKs). Migration InitialAccounting generiert UND angewendet. Repos mit idempotentem Upsert (true=neu/false=Duplikat). - AccountingIngestService (BackgroundService): testbarer IngestAccountAsync - Activity + On-Chain- Transfers klassifizieren + idempotent buchen, Rohschnappschuss ablegen, Lauf inkl. Balance-Anker- Delta protokollieren; Backfill vs. inkrementell (Lookback-Ueberlappung gegen API-Lag). - Quellen hinter Interfaces (IActivitySource/ITransferSource/IBalanceAnchorSource) mit Null-Stubs: Modul laeuft offline und bucht korrekt nichts. Live-Abruf + System-Contract-Whitelist = Zielland. - UI designerfaehig (partial + .Designer.cs): Tabs Ledger (filterbar) + Abruf/Status (Ingest-Laeufe, Balance-Anker, manueller Backfill/Inkrement). - Program.cs (beide Modul-Listen) + sln + App/Tests-Referenzen. A-2 (Abrechnung/BWA/FX), A-3 (US-Steuerschicht FIFO/Form-8949), A-4 (CSV/PDF via PDFsharp/MigraDoc) folgen. Tests: +12 (Klassifikation, intern/extern-Transfer, Ingest-Idempotenz, Balance-Anker, Inkrement-Fenster). Build 0 Fehler, 379 Tests gruen, --smoke-ui alle 6 Views gruen. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Richard
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
c7668170a4
commit
a3c145c0ed
@@ -0,0 +1,163 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using PolyTrader.Modules.Accounting.Logic;
|
||||
using PolyTrader.Modules.Accounting.Models;
|
||||
using PolyTrader.Modules.Accounting.Persistence;
|
||||
using PolyTraderSharp;
|
||||
using PolyTraderSharp.Models;
|
||||
using PolyTraderSharp.Services;
|
||||
|
||||
namespace PolyTrader.Modules.Accounting.Services
|
||||
{
|
||||
/// <summary>
|
||||
/// Ingest-Orchestrierung (A-1): erhebt je Live-Account die unabhängige Buchungsgrundlage von
|
||||
/// Polymarket/On-Chain, klassifiziert sie pur (<see cref="AccountingClassifier"/>) und bucht sie
|
||||
/// idempotent in den append-only Ledger. Protokolliert jeden Lauf (acc_ingest_runs) inkl.
|
||||
/// Balance-Anker (Soll-Ist). Rein LESEND — keine Orders, keine On-Chain-Writes.
|
||||
///
|
||||
/// Der eigentliche Abruf liegt hinter Interfaces (IActivitySource/ITransferSource/IBalanceAnchorSource);
|
||||
/// mit den Null-Quellen läuft das Modul offline (bucht korrekt nichts). Der testbare Kern ist
|
||||
/// <see cref="IngestAccountAsync"/>.
|
||||
/// </summary>
|
||||
public sealed class AccountingIngestService : BackgroundService
|
||||
{
|
||||
/// <summary>Sicherheits-Überlappung gegen API-Lag beim inkrementellen Abruf.</summary>
|
||||
internal const int IncrementalLookbackHours = 6;
|
||||
|
||||
private static readonly TimeSpan Interval = TimeSpan.FromHours(1);
|
||||
|
||||
private readonly TradingState _state;
|
||||
private readonly ILedgerRepository _ledger;
|
||||
private readonly IIngestRunRepository _runs;
|
||||
private readonly IRawSnapshotRepository _raw;
|
||||
private readonly IActivitySource _activity;
|
||||
private readonly ITransferSource _transfers;
|
||||
private readonly IBalanceAnchorSource _balance;
|
||||
private readonly AccountingSystemContracts _contracts;
|
||||
private readonly TerminalLogger _logger;
|
||||
|
||||
public AccountingIngestService(
|
||||
TradingState state, ILedgerRepository ledger, IIngestRunRepository runs, IRawSnapshotRepository raw,
|
||||
IActivitySource activity, ITransferSource transfers, IBalanceAnchorSource balance,
|
||||
AccountingSystemContracts contracts, TerminalLogger logger)
|
||||
{
|
||||
_state = state;
|
||||
_ledger = ledger;
|
||||
_runs = runs;
|
||||
_raw = raw;
|
||||
_activity = activity;
|
||||
_transfers = transfers;
|
||||
_balance = balance;
|
||||
_contracts = contracts;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||||
{
|
||||
await Task.Delay(TimeSpan.FromMinutes(3), stoppingToken); // nach Hydration/Trading-Services
|
||||
_logger.Info("Accounting-Ingest gestartet (unabhängiger Polymarket-/On-Chain-Abruf, read-only).");
|
||||
|
||||
while (!stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
try { await IngestAllAsync(backfill: false, stoppingToken); }
|
||||
catch (OperationCanceledException) { break; }
|
||||
catch (Exception ex) { _logger.Error($"Accounting-Ingest Fehler: {ex.Message}"); }
|
||||
|
||||
try { await Task.Delay(Interval, stoppingToken); }
|
||||
catch (OperationCanceledException) { break; }
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Ein Durchlauf über alle Live-Accounts (mit gesetzter WalletAddress).</summary>
|
||||
public async Task IngestAllAsync(bool backfill, CancellationToken ct)
|
||||
{
|
||||
var liveAccounts = _state.Accounts.Values
|
||||
.Where(a => !a.IsDemo && !string.IsNullOrWhiteSpace(a.WalletAddress))
|
||||
.OrderBy(a => a.AccountId)
|
||||
.ToList();
|
||||
|
||||
foreach (var acc in liveAccounts)
|
||||
{
|
||||
if (ct.IsCancellationRequested) break;
|
||||
var run = await IngestAccountAsync(acc, backfill, ct);
|
||||
if (run.NewEntries > 0 || !run.Success)
|
||||
_logger.Info($"📒 [Accounting] {acc.Name}: {run.Message}" +
|
||||
(run.BalanceDeltaUsdc.HasValue ? $" (Balance-Δ {run.BalanceDeltaUsdc:F2} USDC)" : ""));
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Testbarer Kern: erhebt + bucht einen Account, protokolliert den Lauf inkl. Balance-Anker.
|
||||
/// Fehler brechen den Gesamt-Ingest nicht (im Run vermerkt).
|
||||
/// </summary>
|
||||
public async Task<IngestRun> IngestAccountAsync(AccountState account, bool backfill, CancellationToken ct)
|
||||
{
|
||||
var run = new IngestRun { AccountId = account.AccountId, Backfill = backfill, StartedAt = DateTime.UtcNow };
|
||||
_runs.Insert(run); // Id vergeben → dient als IngestBatchId
|
||||
long batchId = run.Id;
|
||||
int newCount = 0, dupCount = 0;
|
||||
|
||||
try
|
||||
{
|
||||
DateTime? since = backfill
|
||||
? null
|
||||
: _ledger.LatestTimestamp(account.AccountId)?.AddHours(-IncrementalLookbackHours);
|
||||
run.FromTimestamp = since;
|
||||
|
||||
// 1) Polymarket-Activity (Trades/Redeems/Rewards/Splits …)
|
||||
var activities = await _activity.GetActivityAsync(account.WalletAddress, since, ct);
|
||||
if (activities.Count > 0)
|
||||
_raw.Insert(new RawSnapshot { IngestRunId = batchId, AccountId = account.AccountId, SourceKind = "activity", Json = SnapshotJson(activities.Select(a => a.RawJson)) });
|
||||
foreach (var a in activities)
|
||||
{
|
||||
var entry = AccountingClassifier.ClassifyActivity(account.AccountId, a, batchId);
|
||||
if (_ledger.Upsert(entry)) newCount++; else dupCount++;
|
||||
}
|
||||
|
||||
// 2) On-Chain-USDC-Transfers → nur EXTERNE als Deposit/Withdrawal
|
||||
var transfers = await _transfers.GetTransfersAsync(account.WalletAddress, since, ct);
|
||||
if (transfers.Count > 0)
|
||||
_raw.Insert(new RawSnapshot { IngestRunId = batchId, AccountId = account.AccountId, SourceKind = "transfers", Json = SnapshotJson(transfers.Select(t => t.RawJson)) });
|
||||
foreach (var t in transfers)
|
||||
{
|
||||
var entry = AccountingClassifier.ClassifyTransfer(account.AccountId, t, batchId, _contracts.Addresses);
|
||||
if (entry == null) continue; // interne Bewegung
|
||||
if (_ledger.Upsert(entry)) newCount++; else dupCount++;
|
||||
}
|
||||
|
||||
// 3) Balance-Anker (Vollständigkeits-Wächter)
|
||||
decimal? anchor = await _balance.GetBalanceAsync(account.WalletAddress, ct);
|
||||
decimal ledgerNet = _ledger.SumNet(account.AccountId);
|
||||
run.BalanceAnchorUsdc = anchor;
|
||||
run.LedgerNetUsdc = ledgerNet;
|
||||
run.BalanceDeltaUsdc = anchor.HasValue ? anchor.Value - ledgerNet : null;
|
||||
|
||||
run.NewEntries = newCount;
|
||||
run.DuplicateEntries = dupCount;
|
||||
run.Success = true;
|
||||
run.Message = $"{newCount} neu, {dupCount} Duplikate ({(backfill ? "Backfill" : "inkrementell")}).";
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
run.Success = false;
|
||||
run.NewEntries = newCount;
|
||||
run.DuplicateEntries = dupCount;
|
||||
run.Message = $"Fehler: {ex.Message}";
|
||||
}
|
||||
|
||||
run.FinishedAt = DateTime.UtcNow;
|
||||
_runs.Update(run);
|
||||
return run;
|
||||
}
|
||||
|
||||
private static string SnapshotJson(IEnumerable<string> rawItems)
|
||||
{
|
||||
var items = rawItems.Where(s => !string.IsNullOrEmpty(s)).ToList();
|
||||
return items.Count == 0 ? "[]" : "[" + string.Join(",", items) + "]";
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using PolyTrader.Modules.Accounting.Models;
|
||||
|
||||
namespace PolyTrader.Modules.Accounting.Services
|
||||
{
|
||||
/// <summary>
|
||||
/// Unabhängige Polymarket-Activity-Quelle (Data-API /activity, ALLE Typen). Interface, damit die
|
||||
/// Buchungslogik ohne Live-API testbar/offline lauffähig ist; die Live-Implementierung (Zielland)
|
||||
/// mappt die reale API-JSON auf <see cref="RawActivity"/> und speichert den Rohschnappschuss.
|
||||
/// </summary>
|
||||
public interface IActivitySource
|
||||
{
|
||||
/// <summary>Activity ab <paramref name="since"/> (null = volle Historie/Backfill), paginiert.</summary>
|
||||
Task<IReadOnlyList<RawActivity>> GetActivityAsync(string wallet, DateTime? since, CancellationToken ct);
|
||||
}
|
||||
|
||||
/// <summary>On-Chain-USDC-Transfers des Safe-Wallets (ERC-20-Logs). Live-Impl über Alchemy (Zielland).</summary>
|
||||
public interface ITransferSource
|
||||
{
|
||||
Task<IReadOnlyList<RawTransfer>> GetTransfersAsync(string wallet, DateTime? since, CancellationToken ct);
|
||||
}
|
||||
|
||||
/// <summary>On-Chain-USDC-Saldo als Balance-Anker (Soll-Ist). Live-Impl über GetUsdcBalanceAsync.</summary>
|
||||
public interface IBalanceAnchorSource
|
||||
{
|
||||
Task<decimal?> GetBalanceAsync(string wallet, CancellationToken ct);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Offline-Stubs: das Modul läuft ohne Live-Anbindung vollständig (Ingest bucht dann korrekt nichts).
|
||||
/// Im Zielland werden die echten Quellen registriert (Muster wie NullFarmingMarketSource).
|
||||
/// </summary>
|
||||
public sealed class NullActivitySource : IActivitySource
|
||||
{
|
||||
public Task<IReadOnlyList<RawActivity>> GetActivityAsync(string wallet, DateTime? since, CancellationToken ct)
|
||||
=> Task.FromResult((IReadOnlyList<RawActivity>)Array.Empty<RawActivity>());
|
||||
}
|
||||
|
||||
public sealed class NullTransferSource : ITransferSource
|
||||
{
|
||||
public Task<IReadOnlyList<RawTransfer>> GetTransfersAsync(string wallet, DateTime? since, CancellationToken ct)
|
||||
=> Task.FromResult((IReadOnlyList<RawTransfer>)Array.Empty<RawTransfer>());
|
||||
}
|
||||
|
||||
public sealed class NullBalanceAnchorSource : IBalanceAnchorSource
|
||||
{
|
||||
public Task<decimal?> GetBalanceAsync(string wallet, CancellationToken ct) => Task.FromResult((decimal?)null);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Whitelist der Polymarket-System-Contract-Adressen (lowercased). Transfers zu/von diesen Adressen
|
||||
/// sind interne Trading-Bewegungen (bereits in der Activity), KEINE Ein-/Auszahlungen. Wird im
|
||||
/// Zielland aus Config/Docs befüllt/verifiziert (CTF-Adresse liegt bereits im Core; Exchange/
|
||||
/// NegRisk-Adapter/USDC ergänzen). Default leer → ohne Live-Transfers unkritisch.
|
||||
/// </summary>
|
||||
public sealed class AccountingSystemContracts
|
||||
{
|
||||
public HashSet<string> Addresses { get; } = new(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
public AccountingSystemContracts(IEnumerable<string>? addresses = null)
|
||||
{
|
||||
if (addresses == null) return;
|
||||
foreach (var a in addresses)
|
||||
if (!string.IsNullOrWhiteSpace(a)) Addresses.Add(a.Trim().ToLowerInvariant());
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user