Sync from development - prepare for v0.5.0

This commit is contained in:
Omni
2026-03-13 14:43:25 +00:00
parent 411addeea2
commit 3556914560
179 changed files with 7126 additions and 1806 deletions

View File

@@ -1,5 +1,56 @@
# Jackify Changelog
## v0.5.0 - Non-Premium Support, Modlist Update Handling and Overall Reliability Improvements
**Release Date:** 13/03/26
### New in v0.5.0
- Full non-premium install support in both GUI and CLI. Feedback is welcome on this new feature, both positive and negative
- New Jackify Download Manager for Non-Premium accounts, or files Jackify cannot auto-download.
- Improved modlist update handling so existing installs are detected more reliably and Jackify can reuse the existing setup instead of creating duplicate Steam shortcuts.
- Improved Viva New Vegas automation across GUI and CLI paths.
- Improved Wabbajack and Mod Organizer 2 standalone installation workflows.
- Better guidance when Skyrim AE/CC content is missing.
- Further improvements on user-facing logging and error handling
### Manual Download Improvements
- Handles manual downloads more smoothly from start to finish:
- opens required links for you in your system Browser
- watches your download folder
- verifies files and moves them to the correct location automatically,continues with the rest of the modlist install when ready
- Better controls in both GUI and CLI:
- pause/resume download flows, or defer individual archives (useful if one is temporarily unavailable)
- retry deferred items
- reopen file links
- change concurrent browser tab count
- change watch folder
- Deferred items (e.g temporarily unavailable) are retried correctly on later retry/recheck passes.
### Update and Install Reliability
- Worked to improve feature parity between the GUI and CLI frontends, tidying up a few edge cases where CLI behavior did not yet match GUI workflows closely enough.
- Improved update messaging (clearer wording on success/failure).
- Better cancellation handling so stopping a workflow is less likely to leave background processes running.
- Better focus recovery after Steam restart in key workflows.
- Better handling when both Flatpak and native Steam are installed: Jackify now prefers the Steam install that actually contains your installed games, with safe fallback rules if both look valid.
- Install Proton selection now self-heals on startup if the configured Proton was removed, automatically falling back to the best available installed Proton.
- For `/var/home`-based installs (for example Bazzite layouts), ModOrganizer.ini path basis is now aligned so executable/working/game paths resolve correctly.
### Nexus Authentication
- OAuth protocol handler desktop file is now updated if the registered AppImage path no longer matches the current location, preventing silent callback failures after the AppImage is moved or renamed.
- OAuth waiting dialog now includes a "Paste callback URL" button for manual fallback if the browser does not dispatch the jackify:// callback automatically.
### Logging and Error Quality
- Better targeted guidance when required prerequisites or content are missing.
- Improved logging around updater source selection (Nexus/GitHub fallback behavior).
- Better error context while keeping sensitive tokens/keys redacted.
- Install failure fallback now surfaces recent actionable engine output (and resource-limit warnings) instead of only a generic exit-code message.
### Updated jackify-engine to 0.5.0:
- Improved non-premium/manual-download support through a structured manual-download protocol that lets Jackify pause, guide the user, recheck files, and continue installation cleanly once required archives are present.
- Better pre-flight validation before large downloads begin, including earlier checks for game availability, disk space, and filesystem path-length limits.
- More accurate structured error handling for installation failures, with better classification of storage, permission, network, authentication, and validation issues.
---
## v0.4.0 - Error Handling Rewrite
**Release Date:** 2026-02-25

View File

@@ -44,7 +44,7 @@ Jackify is a Linux application for installing and configuring Wabbajack modlists
- Non-Premium users can still install modlists via Wabbajack under Proton
- Native non-premium support planned for a future release
- See the [User Guide](https://github.com/Omni-guides/Jackify/wiki/User-Guide) for full details on the options available
- **FUSE** (required for AppImage execution, pre-installed on most distributions)
- **FUSE2 compatibility (libfuse.so.2) is required for AppImage execution**
- **Ubuntu/Debian-based distros only** (Ubuntu, Kubuntu, Linux Mint, Pop!_OS, Zorin OS, elementary OS, and others): Qt platform plugin library
- `sudo apt install libxcb-cursor-dev`

View File

Before

Width:  |  Height:  |  Size: 150 KiB

After

Width:  |  Height:  |  Size: 150 KiB

View File

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 100 KiB

View File

Before

Width:  |  Height:  |  Size: 30 KiB

After

Width:  |  Height:  |  Size: 30 KiB

View File

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 77 KiB

View File

Before

Width:  |  Height:  |  Size: 44 KiB

After

Width:  |  Height:  |  Size: 44 KiB

View File

Before

Width:  |  Height:  |  Size: 7.2 KiB

After

Width:  |  Height:  |  Size: 7.2 KiB

View File

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 17 KiB

View File

Before

Width:  |  Height:  |  Size: 117 KiB

After

Width:  |  Height:  |  Size: 117 KiB

View File

Before

Width:  |  Height:  |  Size: 33 KiB

After

Width:  |  Height:  |  Size: 33 KiB

View File

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 27 KiB

View File

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

View File

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 13 KiB

View File

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 67 KiB

View File

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

View File

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 88 KiB

View File

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 47 KiB

View File

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 69 KiB

View File

Before

Width:  |  Height:  |  Size: 7.2 KiB

After

Width:  |  Height:  |  Size: 7.2 KiB

View File

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 31 KiB

View File

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

View File

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

Before

Width:  |  Height:  |  Size: 30 KiB

After

Width:  |  Height:  |  Size: 30 KiB

View File

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

View File

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 130 KiB

View File

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

View File

Before

Width:  |  Height:  |  Size: 8.7 KiB

After

Width:  |  Height:  |  Size: 8.7 KiB

View File

Before

Width:  |  Height:  |  Size: 138 KiB

After

Width:  |  Height:  |  Size: 138 KiB

View File

Before

Width:  |  Height:  |  Size: 154 KiB

After

Width:  |  Height:  |  Size: 154 KiB

View File

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 100 KiB

View File

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

View File

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 130 KiB

View File

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

View File

Before

Width:  |  Height:  |  Size: 8.7 KiB

After

Width:  |  Height:  |  Size: 8.7 KiB

View File

Before

Width:  |  Height:  |  Size: 405 KiB

After

Width:  |  Height:  |  Size: 405 KiB

View File

@@ -5,4 +5,4 @@ This package provides both CLI and GUI interfaces for managing
Wabbajack modlists natively on Linux systems.
"""
__version__ = "0.4.0"
__version__ = "0.5.0"

View File

@@ -107,7 +107,7 @@ def get_jackify_engine_path():
logger.warning(f"AppImage engine not found at expected path: {engine_path}")
# Priority 3: Check if THIS process is actually running from Jackify AppImage
# (not just inheriting APPDIR from another AppImage like Cursor)
# (not just inheriting APPDIR from another AppImage context)
appdir = os.environ.get('APPDIR')
if appdir and sys.argv[0] and 'jackify' in sys.argv[0].lower() and '/tmp/.mount_' in sys.argv[0]:
# Only use AppImage path if we're actually running a Jackify AppImage
@@ -179,6 +179,92 @@ class ModlistInstallCLI(
# Initialize process tracking for cleanup
self._current_process = None
@staticmethod
def _normalize_version_token(value: str | None) -> str | None:
if value is None:
return None
token = str(value).strip()
if not token:
return None
return token.lstrip("vV").lower()
@staticmethod
def _normalize_modlist_name(value: str | None) -> str:
return " ".join((value or "").strip().lower().split())
def _get_requested_modlist_version(self) -> str | None:
info = self.context.get("selected_modlist_info") or {}
return self._normalize_version_token(info.get("version"))
def _evaluate_update_candidate(
self,
modlist_name: str,
install_dir: str,
existing_appid: str | None,
) -> tuple[bool, dict]:
from jackify.backend.utils.modlist_meta import read_modlist_meta
result = {
"eligible": False,
"reason": "unknown",
"requested_version": None,
"installed_version": None,
"version_relation": "unknown",
"installed_name": None,
}
if not existing_appid:
result["reason"] = "missing_shortcut_appid"
return False, result
meta = read_modlist_meta(install_dir)
if not meta:
result["reason"] = "missing_meta"
return False, result
installed_name = (meta.get("modlist_name") or "").strip()
result["installed_name"] = installed_name
if self._normalize_modlist_name(installed_name) != self._normalize_modlist_name(modlist_name):
result["reason"] = "modlist_name_mismatch"
return False, result
requested_version = self._get_requested_modlist_version()
installed_version = self._normalize_version_token(meta.get("modlist_version"))
result["requested_version"] = requested_version
result["installed_version"] = installed_version
if requested_version and installed_version:
result["version_relation"] = "same" if requested_version == installed_version else "different"
result["eligible"] = True
result["reason"] = "eligible"
return True, result
def _find_existing_shortcut_appid(self, modlist_name: str, install_dir: str) -> str | None:
try:
install_real = os.path.realpath(install_dir)
candidate_exes = [
os.path.join(install_real, "ModOrganizer.exe"),
os.path.join(install_real, "files", "ModOrganizer.exe"),
]
for exe_path in candidate_exes:
if not os.path.exists(exe_path):
continue
appid = self.shortcut_handler.get_appid_from_vdf(modlist_name, exe_path)
if appid:
return appid
for shortcut in self.shortcut_handler.find_shortcuts_by_exe("ModOrganizer.exe"):
if (
shortcut.get("AppName", "").strip() == modlist_name.strip()
and os.path.realpath(shortcut.get("StartDir", "")) == install_real
):
raw_appid = shortcut.get("appid")
if raw_appid is not None:
return str(int(raw_appid) & 0xFFFFFFFF)
except Exception as e:
self.logger.warning("CLI update detection: failed shortcut lookup: %s", e)
return None
def cleanup(self):
"""Clean up any running jackify-engine process"""
if self._current_process and self._current_process.poll() is None:

View File

@@ -1,4 +1,5 @@
"""CLI configuration phase methods for ModlistInstallCLI (Mixin)."""
import json
import logging
import os
import subprocess
@@ -166,19 +167,81 @@ class ModlistOperationsConfigurationCLIMixin:
from jackify.backend.handlers.subprocess_utils import get_clean_subprocess_env
clean_env = get_clean_subprocess_env()
self._current_process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=False, env=clean_env, cwd=engine_dir)
self._current_process = subprocess.Popen(
cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=False,
env=clean_env,
cwd=engine_dir,
)
proc = self._current_process
def _write_stdin(payload: str) -> bool:
if not proc.stdin or proc.poll() is not None:
return False
try:
proc.stdin.write((payload + '\n').encode('utf-8'))
proc.stdin.flush()
return True
except Exception:
self.logger.debug("Failed writing to engine stdin", exc_info=True)
return False
buffer = b''
inline_progress_active = False
pending_manual = []
while True:
chunk = proc.stdout.read(1)
if not chunk:
break
buffer += chunk
if chunk == b'\n':
if chunk in (b'\n', b'\r'):
line = buffer.decode('utf-8', errors='replace')
decoded = line.rstrip('\r\n')
if decoded.startswith('{'):
try:
event = json.loads(decoded)
except (json.JSONDecodeError, ValueError):
event = None
if event:
event_name = event.get('event')
if event_name == 'manual_download_required':
pending_manual.append(event)
buffer = b''
continue
if event_name == 'manual_download_list_complete':
loop_iter = event.get('loop_iteration', 1)
for item in pending_manual:
item['loop_iteration'] = loop_iter
from jackify.backend.handlers.config_handler import ConfigHandler
raw_limit = ConfigHandler().get('manual_download_concurrent_limit', 2)
try:
manual_limit = int(raw_limit)
except (TypeError, ValueError):
manual_limit = 2
from jackify.frontends.cli.commands.manual_download_flow import run_cli_manual_download_phase
completed = run_cli_manual_download_phase(
events=list(pending_manual),
loop_iteration=loop_iter,
download_dir=actual_download_path,
stdin_write=_write_stdin,
concurrent_limit=max(1, min(5, manual_limit)),
)
if not completed:
if proc.poll() is None:
proc.terminate()
buffer = b''
break
pending_manual.clear()
buffer = b''
continue
if event_name == 'manual_download_phase_complete':
print("All manual downloads confirmed. Resuming installation...")
buffer = b''
continue
if '[FILE_PROGRESS]' in line:
parts = line.split('[FILE_PROGRESS]', 1)
if parts[0].strip():
@@ -197,26 +260,6 @@ class ModlistOperationsConfigurationCLIMixin:
inline_progress_active = False
print(line, end='')
buffer = b''
elif chunk == b'\r':
line = buffer.decode('utf-8', errors='replace')
if '[FILE_PROGRESS]' in line:
parts = line.split('[FILE_PROGRESS]', 1)
if parts[0].strip():
line = parts[0].rstrip()
else:
buffer = b''
continue
clean_line = line.rstrip('\r\n')
if clean_line.startswith("Installing files "):
print(f"\r{clean_line}", end='')
inline_progress_active = True
else:
if inline_progress_active:
print()
inline_progress_active = False
print(line, end='')
sys.stdout.flush()
buffer = b''
if buffer:
line = buffer.decode('utf-8', errors='replace')
@@ -400,6 +443,16 @@ class ModlistOperationsConfigurationCLIMixin:
app_id = None
use_automated_prefix = os.environ.get('JACKIFY_USE_AUTOMATED_PREFIX', '1') == '1'
existing_shortcut_appid = self.context.get('existing_shortcut_appid')
update_existing_install = bool(self.context.get('update_existing_install'))
if update_existing_install and existing_shortcut_appid:
app_id = str(existing_shortcut_appid)
success = True
prefix_path = None
result = True
print(f"\n{COLOR_INFO}Update mode selected. Reusing existing Steam shortcut AppID {app_id}.{COLOR_RESET}")
use_automated_prefix = False
if use_automated_prefix:
print(f"\n{COLOR_INFO}Using automated Steam setup workflow...{COLOR_RESET}")
@@ -535,17 +588,20 @@ class ModlistOperationsConfigurationCLIMixin:
success, prefix_path, app_id = True, None, None
else:
success, prefix_path, app_id = False, None, None
if success:
if success:
if update_existing_install and app_id:
print(f"{COLOR_SUCCESS}Update mode Steam setup confirmed.{COLOR_RESET}")
print(f"{COLOR_INFO}Reusing Steam AppID: {app_id}{COLOR_RESET}")
else:
print(f"{COLOR_SUCCESS}Automated Steam setup completed successfully!{COLOR_RESET}")
if prefix_path:
print(f"{COLOR_INFO}Proton prefix created at: {prefix_path}{COLOR_RESET}")
if app_id:
print(f"{COLOR_INFO}Steam AppID: {app_id}{COLOR_RESET}")
else:
print(f"{COLOR_ERROR}Automated Steam setup failed. Result: {result}{COLOR_RESET}")
print(f"{COLOR_ERROR}Steam integration was not completed. Please check the logs for details.{COLOR_RESET}")
return
else:
print(f"{COLOR_ERROR}Automated Steam setup failed. Result: {result}{COLOR_RESET}")
print(f"{COLOR_ERROR}Steam integration was not completed. Please check the logs for details.{COLOR_RESET}")
return
from jackify.backend.services.modlist_service import ModlistService
from jackify.backend.models.modlist import ModlistContext
@@ -572,18 +628,28 @@ class ModlistOperationsConfigurationCLIMixin:
progress_callback("")
progress_callback("=== Configuration Phase ===")
print(f"\n{COLOR_INFO}=== Configuration Phase ==={COLOR_RESET}")
self.logger.info("Running post-installation configuration phase using ModlistService")
print(f"\n{COLOR_INFO}=== Configuration Phase ==={COLOR_RESET}")
self.logger.info("Running post-installation configuration phase using ModlistService")
configuration_success = modlist_service.configure_modlist_post_steam(modlist_context)
if configuration_success:
print(f"{COLOR_SUCCESS}Configuration completed successfully!{COLOR_RESET}")
self.logger.info("Post-installation configuration completed successfully")
print(f"{COLOR_INFO}Core configuration complete. Checking post-install automation...{COLOR_RESET}")
try:
# Ensure CLI install flow gets the same VNV automation behavior as GUI.
from jackify.backend.services.vnv_integration_helper import run_vnv_automation_if_applicable
from jackify.backend.services.vnv_integration_helper import (
run_vnv_automation_if_applicable,
should_offer_vnv_automation,
)
from jackify.backend.services.automated_prefix_service import AutomatedPrefixService
from jackify.backend.services.vnv_post_install_service import VNVPostInstallService
from jackify.backend.handlers.path_handler import PathHandler
from jackify.frontends.cli.commands.vnv_manual_downloads import (
build_vnv_cli_manual_file_callback,
create_vnv_cli_progress_callback,
ensure_vnv_cli_manual_downloads,
)
modlist_name_for_automation = self.context.get('modlist_name') or shortcut_name or ""
def _confirm_vnv(description: str) -> bool:
@@ -593,31 +659,47 @@ class ModlistOperationsConfigurationCLIMixin:
except (EOFError, KeyboardInterrupt):
return False
return user_input in ("", "y", "yes")
def _manual_vnv_file(title: str, instructions: str):
print(f"\n{COLOR_WARNING}{title}{COLOR_RESET}")
print(instructions)
try:
file_input = input(f"{COLOR_PROMPT}Path to downloaded file: {COLOR_RESET}").strip()
except (EOFError, KeyboardInterrupt):
return None
if not file_input:
return None
selected = Path(file_input).expanduser().resolve()
return selected if selected.exists() else None
automation_ran, vnv_error = run_vnv_automation_if_applicable(
modlist_name=modlist_name_for_automation,
modlist_install_location=Path(install_dir_str),
game_root=None, # Auto-detect from modlist structure.
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
progress_callback=lambda msg: print(msg),
manual_file_callback=_manual_vnv_file,
confirmation_callback=_confirm_vnv,
)
if automation_ran and not vnv_error:
print(f"{COLOR_INFO}VNV post-install automation completed.{COLOR_RESET}")
if vnv_error:
print(f"{COLOR_WARNING}VNV automation encountered an error: {vnv_error}{COLOR_RESET}")
print(f"{COLOR_INFO}You can complete these steps manually by following: https://vivanewvegas.moddinglinked.com/wabbajack.html{COLOR_RESET}")
install_path = Path(install_dir_str)
if should_offer_vnv_automation(modlist_name_for_automation, install_path):
game_paths = PathHandler().find_vanilla_game_paths()
resolved_game_root = game_paths.get('Fallout New Vegas')
vnv_service = VNVPostInstallService(
modlist_install_location=install_path,
game_root=resolved_game_root or install_path,
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
)
completed = vnv_service.check_already_completed()
all_vnv_steps_done = (
completed['root_mods']
and completed['4gb_patch']
and completed['bsa_decompressed']
)
if all_vnv_steps_done:
print(f"{COLOR_INFO}VNV post-install steps are already complete.{COLOR_RESET}")
elif _confirm_vnv(vnv_service.get_automation_description()):
if not ensure_vnv_cli_manual_downloads(vnv_service, output_callback=print):
print(f"{COLOR_WARNING}VNV manual downloads were not completed. Skipping VNV automation.{COLOR_RESET}")
else:
progress_callback, close_progress = create_vnv_cli_progress_callback(print)
try:
automation_ran, vnv_error = run_vnv_automation_if_applicable(
modlist_name=modlist_name_for_automation,
modlist_install_location=install_path,
game_root=None, # Auto-detect from modlist structure.
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
progress_callback=progress_callback,
manual_file_callback=build_vnv_cli_manual_file_callback(vnv_service, output_callback=print),
confirmation_callback=lambda _description: True,
)
finally:
close_progress()
if automation_ran and not vnv_error:
print(f"{COLOR_INFO}VNV post-install automation completed.{COLOR_RESET}")
if vnv_error:
print(f"{COLOR_WARNING}VNV automation encountered an error: {vnv_error}{COLOR_RESET}")
print(f"{COLOR_INFO}You can complete these steps manually by following: https://vivanewvegas.moddinglinked.com/wabbajack.html{COLOR_RESET}")
else:
print(f"{COLOR_INFO}VNV automation skipped by user.{COLOR_RESET}")
except Exception as vnv_err:
self.logger.error("VNV post-install automation failed: %s", vnv_err, exc_info=True)
print(f"{COLOR_WARNING}VNV automation could not be completed. Check logs for details.{COLOR_RESET}")
@@ -632,6 +714,7 @@ class ModlistOperationsConfigurationCLIMixin:
except Exception as ttw_err:
self.logger.error("TTW post-install prompt failed: %s", ttw_err, exc_info=True)
print(f"{COLOR_WARNING}TTW integration prompt failed. Check logs for details.{COLOR_RESET}")
print(f"{COLOR_SUCCESS}Configuration completed successfully!{COLOR_RESET}")
else:
print(f"{COLOR_WARNING}Configuration had some issues but completed.{COLOR_RESET}")
self.logger.warning("Post-installation configuration had issues")

View File

@@ -68,7 +68,7 @@ class ModlistOperationsConfigurationGUIMixin:
if result:
if completion_callback:
completion_callback(True, "Configuration completed successfully!", config_context['name'])
completion_callback(True, "Core configuration complete", config_context['name'])
return True
else:
retry_count += 1
@@ -139,7 +139,7 @@ class ModlistOperationsConfigurationGUIMixin:
if result:
if completion_callback:
completion_callback(True, "Configuration completed successfully!", config_context['name'])
completion_callback(True, "Core configuration complete", config_context['name'])
return True
else:
if progress_callback:

View File

@@ -243,6 +243,46 @@ class ModlistOperationsDiscoveryMixin:
self.context['download_dir'] = download_dir_path
self.logger.debug(f"Download directory context set to: {self.context['download_dir']}")
install_dir_value = self.context.get('install_dir')
install_dir_real = os.path.realpath(str(install_dir_value[0] if isinstance(install_dir_value, tuple) else install_dir_value))
existing_appid = self._find_existing_shortcut_appid(self.context['modlist_name'], install_dir_real)
eligible_update, update_meta = self._evaluate_update_candidate(
self.context['modlist_name'],
install_dir_real,
existing_appid,
)
if eligible_update:
print("\n" + "-" * 28)
print(f"{COLOR_WARNING}Existing modlist installation detected in this directory.{COLOR_RESET}")
relation = update_meta.get("version_relation")
if relation == "different":
print(
f"{COLOR_INFO}Detected version change: installed v{update_meta.get('installed_version')} -> "
f"selected v{update_meta.get('requested_version')}.{COLOR_RESET}"
)
elif relation == "same" and update_meta.get("installed_version"):
print(
f"{COLOR_INFO}Detected same version (v{update_meta.get('installed_version')}). "
"Use update mode for repair/reconfigure behavior." + f"{COLOR_RESET}"
)
print("Choose how to proceed:")
print(" 1. Update existing install (recommended)")
print(" 2. New install with a different Steam shortcut name")
print(" 0. Cancel")
update_choice = input(f"{COLOR_PROMPT}Enter your selection (0-2): {COLOR_RESET}").strip()
if update_choice == "1":
self.context['update_existing_install'] = True
self.context['existing_shortcut_appid'] = existing_appid
self.logger.info("CLI update mode selected; reusing AppID %s", existing_appid)
elif update_choice == "2":
print(
f"{COLOR_WARNING}For a new install, choose a different Modlist Name before proceeding.{COLOR_RESET}"
)
return None
else:
self.logger.info("User cancelled at CLI update detection prompt.")
return None
if 'nexus_api_key' not in self.context or not self.context.get('nexus_api_key'):
from jackify.backend.services.nexus_auth_service import NexusAuthService
auth_service = NexusAuthService()

View File

@@ -17,6 +17,10 @@ from typing import Optional
from .config_handler_encryption import ConfigEncryptionMixin
from .config_handler_directories import ConfigDirectoriesMixin
from .config_handler_proton import ConfigProtonMixin
from jackify.shared.steam_utils import (
STEAM_PREFERENCE_AUTO,
resolve_preferred_steam_installation,
)
logger = logging.getLogger(__name__)
@@ -50,6 +54,7 @@ class ConfigHandler(ConfigEncryptionMixin, ConfigDirectoriesMixin, ConfigProtonM
"resolution": None,
"protontricks_path": None,
"steam_path": None,
"steam_install_preference": STEAM_PREFERENCE_AUTO, # auto|flatpak|native
"nexus_api_key": None, # Base64 encoded API key
"default_install_parent_dir": None, # Parent directory for modlist installations
"default_download_parent_dir": None, # Parent directory for downloads
@@ -62,6 +67,8 @@ class ConfigHandler(ConfigEncryptionMixin, ConfigDirectoriesMixin, ConfigProtonM
"proton_path": None, # Install Proton path (for jackify-engine) - None means auto-detect
"proton_version": None, # Install Proton version name - None means auto-detect
"steam_restart_strategy": "jackify", # "jackify" (default) or "simple"
"manual_download_concurrent_limit": 2, # Shared GUI/CLI default for manual download browser tabs
"manual_download_watch_directory": None, # Optional override for manual-download watcher folder
"window_width": None, # Saved window width (None = use dynamic sizing)
"window_height": None # Saved window height (None = use dynamic sizing)
}
@@ -72,14 +79,13 @@ class ConfigHandler(ConfigEncryptionMixin, ConfigDirectoriesMixin, ConfigProtonM
# Perform version migrations
self._migrate_config()
# Normalize/repair Proton selections on every startup so stale deleted versions
# cannot break workflows.
self.normalize_proton_paths_on_boot()
# If steam_path is not set, detect it
if not self.settings["steam_path"]:
self.settings["steam_path"] = self._detect_steam_path()
# Auto-detect and set Proton version ONLY on first run (config file doesn't exist)
# Do NOT overwrite user's saved settings!
if not os.path.exists(self.config_file) and not self.settings.get("proton_path"):
self._auto_detect_proton()
# If jackify_data_dir is not set, initialize it to default
if not self.settings.get("jackify_data_dir"):
@@ -95,35 +101,16 @@ class ConfigHandler(ConfigEncryptionMixin, ConfigDirectoriesMixin, ConfigProtonM
str: Path to the Steam installation or None if not found
"""
logger.info("Detecting Steam installation path...")
# Common Steam installation paths
steam_paths = [
os.path.expanduser("~/.steam/steam"),
os.path.expanduser("~/.local/share/Steam"),
os.path.expanduser("~/.steam/root")
]
# Check each path
for path in steam_paths:
if os.path.exists(path):
logger.info(f"Found Steam installation at: {path}")
return path
# If not found in common locations, try to find using libraryfolders.vdf
libraryfolders_vdf_paths = [
os.path.expanduser("~/.steam/steam/config/libraryfolders.vdf"),
os.path.expanduser("~/.local/share/Steam/config/libraryfolders.vdf"),
os.path.expanduser("~/.steam/root/config/libraryfolders.vdf"),
os.path.expanduser("~/.var/app/com.valvesoftware.Steam/.local/share/Steam/config/libraryfolders.vdf") # Flatpak
]
for vdf_path in libraryfolders_vdf_paths:
if os.path.exists(vdf_path):
# Extract the Steam path from the libraryfolders.vdf path
steam_path = os.path.dirname(os.path.dirname(vdf_path))
logger.info(f"Found Steam installation at: {steam_path}")
return steam_path
preference = self.settings.get("steam_install_preference", STEAM_PREFERENCE_AUTO)
install_type, install_root = resolve_preferred_steam_installation(preference=preference)
if install_root:
logger.info(
"Selected Steam installation: %s (%s)",
install_type,
install_root,
)
return str(install_root)
logger.error("Steam installation not found")
return None
@@ -376,4 +363,4 @@ class ConfigHandler(ConfigEncryptionMixin, ConfigDirectoriesMixin, ConfigProtonM

View File

@@ -3,6 +3,8 @@ Config handler Proton path and version getters and auto-detect.
"""
import logging
from pathlib import Path
from typing import Optional, Dict, Any
logger = logging.getLogger(__name__)
@@ -10,6 +12,105 @@ logger = logging.getLogger(__name__)
class ConfigProtonMixin:
"""Mixin providing Proton path/version and auto-detect for ConfigHandler."""
@staticmethod
def _is_usable_proton_path(proton_path: Optional[str]) -> bool:
"""Return True when path looks like a valid Proton install directory."""
if not proton_path:
return False
try:
p = Path(str(proton_path)).expanduser()
if not p.is_dir():
return False
# Valve Proton structure
if (p / "dist" / "bin" / "wine").exists():
return True
# GE-Proton structure
if (p / "files" / "bin" / "wine").exists():
return True
return False
except Exception:
return False
@staticmethod
def _best_proton_entry() -> Optional[Dict[str, Any]]:
"""Get best detected Proton entry or None."""
try:
from .wine_utils import WineUtils
return WineUtils.select_best_proton()
except Exception:
return None
def normalize_proton_paths_on_boot(self) -> bool:
"""
Ensure stored Proton paths are valid at startup, repairing stale selections.
Rules:
- If install proton path is missing/invalid, auto-detect next best and persist it.
- If no compatible Proton exists, persist install path/version as null.
- If game proton path is set and invalid, reset it to install proton (or null).
Returns:
True if config values were changed and saved, False otherwise.
"""
changed = False
install_path = self.settings.get("proton_path")
if install_path == "auto":
install_path = None
install_valid = self._is_usable_proton_path(install_path)
if not install_valid:
best = self._best_proton_entry()
if best:
best_path = str(best["path"])
best_name = str(best.get("name") or Path(best_path).name)
if self.settings.get("proton_path") != best_path:
self.settings["proton_path"] = best_path
changed = True
if self.settings.get("proton_version") != best_name:
self.settings["proton_version"] = best_name
changed = True
logger.warning(
"Install Proton path was missing/invalid; auto-selected %s (%s)",
best_name,
best_path,
)
else:
if self.settings.get("proton_path") is not None:
self.settings["proton_path"] = None
changed = True
if self.settings.get("proton_version") is not None:
self.settings["proton_version"] = None
changed = True
logger.warning(
"Install Proton path was missing/invalid and no compatible Proton was found"
)
else:
# Keep proton_version in sync with existing valid path when missing/legacy.
if not self.settings.get("proton_version"):
self.settings["proton_version"] = Path(str(install_path)).name
changed = True
effective_install = self.settings.get("proton_path")
game_path = self.settings.get("game_proton_path")
# Legacy/placeholder values should not persist for runtime resolution.
if game_path in ("same_as_install", "auto"):
target = effective_install
if self.settings.get("game_proton_path") != target:
self.settings["game_proton_path"] = target
changed = True
elif game_path and not self._is_usable_proton_path(game_path):
self.settings["game_proton_path"] = effective_install
changed = True
logger.warning(
"Game Proton path was missing/invalid; reset to install Proton path"
)
if changed:
self.save_config()
return changed
def get_proton_path(self):
"""Retrieve the saved Install Proton path. Always reads fresh from disk."""
try:

View File

@@ -279,46 +279,56 @@ class ModlistMenuHandler:
timestamp = f"[{hours:02d}:{minutes:02d}:{seconds:02d}]"
print(f"{COLOR_INFO}{timestamp} {message}{COLOR_RESET}")
# Run the automated workflow
result = prefix_service.run_working_workflow(
modlist_name, install_dir, mo2_path, progress_callback, steamdeck=self.steamdeck
)
# Handle the result
if isinstance(result, tuple) and len(result) == 4:
if result[0] == "CONFLICT":
# Handle conflict - ask user what to do
conflicts = result[1]
print(f"\n{COLOR_WARNING}Found existing Steam shortcut(s) with the same name and path:{COLOR_RESET}")
for i, conflict in enumerate(conflicts, 1):
print(f" {i}. Name: {conflict['name']}")
print(f" Executable: {conflict['exe']}")
print(f" Start Directory: {conflict['startdir']}")
print(f"\n{COLOR_PROMPT}Options:{COLOR_RESET}")
print(" 1. Use existing shortcut (recommended)")
print(" 2. Create new shortcut anyway")
choice = input(f"{COLOR_PROMPT}Enter choice (1-2): {COLOR_RESET}").strip()
if choice == "1":
# Use existing shortcut
existing_appid = conflicts[0].get('appid')
if existing_appid:
context = {
"name": modlist_name,
"appid": str(existing_appid),
"path": mo2_dir,
"manual_steps_completed": True,
"resolution": None
}
return self.run_modlist_configuration_phase(context)
elif choice == "2":
# Create new shortcut - would need to handle this, but for now just fail
print(f"{COLOR_ERROR}Creating new shortcut with same name not supported in this flow.{COLOR_RESET}")
return True
else:
while True:
result = prefix_service.run_working_workflow(
modlist_name, install_dir, mo2_path, progress_callback, steamdeck=self.steamdeck
)
if isinstance(result, tuple) and len(result) == 4:
if result[0] == "CONFLICT":
conflicts = result[1]
print(f"\n{COLOR_WARNING}Found existing Steam shortcut(s) with the same name and path:{COLOR_RESET}")
for i, conflict in enumerate(conflicts, 1):
print(f" {i}. Name: {conflict['name']}")
print(f" Executable: {conflict['exe']}")
print(f" Start Directory: {conflict['startdir']}")
print(f"\n{COLOR_PROMPT}Options:{COLOR_RESET}")
print(" 1. Use existing shortcut (recommended)")
print(" 2. Choose a different shortcut name")
choice = input(f"{COLOR_PROMPT}Enter choice (1-2): {COLOR_RESET}").strip()
if choice == "1":
existing_appid = conflicts[0].get('appid')
if existing_appid:
context = {
"name": modlist_name,
"appid": str(existing_appid),
"path": mo2_dir,
"manual_steps_completed": True,
"resolution": None
}
return self.run_modlist_configuration_phase(context)
print(f"{COLOR_ERROR}Could not determine existing shortcut AppID.{COLOR_RESET}")
return True
if choice == "2":
print("")
print(f"{COLOR_PROMPT}Enter a different shortcut name for this modlist.{COLOR_RESET}")
print(f"{COLOR_INFO}(Current conflicting name: {modlist_name}){COLOR_RESET}")
new_name = input(f"{COLOR_PROMPT}New shortcut name (or 'q' to cancel): {COLOR_RESET}").strip()
if new_name.lower() == 'q':
print(f"{COLOR_INFO}Configuration cancelled by user.{COLOR_RESET}")
return True
if not new_name:
print(f"{COLOR_ERROR}Name cannot be empty.{COLOR_RESET}")
continue
if new_name == modlist_name:
print(f"{COLOR_ERROR}Please enter a different name to resolve the conflict.{COLOR_RESET}")
continue
modlist_name = new_name
print(f"{COLOR_INFO}Retrying Steam setup with shortcut name: {modlist_name}{COLOR_RESET}")
continue
print(f"{COLOR_ERROR}Invalid choice.{COLOR_RESET}")
return True
else:
# Success - get the results
success, prefix_path, appid_int, last_timestamp = result
if success and appid_int:
context = {
@@ -330,10 +340,9 @@ class ModlistMenuHandler:
}
self.logger.debug(f"[DEBUG] New Modlist Context (automated workflow): {context}")
return self.run_modlist_configuration_phase(context)
else:
print(f"{COLOR_ERROR}Automated workflow completed but no AppID was returned.{COLOR_RESET}")
return True
else:
print(f"{COLOR_ERROR}Automated workflow completed but no AppID was returned.{COLOR_RESET}")
return True
# Unexpected result format
print(f"{COLOR_ERROR}Automated workflow returned unexpected format.{COLOR_RESET}")
self.logger.error(f"Unexpected result format from automated workflow: {result}")
@@ -566,8 +575,18 @@ class ModlistMenuHandler:
# Run modlist-specific post-install automation (e.g., VNV) before showing completion
# Only in CLI mode - GUI handles this in install_modlist.py
if not gui_mode:
from jackify.backend.services.vnv_integration_helper import run_vnv_automation_if_applicable
from jackify.backend.services.vnv_integration_helper import (
run_vnv_automation_if_applicable,
should_offer_vnv_automation,
)
from jackify.backend.services.automated_prefix_service import AutomatedPrefixService
from jackify.backend.services.vnv_post_install_service import VNVPostInstallService
from jackify.backend.handlers.path_handler import PathHandler
from jackify.frontends.cli.commands.vnv_manual_downloads import (
build_vnv_cli_manual_file_callback,
create_vnv_cli_progress_callback,
ensure_vnv_cli_manual_downloads,
)
from pathlib import Path
modlist_name = context.get('name', '')
@@ -581,33 +600,46 @@ class ModlistMenuHandler:
except (EOFError, KeyboardInterrupt):
return False
return user_input in ("", "y", "yes")
def _manual_vnv_file(title: str, instructions: str):
print(f"\n{COLOR_WARNING}{title}{COLOR_RESET}")
print(instructions)
try:
file_input = input(f"{COLOR_PROMPT}Path to downloaded file: {COLOR_RESET}").strip()
except (EOFError, KeyboardInterrupt):
return None
if not file_input:
return None
selected = Path(file_input).expanduser().resolve()
return selected if selected.exists() else None
automation_ran, error = run_vnv_automation_if_applicable(
modlist_name=modlist_name,
modlist_install_location=modlist_path,
game_root=None, # Will be auto-detected
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
progress_callback=lambda msg: print(msg),
manual_file_callback=_manual_vnv_file,
confirmation_callback=_confirm_vnv
)
if automation_ran and not error:
print(f"{COLOR_INFO}VNV post-install automation completed.{COLOR_RESET}")
if error:
print(f"{COLOR_WARNING}VNV automation encountered an error: {error}{COLOR_RESET}")
print(f"{COLOR_INFO}You can complete these steps manually by following: https://vivanewvegas.moddinglinked.com/wabbajack.html{COLOR_RESET}")
if should_offer_vnv_automation(modlist_name, modlist_path):
game_paths = PathHandler().find_vanilla_game_paths()
resolved_game_root = game_paths.get('Fallout New Vegas')
vnv_service = VNVPostInstallService(
modlist_install_location=modlist_path,
game_root=resolved_game_root or modlist_path,
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
)
completed = vnv_service.check_already_completed()
all_vnv_steps_done = (
completed['root_mods']
and completed['4gb_patch']
and completed['bsa_decompressed']
)
if all_vnv_steps_done:
print(f"{COLOR_INFO}VNV post-install steps are already complete.{COLOR_RESET}")
elif _confirm_vnv(vnv_service.get_automation_description()):
if not ensure_vnv_cli_manual_downloads(vnv_service, output_callback=print):
print(f"{COLOR_WARNING}VNV manual downloads were not completed. Skipping VNV automation.{COLOR_RESET}")
else:
progress_callback, close_progress = create_vnv_cli_progress_callback(print)
try:
automation_ran, error = run_vnv_automation_if_applicable(
modlist_name=modlist_name,
modlist_install_location=modlist_path,
game_root=None, # Will be auto-detected
ttw_installer_path=AutomatedPrefixService.get_ttw_installer_path(),
progress_callback=progress_callback,
manual_file_callback=build_vnv_cli_manual_file_callback(vnv_service, output_callback=print),
confirmation_callback=lambda _description: True,
)
finally:
close_progress()
if automation_ran and not error:
print(f"{COLOR_INFO}VNV post-install automation completed.{COLOR_RESET}")
if error:
print(f"{COLOR_WARNING}VNV automation encountered an error: {error}{COLOR_RESET}")
print(f"{COLOR_INFO}You can complete these steps manually by following: https://vivanewvegas.moddinglinked.com/wabbajack.html{COLOR_RESET}")
else:
print(f"{COLOR_INFO}VNV automation skipped by user.{COLOR_RESET}")
except Exception as e:
self.logger.debug(f"VNV automation check skipped: {e}")
# Not an error - just means VNV automation wasn't applicable

View File

@@ -401,6 +401,18 @@ class ModlistConfigurationMixin:
else:
self.logger.warning("Could not set download_directory in ModOrganizer.ini")
# Step 8.5: Align /home vs /var/home basis for Z: paths to match modlist install directory.
# This is intentionally separate from broad binary-path rewriting so it still runs when
# engine-installed workflows skip edit_binary_working_paths.
if not self.path_handler.align_home_path_basis(
modlist_ini_path=modlist_ini_path_obj,
modlist_dir_path=modlist_dir_path_obj,
modlist_sdcard=self.modlist_sdcard,
):
self.logger.error("Failed to align home-path basis in ModOrganizer.ini. Configuration aborted.")
self.logger.error("Failed to align /home path basis in ModOrganizer.ini.")
return False
self.logger.info("Step 8: Updating ModOrganizer.ini paths... Done")
# Step 9: Update Resolution Settings (if applicable)
@@ -539,6 +551,9 @@ class ModlistConfigurationMixin:
else:
self.logger.debug("Step 13: No special launch options needed for this modlist type")
if status_callback:
status_callback(f"{self._get_progress_timestamp()} Finalizing post-install configuration")
# Do not call status_callback here, the final message is handled in menu_handler
# if status_callback:
# status_callback("Configuration completed successfully!")
@@ -546,6 +561,8 @@ class ModlistConfigurationMixin:
self.logger.info("Configuration steps completed successfully.")
# Step 14: Re-enforce Windows 10 mode after modlist-specific configurations (matches legacy script line 1333)
if status_callback:
status_callback(f"{self._get_progress_timestamp()} Re-applying final Windows compatibility settings")
self._re_enforce_windows_10_mode()
return True # Return True on success
@@ -581,4 +598,3 @@ class ModlistConfigurationMixin:
else:
self.selected_resolution = None
self.logger.info("Resolution setup skipped by user.")

View File

@@ -7,6 +7,7 @@ import shutil
from pathlib import Path
from .ui_colors import COLOR_PROMPT, COLOR_INFO, COLOR_ERROR, COLOR_RESET, COLOR_WARNING
from jackify.shared.paths import get_jackify_logs_dir
logger = logging.getLogger(__name__)
@@ -352,10 +353,16 @@ class ModlistInstallCLITTWMixin:
print(f"\nTTW has been installed to: {ttw_output_path}")
print(f"TTW has been integrated into '{modlist_name}' (modlist.txt + plugins.txt updated).")
print(f"The modlist '{modlist_name}' is now ready to use with TTW.")
print(f"Detailed log available at: {get_jackify_logs_dir() / 'TTW_Install_workflow.log'}")
input(f"\n{COLOR_PROMPT}Press Enter to continue...{COLOR_RESET}")
else:
print(f"\n{COLOR_ERROR}TTW installation failed. Check the logs for details.{COLOR_RESET}")
print(f"{COLOR_ERROR}Error: {message}{COLOR_RESET}")
print(f"{COLOR_INFO}Detailed log available at: {get_jackify_logs_dir() / 'TTW_Install_workflow.log'}{COLOR_RESET}")
input(f"\n{COLOR_PROMPT}Press Enter to continue...{COLOR_RESET}")
except Exception as e:
self.logger.error(f"Error during TTW installation: {e}", exc_info=True)
print(f"{COLOR_ERROR}Error during TTW installation: {e}{COLOR_RESET}")
print(f"{COLOR_INFO}Detailed log available at: {get_jackify_logs_dir() / 'TTW_Install_workflow.log'}{COLOR_RESET}")
input(f"\n{COLOR_PROMPT}Press Enter to continue...{COLOR_RESET}")

View File

@@ -28,6 +28,95 @@ SDCARD_PREFIX = '/run/media/mmcblk0p1/'
class PathHandlerMO2Mixin:
"""Mixin providing ModOrganizer.ini path updates and formatting."""
@staticmethod
def _desired_home_basis_from_modlist_dir(modlist_dir_path: Path) -> Optional[str]:
"""
Determine desired Linux home-path basis from modlist install directory.
Returns:
"/var/home" when modlist dir is under /var/home,
"/home" when modlist dir is under /home,
None otherwise.
"""
try:
posix = modlist_dir_path.as_posix()
except Exception:
posix = str(modlist_dir_path).replace("\\", "/")
if posix.startswith("/var/home/"):
return "/var/home"
if posix.startswith("/home/"):
return "/home"
return None
@staticmethod
def _rewrite_z_home_basis_in_line(line: str, desired_home_basis: str) -> str:
"""
Rewrite only Z:-drive /home -> /var/home path basis in a single INI line.
Preserves slash style (forward or backslash), and leaves D: paths untouched.
"""
if desired_home_basis == "/var/home":
# Z:/home/... -> Z:/var/home/...
# Z:\\home\\... -> Z:\\var\\home\\...
return re.sub(r'([Zz]:[/\\]+)home([/\\]+)', r'\1var\2home\2', line)
return line
def align_home_path_basis(self, modlist_ini_path: Path, modlist_dir_path: Path, modlist_sdcard: bool) -> bool:
"""
Align gamePath/binary/workingDirectory home-path basis to modlist_dir_path.
This is a targeted post-processing step for Z: paths only:
- If install path is /var/home/... then rewrite Z:/home/... to Z:/var/home/...
- Otherwise do nothing.
"""
if modlist_sdcard:
return True
desired_home_basis = self._desired_home_basis_from_modlist_dir(modlist_dir_path)
# This alignment pass is intentionally one-way:
# only promote Z:/home -> Z:/var/home when install dir uses /var/home.
if desired_home_basis != "/var/home":
return True
if not modlist_ini_path.is_file():
logger.error(f"INI file {modlist_ini_path} does not exist for home-basis alignment")
return False
try:
with open(modlist_ini_path, 'r', encoding='utf-8') as f:
lines = f.readlines()
changed = 0
for i, line in enumerate(lines):
stripped = line.strip()
if not (
re.match(r'^\s*gamepath\s*=.*$', stripped, re.IGNORECASE)
or re.match(r'^(\d+)(\\+)\s*binary\s*=.*$', stripped, re.IGNORECASE)
or re.match(r'^(\d+)(\\+)\s*workingDirectory\s*=.*$', stripped, re.IGNORECASE)
):
continue
rewritten = self._rewrite_z_home_basis_in_line(line, desired_home_basis)
if rewritten != line:
lines[i] = rewritten
changed += 1
if changed > 0:
with open(modlist_ini_path, 'w', encoding='utf-8') as f:
f.writelines(lines)
logger.info(
"Aligned ModOrganizer.ini home-path basis to %s for %d line(s): %s",
desired_home_basis,
changed,
modlist_ini_path,
)
else:
logger.debug(
"No home-path basis alignment needed for %s (target %s)",
modlist_ini_path,
desired_home_basis,
)
return True
except Exception as e:
logger.error(f"Error aligning home path basis in {modlist_ini_path}: {e}")
return False
@staticmethod
def _strip_sdcard_path_prefix(path_obj: Path) -> str:
"""Removes SD card mount prefix. Returns path as POSIX-style string."""

View File

@@ -12,6 +12,10 @@ from pathlib import Path
from typing import Optional, List
from datetime import datetime
import vdf
from jackify.shared.steam_utils import (
get_ordered_steam_roots,
STEAM_PREFERENCE_AUTO,
)
logger = logging.getLogger(__name__)
@@ -23,11 +27,7 @@ class PathHandlerSteamMixin:
def find_steam_config_vdf() -> Optional[Path]:
"""Finds the active Steam config.vdf file."""
logger.debug("Searching for Steam config.vdf...")
possible_steam_paths = [
Path.home() / ".steam/steam",
Path.home() / ".local/share/Steam",
Path.home() / ".steam/root"
]
possible_steam_paths = get_ordered_steam_roots(STEAM_PREFERENCE_AUTO)
for steam_path in possible_steam_paths:
potential_path = steam_path / "config/config.vdf"
if potential_path.is_file():
@@ -40,10 +40,9 @@ class PathHandlerSteamMixin:
def find_steam_library() -> Optional[Path]:
"""Find the primary Steam library common directory containing games."""
logger.debug("Attempting to find Steam library...")
ordered_roots = get_ordered_steam_roots(STEAM_PREFERENCE_AUTO)
libraryfolders_vdf_paths = [
os.path.expanduser("~/.steam/steam/config/libraryfolders.vdf"),
os.path.expanduser("~/.local/share/Steam/config/libraryfolders.vdf"),
os.path.expanduser("~/.var/app/com.valvesoftware.Steam/.local/share/Steam/config/libraryfolders.vdf"),
str(root / "config" / "libraryfolders.vdf") for root in ordered_roots
]
for path in libraryfolders_vdf_paths:
if os.path.exists(path):
@@ -92,14 +91,11 @@ class PathHandlerSteamMixin:
logger.info(f"Using Steam library common path: {library_paths[0]}")
return library_paths[0]
logger.debug("No valid common paths found in VDF, checking default location...")
default_common_path = Path.home() / ".steam/steam/steamapps/common"
if default_common_path.is_dir():
logger.info(f"Using default Steam library common path: {default_common_path}")
return default_common_path
default_common_path_local = Path.home() / ".local/share/Steam/steamapps/common"
if default_common_path_local.is_dir():
logger.info(f"Using default local Steam library common path: {default_common_path_local}")
return default_common_path_local
for root in ordered_roots:
default_common_path = root / "steamapps" / "common"
if default_common_path.is_dir():
logger.info(f"Using default Steam library common path: {default_common_path}")
return default_common_path
logger.error("No valid Steam library common path found in VDF or default locations.")
return None
except Exception as e:
@@ -181,12 +177,8 @@ class PathHandlerSteamMixin:
def get_all_steam_library_paths() -> List[Path]:
"""Finds all Steam library paths listed in all known libraryfolders.vdf files (including Flatpak)."""
logger.info("[DEBUG] Searching for all Steam libraryfolders.vdf files...")
vdf_paths = [
Path.home() / ".steam/steam/config/libraryfolders.vdf",
Path.home() / ".local/share/Steam/config/libraryfolders.vdf",
Path.home() / ".steam/root/config/libraryfolders.vdf",
Path.home() / ".var/app/com.valvesoftware.Steam/.local/share/Steam/config/libraryfolders.vdf",
]
ordered_roots = get_ordered_steam_roots(STEAM_PREFERENCE_AUTO)
vdf_paths = [root / "config" / "libraryfolders.vdf" for root in ordered_roots]
library_paths = set()
for vdf_path in vdf_paths:
if vdf_path.is_file():

View File

@@ -6,6 +6,7 @@ import resource
import sys
import shutil
import logging
import threading
def get_safe_python_executable():
"""
@@ -154,7 +155,7 @@ class ProcessManager:
"""
Shared process manager for robust subprocess launching, tracking, and cancellation.
"""
def __init__(self, cmd, env=None, cwd=None, text=False, bufsize=0, separate_stderr=False):
def __init__(self, cmd, env=None, cwd=None, text=False, bufsize=0, separate_stderr=False, enable_stdin=False):
self.cmd = cmd
# Default to cleaned environment if None to prevent AppImage variable inheritance
if env is None:
@@ -165,14 +166,18 @@ class ProcessManager:
self.text = text
self.bufsize = bufsize
self.separate_stderr = separate_stderr
self.enable_stdin = enable_stdin
self.proc = None
self.process_group_pid = None
self._stdin_lock = threading.Lock()
self._start_process()
def _start_process(self):
stderr_arg = subprocess.PIPE if self.separate_stderr else subprocess.STDOUT
stdin_arg = subprocess.PIPE if self.enable_stdin else None
self.proc = subprocess.Popen(
self.cmd,
stdin=stdin_arg,
stdout=subprocess.PIPE,
stderr=stderr_arg,
env=self.env,
@@ -190,31 +195,45 @@ class ProcessManager:
cleanup_attempts = 0
try:
if self.proc:
# Terminate process group first so child tools don't survive parent exit.
if self.process_group_pid:
try:
os.killpg(self.process_group_pid, signal.SIGTERM)
except Exception:
pass
try:
self.proc.terminate()
try:
self.proc.wait(timeout=timeout_terminate)
return
except subprocess.TimeoutExpired:
pass
except Exception:
pass
try:
self.proc.kill()
try:
self.proc.wait(timeout=timeout_kill)
return
except subprocess.TimeoutExpired:
pass
self.proc.wait(timeout=timeout_terminate)
except subprocess.TimeoutExpired:
pass
except Exception:
pass
# Kill entire process group (catches 7zz and other child processes)
# Escalate to SIGKILL for stubborn children/process group.
if self.process_group_pid:
try:
os.killpg(self.process_group_pid, signal.SIGKILL)
except Exception:
pass
# Last resort: pkill by command name
try:
self.proc.kill()
except Exception:
pass
try:
self.proc.wait(timeout=timeout_kill)
except subprocess.TimeoutExpired:
pass
except Exception:
pass
# Last resort: pkill by command name (kept bounded).
while cleanup_attempts < max_cleanup_attempts:
try:
subprocess.run(['pkill', '-f', os.path.basename(self.cmd[0])], timeout=5, capture_output=True)
@@ -224,7 +243,7 @@ class ProcessManager:
finally:
# Always close pipes — unblocks threads blocked on read(1) or iterating stderr
if self.proc:
for pipe in (self.proc.stdout, self.proc.stderr):
for pipe in (self.proc.stdin, self.proc.stdout, self.proc.stderr):
if pipe:
try:
pipe.close()
@@ -250,4 +269,20 @@ class ProcessManager:
return self.proc.stdout.read(1)
except (ValueError, OSError):
return None
return None
return None
def write_stdin(self, line: str) -> bool:
"""
Write a line to the process stdin. Thread-safe.
Returns True on success, False if stdin is not available or process is gone.
"""
if not self.enable_stdin or not self.proc or not self.proc.stdin:
return False
with self._stdin_lock:
try:
payload = line if line.endswith('\n') else line + '\n'
self.proc.stdin.write(payload.encode())
self.proc.stdin.flush()
return True
except (OSError, BrokenPipeError):
return False

View File

@@ -64,17 +64,29 @@ class TTWInstallerBackendMixin:
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True, bufsize=1, universal_newlines=True
)
error_context: list = []
capturing_explanation = False
if process.stdout:
for line in process.stdout:
line = line.rstrip()
if line:
self.logger.info("TTW_Linux_Installer: %s", line)
lower = line.lower()
if 'failed' in lower or 'cannot continue' in lower or 'error:' in lower:
error_context.append(line.strip())
capturing_explanation = True
elif capturing_explanation and line.startswith(' '):
error_context.append(line.strip())
else:
capturing_explanation = False
process.wait()
ret = process.returncode
if ret == 0:
self.logger.info("TTW installation completed successfully.")
return True, "TTW installation completed successfully!"
self.logger.error("TTW installation process returned non-zero exit code: %s", ret)
if error_context:
return False, "TTW installation failed:\n" + "\n".join(error_context)
return False, f"TTW installation failed with exit code {ret}"
except Exception as e:
self.logger.error("Error executing TTW_Linux_Installer: %s", e, exc_info=True)
@@ -210,6 +222,8 @@ class TTWInstallerBackendMixin:
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True, bufsize=1, universal_newlines=True
)
error_context: list = []
capturing_explanation = False
if process.stdout:
for line in process.stdout:
line = line.rstrip()
@@ -217,12 +231,22 @@ class TTWInstallerBackendMixin:
self.logger.info("TTW_Linux_Installer: %s", line)
if output_callback:
output_callback(line)
lower = line.lower()
if 'failed' in lower or 'cannot continue' in lower or 'error:' in lower:
error_context.append(line.strip())
capturing_explanation = True
elif capturing_explanation and line.startswith(' '):
error_context.append(line.strip())
else:
capturing_explanation = False
process.wait()
ret = process.returncode
if ret == 0:
self.logger.info("TTW installation completed successfully.")
return True, "TTW installation completed successfully!"
self.logger.error("TTW installation process returned non-zero exit code: %s", ret)
if error_context:
return False, "TTW installation failed:\n" + "\n".join(error_context)
return False, f"TTW installation failed with exit code {ret}"
except Exception as e:
self.logger.error("Error executing TTW_Linux_Installer: %s", e, exc_info=True)

View File

@@ -269,10 +269,8 @@ exit"""
def get_ttw_installer_path() -> Optional[Path]:
"""Get path to TTW_Linux_Installer if available"""
try:
from jackify.shared.paths import get_jackify_data_dir
ttw_path = get_jackify_data_dir() / "TTW_Linux_Installer" / "ttw_linux_gui"
if ttw_path.exists():
return ttw_path
from .ttw_installer_service import get_ttw_installer_path
return get_ttw_installer_path()
except Exception:
pass
return None
@@ -405,4 +403,3 @@ exit"""
return prefix_dir
else:
return None

View File

@@ -47,11 +47,19 @@ class WorkflowMixin:
startdir_matches = shortcut_startdir == modlist_install_dir
if (name_matches and (exe_matches or startdir_matches)):
raw_appid = shortcut.get('appid')
normalized_appid = None
if raw_appid is not None:
try:
normalized_appid = str(int(raw_appid) & 0xFFFFFFFF)
except Exception:
normalized_appid = str(raw_appid)
conflicts.append({
'index': i,
'name': name,
'exe': shortcut_exe,
'startdir': shortcut_startdir
'startdir': shortcut_startdir,
'appid': normalized_appid,
})
if conflicts:
@@ -124,42 +132,59 @@ class WorkflowMixin:
Tuple of (success, prefix_path, appid, last_timestamp)
"""
logger.info("Starting proven working automated prefix creation workflow")
# Show installation complete and configuration start headers FIRST
if progress_callback:
progress_callback("")
progress_callback("=" * 64)
progress_callback("= Installation phase complete =")
progress_callback("=" * 64)
progress_callback("")
progress_callback("=" * 64)
progress_callback("= Starting Configuration Phase =")
progress_callback("=" * 64)
progress_callback("")
# Reset timing for Steam Integration section (part of Configuration Phase)
from jackify.shared.timing import start_new_phase
start_new_phase()
# Show immediate feedback to user with section header
if progress_callback:
progress_callback("") # Blank line before Steam Integration
progress_callback("=== Steam Integration ===")
progress_callback(f"{self._get_progress_timestamp()} Creating Steam shortcut with native service")
# Registry injection approach for both FNV and Enderal
from ..handlers.modlist_handler import ModlistHandler
modlist_handler = ModlistHandler()
special_game_type = modlist_handler.detect_special_game_type(modlist_install_dir)
# No launch options needed - FNV, FO3 and Enderal use registry injection
custom_launch_options = None
if special_game_type in ["fnv", "fo3", "enderal"]:
logger.info(f"Using registry injection approach for {special_game_type.upper()} modlist")
else:
logger.debug("Standard modlist - no special game handling needed")
try:
conflict_result = self.handle_existing_shortcut_conflict(
shortcut_name,
final_exe_path,
modlist_install_dir,
)
if isinstance(conflict_result, list):
logger.warning(
"Found %d existing shortcut(s) with same name and path before Steam integration",
len(conflict_result),
)
return ("CONFLICT", conflict_result, None, None)
if conflict_result is False:
logger.error("User cancelled due to shortcut conflict")
return False, None, None, None
# Show installation complete and configuration start headers only after
# conflict checks pass, so users do not see Steam integration start
# messages when Jackify is about to stop for duplicate-shortcut review.
if progress_callback:
progress_callback("")
progress_callback("=" * 64)
progress_callback("= Installation phase complete =")
progress_callback("=" * 64)
progress_callback("")
progress_callback("=" * 64)
progress_callback("= Starting Configuration Phase =")
progress_callback("=" * 64)
progress_callback("")
# Reset timing for Steam Integration section (part of Configuration Phase)
from jackify.shared.timing import start_new_phase
start_new_phase()
# Show immediate feedback to user with section header
if progress_callback:
progress_callback("") # Blank line before Steam Integration
progress_callback("=== Steam Integration ===")
progress_callback(f"{self._get_progress_timestamp()} Creating Steam shortcut with native service")
# Registry injection approach for both FNV and Enderal
from ..handlers.modlist_handler import ModlistHandler
modlist_handler = ModlistHandler()
special_game_type = modlist_handler.detect_special_game_type(modlist_install_dir)
# No launch options needed - FNV, FO3 and Enderal use registry injection
custom_launch_options = None
if special_game_type in ["fnv", "fo3", "enderal"]:
logger.info(f"Using registry injection approach for {special_game_type.upper()} modlist")
else:
logger.debug("Standard modlist - no special game handling needed")
# Step 0: Shut down Steam before modifying VDF files
# Required to safely modify shortcuts.vdf and config.vdf without race conditions
logger.info("Step 0: Shutting down Steam before modifying VDF files")
@@ -179,22 +204,6 @@ class WorkflowMixin:
# Step 1: Create shortcut with native Steam service (Steam is now shut down)
logger.info("Step 1: Creating shortcut with native Steam service")
# DISABLED: Shortcut conflict detection temporarily disabled pending rework
# Re-enable after conflict resolution workflow refactor
# When re-enabled, this will detect and handle cases where shortcuts with the same
# name and path already exist in Steam, allowing users to resolve conflicts
# Disabled pending workflow improvements - planned for future release
# conflict_result = self.handle_existing_shortcut_conflict(shortcut_name, final_exe_path, modlist_install_dir)
# if isinstance(conflict_result, list): # Conflicts found
# logger.warning(f"Found {len(conflict_result)} existing shortcut(s) with same name and path")
# # Return a special tuple to indicate conflict that needs user resolution
# return ("CONFLICT", conflict_result, None)
# elif not conflict_result: # User cancelled or other failure
# logger.error("User cancelled due to shortcut conflict")
# return False, None, None, None
logger.info("Conflict detection temporarily disabled - proceeding with shortcut creation")
# Create shortcut using native Steam service with special game launch options
success, appid = self.create_shortcut_with_native_service(
shortcut_name, final_exe_path, modlist_install_dir, custom_launch_options, download_dir=download_dir
@@ -387,4 +396,3 @@ class WorkflowMixin:
if progress_callback:
progress_callback(f"Error: {str(e)}")
return False, None, None, None

View File

@@ -0,0 +1,138 @@
"""
Watches a directory for newly downloaded files and matches them against a
list of pending manual download items by lax filename comparison.
"""
import os
import time
import logging
from dataclasses import dataclass, field
from pathlib import Path
from threading import Thread, Event
from typing import Callable, Optional
logger = logging.getLogger(__name__)
@dataclass
class WatcherConfig:
watch_directory: Path
watch_recursive: bool = False
debounce_seconds: float = 2.0
additional_dirs: list = field(default_factory=list)
class DownloadWatcherService:
"""
Monitors a directory for files that match pending download items.
Caller sets pending_items (list of dicts with at least 'file_name') and
registers an on_candidate callback that receives (Path, dict) when a
potential match is detected (after debounce, before hash validation).
"""
def __init__(self, config: WatcherConfig, on_candidate: Callable[[Path, dict], None]):
self._config = config
self._on_candidate = on_candidate
self._pending_items: list[dict] = []
self._pending_exact: list[tuple[str, dict]] = []
self._stop_event = Event()
self._thread: Optional[Thread] = None
# Track known files so we only react to new/changed ones
self._known: dict[Path, float] = {}
def set_pending_items(self, items: list[dict]) -> None:
"""Replace the pending items list. Thread-safe for simple list swap."""
self._pending_items = list(items)
self._pending_exact = [
(str(item.get('file_name', '')).lower(), item)
for item in self._pending_items
if item.get('file_name')
]
def start(self) -> None:
if self._thread and self._thread.is_alive():
return
self._stop_event.clear()
self._thread = Thread(target=self._watch_loop, daemon=True, name='DownloadWatcher')
self._thread.start()
logger.debug(f"Download watcher started on: {self._config.watch_directory}")
def stop(self) -> None:
self._stop_event.set()
if self._thread:
self._thread.join(timeout=5)
logger.debug("Download watcher stopped")
def _all_watch_dirs(self) -> list[Path]:
dirs = [self._config.watch_directory]
dirs.extend(self._config.additional_dirs)
return [d for d in dirs if d.is_dir()]
def _scan(self) -> None:
for watch_dir in self._all_watch_dirs():
try:
entries = list(watch_dir.iterdir()) if not self._config.watch_recursive else \
[p for p in watch_dir.rglob('*') if p.is_file()]
for path in entries:
if not path.is_file():
continue
# Skip browser temp files
if path.suffix in ('.part', '.crdownload', '.tmp'):
continue
try:
mtime = path.stat().st_mtime
except OSError:
continue
prev_mtime = self._known.get(path)
if prev_mtime == mtime:
continue
self._known[path] = mtime
self._check_candidate(path)
except OSError as e:
logger.debug(f"Watcher scan error on {watch_dir}: {e}")
def _check_candidate(self, path: Path) -> None:
candidate_name = path.name.lower()
# Exact filename match (case-insensitive).
for expected_name, item in self._pending_exact:
if expected_name == candidate_name:
logger.debug(f"Candidate exact match: {path.name}")
self._debounce_and_emit(path, item)
return
# Some modlist metadata stores filenames with a leading dot that browsers
# strip when saving the download. Match against the stripped expected name.
for expected_name, item in self._pending_exact:
if expected_name.lstrip('.') == candidate_name:
logger.debug(f"Candidate dot-normalized match: {path.name} -> {expected_name}")
self._debounce_and_emit(path, item)
return
def _debounce_and_emit(self, path: Path, item: dict) -> None:
def _wait_and_emit():
prev_size = -1
stable_count = 0
needed = max(1, int(self._config.debounce_seconds / 0.5))
for _ in range(needed * 4): # max ~2× debounce time
if self._stop_event.is_set():
return
time.sleep(0.5)
try:
size = path.stat().st_size
except OSError:
return
if size == prev_size:
stable_count += 1
if stable_count >= needed:
break
else:
stable_count = 0
prev_size = size
if path.exists():
self._on_candidate(path, item)
Thread(target=_wait_and_emit, daemon=True, name=f'Debounce-{path.name[:20]}').start()
def _watch_loop(self) -> None:
while not self._stop_event.is_set():
self._scan()
self._stop_event.wait(timeout=1.0)

View File

@@ -0,0 +1,261 @@
"""
Hash validation and file move for manually downloaded archives.
Uses xxhash64 to match the engine's hash format exactly.
"""
import struct
import shutil
import logging
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional
logger = logging.getLogger(__name__)
# xxhash produces 16-char lowercase hex with no prefix - matches engine Hash.ToHex()
# C extension is ABI-locked to the Python version it was compiled against, so
# AppImage builds need a pure-Python fallback for cross-version compatibility.
try:
import xxhash
_XXHASH_IMPL = 'native'
except ImportError:
xxhash = None
_XXHASH_IMPL = 'fallback'
logger.info("xxhash C extension not available, using pure-Python fallback")
class _XXH64Fallback:
"""Pure-Python xxhash64 implementation for when the C extension can't load.
Reference: https://github.com/Cyan4973/xxHash/blob/dev/doc/xxhash_spec.md"""
_P1 = 11400714785074694791
_P2 = 14029467366897019727
_P3 = 1609587929392839161
_P4 = 9650029242287828579
_P5 = 2870177450012600261
_M64 = 0xFFFFFFFFFFFFFFFF
def __init__(self, seed: int = 0):
self._seed = seed & self._M64
self._total_len = 0
self._buf = b""
self._v1 = (seed + self._P1 + self._P2) & self._M64
self._v2 = (seed + self._P2) & self._M64
self._v3 = seed & self._M64
self._v4 = (seed - self._P1) & self._M64
@staticmethod
def _rotl64(x: int, r: int) -> int:
return ((x << r) | (x >> (64 - r))) & 0xFFFFFFFFFFFFFFFF
def _round(self, acc: int, inp: int) -> int:
acc = (acc + inp * self._P2) & self._M64
acc = self._rotl64(acc, 31)
acc = (acc * self._P1) & self._M64
return acc
def _merge_round(self, acc: int, val: int) -> int:
val = self._round(0, val)
acc ^= val
acc = (acc * self._P1 + self._P4) & self._M64
return acc
def update(self, data: bytes) -> None:
self._buf += data
self._total_len += len(data)
if len(self._buf) < 32:
return
p = 0
end = len(self._buf) - 31 # process 32-byte blocks
while p < end:
self._v1 = self._round(self._v1, struct.unpack_from('<Q', self._buf, p)[0])
self._v2 = self._round(self._v2, struct.unpack_from('<Q', self._buf, p + 8)[0])
self._v3 = self._round(self._v3, struct.unpack_from('<Q', self._buf, p + 16)[0])
self._v4 = self._round(self._v4, struct.unpack_from('<Q', self._buf, p + 24)[0])
p += 32
self._buf = self._buf[p:]
def hexdigest(self) -> str:
return format(self._digest(), '016x')
def _digest(self) -> int:
M = self._M64
if self._total_len >= 32:
h = self._rotl64(self._v1, 1)
h = (h + self._rotl64(self._v2, 7)) & M
h = (h + self._rotl64(self._v3, 12)) & M
h = (h + self._rotl64(self._v4, 18)) & M
h = self._merge_round(h, self._v1)
h = self._merge_round(h, self._v2)
h = self._merge_round(h, self._v3)
h = self._merge_round(h, self._v4)
else:
h = (self._seed + self._P5) & M
h = (h + self._total_len) & M
buf = self._buf
p = 0
remaining = len(buf)
while remaining >= 8:
k1 = struct.unpack_from('<Q', buf, p)[0]
k1 = self._round(0, k1)
h ^= k1
h = (self._rotl64(h, 27) * self._P1 + self._P4) & M
p += 8
remaining -= 8
while remaining >= 4:
k1 = struct.unpack_from('<I', buf, p)[0]
h ^= (k1 * self._P1) & M
h = (self._rotl64(h, 23) * self._P2 + self._P3) & M
p += 4
remaining -= 4
while remaining > 0:
h ^= (buf[p] * self._P5) & M
h = (self._rotl64(h, 11) * self._P1) & M
p += 1
remaining -= 1
# Avalanche
h ^= h >> 33
h = (h * self._P2) & M
h ^= h >> 29
h = (h * self._P3) & M
h ^= h >> 32
return h
_CHUNK = 1024 * 1024 # 1 MB
def _reverse_hex_byte_order(hex_value: str) -> str:
"""Reverse byte order of a hex string (e.g. aabbccdd -> ddccbbaa)."""
value = (hex_value or "").strip().lower()
if len(value) % 2 != 0:
return value
return "".join(reversed([value[i:i + 2] for i in range(0, len(value), 2)]))
def _hash_matches_expected(computed_hash: str, expected_hash: str) -> bool:
"""Accept either canonical or byte-reversed xxhash64 representations."""
computed = (computed_hash or "").strip().lower()
expected = (expected_hash or "").strip().lower()
if not computed or not expected:
return False
return computed == expected or _reverse_hex_byte_order(computed) == expected
@dataclass
class ValidationResult:
matches: bool
computed_hash: Optional[str]
file_path: Path
error: Optional[str] = None
class FileValidatorService:
"""
Validates downloaded files against expected xxhash64 and moves them to
the modlist downloads directory on success.
"""
def __init__(self, max_workers: int = 2):
self._executor = ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix='FileValidator')
def validate_async(
self,
file_path: Path,
expected_hash: str,
modlist_download_dir: Path,
on_result: Callable[[ValidationResult, Optional[Path]], None],
dest_name: Optional[str] = None,
) -> None:
"""
Validate file_path against expected_hash in a thread pool worker.
on_result(result, dest_path) is called on the worker thread when done.
dest_path is the moved file location if validation succeeded, else None.
dest_name overrides the destination filename (used when the engine's
canonical name differs from the downloaded file's name, e.g. leading dot).
"""
self._executor.submit(
self._validate_and_move,
file_path, expected_hash, modlist_download_dir, on_result, dest_name
)
def _validate_and_move(
self,
file_path: Path,
expected_hash: str,
modlist_download_dir: Path,
on_result: Callable,
dest_name: Optional[str] = None,
) -> None:
result = self._validate(file_path, expected_hash)
dest: Optional[Path] = None
if result.matches:
try:
dest = self._move_file(file_path, modlist_download_dir, dest_name=dest_name)
logger.info(
"[MDL-1026] Archive move complete | "
f"source_path={file_path} destination_path={dest} hash={result.computed_hash or 'missing'}"
)
except OSError as e:
logger.warning(
"[MDL-9020] Archive move failed after hash validation | "
f"source_path={file_path} destination_dir={modlist_download_dir} reason={e}"
)
result = ValidationResult(
matches=False,
computed_hash=result.computed_hash,
file_path=file_path,
error=f"Move failed: {e}",
)
on_result(result, dest)
def _validate(self, file_path: Path, expected_hash: str) -> ValidationResult:
try:
# No expected hash — accept by filename match alone, just move the file.
if not (expected_hash or "").strip():
return ValidationResult(matches=True, computed_hash=None, file_path=file_path)
h = xxhash.xxh64() if xxhash else _XXH64Fallback()
with open(file_path, 'rb') as f:
while True:
chunk = f.read(_CHUNK)
if not chunk:
break
h.update(chunk)
computed = h.hexdigest().lower() # 16-char lowercase hex, no prefix
matches = _hash_matches_expected(computed, expected_hash)
return ValidationResult(
matches=matches,
computed_hash=computed,
file_path=file_path,
)
except OSError as e:
return ValidationResult(matches=False, computed_hash=None, file_path=file_path,
error=str(e))
def _move_file(self, source: Path, dest_dir: Path, dest_name: Optional[str] = None) -> Path:
dest_dir.mkdir(parents=True, exist_ok=True)
dest = dest_dir / (dest_name if dest_name else source.name)
# If the watched file is already in the modlist downloads directory,
# treat it as in-place and avoid a same-path move error.
try:
if source.resolve() == dest.resolve():
logger.debug(f"Validated file already in modlist downloads directory: {source}")
return dest
except OSError:
pass
shutil.move(str(source), str(dest))
logger.debug(f"Moved validated file: {source.name} -> {dest}")
return dest
def shutdown(self) -> None:
self._executor.shutdown(wait=False)

View File

@@ -0,0 +1,124 @@
"""
Orchestrates the manual download workflow:
- Maintains queue of pending items
- Opens browser tabs (sliding window, N concurrent)
- Coordinates directory watcher and file validator
- Sends continue command to engine when all items are done
"""
import logging
import threading
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Literal, Optional
from jackify.backend.services.download_watcher_service import DownloadWatcherService, WatcherConfig
from jackify.backend.services.file_validator_service import FileValidatorService
from jackify.backend.services.manual_download_manager_api_mixin import ManualDownloadManagerApiMixin
from jackify.backend.services.manual_download_manager_runtime_mixin import ManualDownloadManagerRuntimeMixin
logger = logging.getLogger(__name__)
STATUS = Literal["pending", "browser_opened", "validating", "complete", "deferred", "skipped", "error"]
_STATE_FILE = Path.home() / '.local' / 'share' / 'jackify' / 'manual_download_state.json'
@dataclass
class DownloadItem:
file_name: str
nexus_url: str
expected_hash: str
expected_size: int
mod_name: str
mod_id: int = 0
file_id: int = 0
index: int = 0
total: int = 0
loop_iteration: int = 1
status: STATUS = "pending"
local_path: Optional[str] = None
error_message: Optional[str] = None
needs_user_retry: bool = False
@classmethod
def from_event(cls, evt: dict, loop_iteration: int = 1) -> 'DownloadItem':
# Engine historically emitted `nexus_url`, but manual-only/external sources
# may arrive as generic URL fields depending on engine version.
source_url = (
evt.get('nexus_url')
or evt.get('download_url')
or evt.get('manual_url')
or evt.get('url')
or ''
)
item = cls(
file_name=evt.get('file_name', ''),
nexus_url=source_url,
expected_hash=evt.get('expected_hash', ''),
expected_size=evt.get('expected_size', 0),
mod_name=evt.get('mod_name', evt.get('file_name', '')),
mod_id=evt.get('mod_id', 0),
file_id=evt.get('file_id', 0),
index=evt.get('index', 0),
total=evt.get('total', 0),
loop_iteration=loop_iteration,
)
if not item.nexus_url:
# Engine contract says nexus_url should be present and non-empty.
# If missing, keep this item out of auto-open rotation and require
# explicit user attention/manual recovery.
item.needs_user_retry = True
item.error_message = "Malformed manual_download_required event: missing nexus_url"
return item
class ManualDownloadManager(ManualDownloadManagerApiMixin, ManualDownloadManagerRuntimeMixin):
"""
Manages the full manual download workflow for one engine session.
Usage:
manager = ManualDownloadManager(
modlist_download_dir=Path(...),
watch_directory=Path(...),
concurrent_limit=2,
on_item_updated=my_callback,
on_send_continue=installer_thread.send_continue,
)
manager.load_items(event_list, loop_iteration=1)
manager.start()
# ... user downloads files ...
# manager sends continue automatically when all done
manager.stop()
"""
def __init__(
self,
modlist_download_dir: Path,
watch_directory: Path,
concurrent_limit: int = 2,
on_item_updated: Optional[Callable[[DownloadItem], None]] = None,
on_send_continue: Optional[Callable[[], None]] = None,
on_all_done: Optional[Callable[[int, int], None]] = None,
):
self._dl_dir = modlist_download_dir
self._watch_dir = watch_directory
self._limit = max(1, min(5, concurrent_limit))
self._on_item_updated = on_item_updated
self._on_send_continue = on_send_continue
self._on_all_done = on_all_done
self._items: list[DownloadItem] = []
self._lock = threading.Lock()
self._active_tabs = 0
self._paused = False
self._started = False
self._startup_precheck_pending = 0
self._run_id = f"mdl-{int(time.time())}-{id(self) % 10000}"
self._last_progress_log_completed = -1
additional = [modlist_download_dir] if modlist_download_dir != watch_directory else []
config = WatcherConfig(watch_directory=watch_directory, additional_dirs=additional)
self._watcher = DownloadWatcherService(config, self._on_candidate)
self._validator = FileValidatorService(max_workers=2)

View File

@@ -0,0 +1,163 @@
from __future__ import annotations
"""Public API methods for ManualDownloadManager."""
import json
from typing import Optional
class ManualDownloadManagerApiMixin:
"""Mixin containing public manager API methods and status properties."""
def load_items(self, events: list[dict], loop_iteration: int = 1) -> None:
"""
Merge a new batch of engine events into the existing item list.
On loop_iteration > 1, engine only emits still-missing files. Items NOT
in the new batch that were pending are confirmed present by the engine
(they passed its rescan) and are marked complete. Genuinely new items
(edge case) are appended. active_tabs resets so the sliding window
opens fresh tabs for the remaining items.
"""
with self._lock:
existing_map = {item.file_name: item for item in self._items}
new_batch_names = {evt.get('file_name', '') for evt in events}
# Items the engine confirmed are now present (not in new batch, were pending)
for item in self._items:
if item.file_name not in new_batch_names and item.status not in ('complete', 'deferred', 'skipped', 'error'):
item.status = 'complete'
item.needs_user_retry = False
# Recheck loop: clear temporary defer state for still-missing files so they can
# re-enter active browser rotation in the new iteration.
if loop_iteration > 1:
for item in self._items:
if item.file_name in new_batch_names and item.status in ('deferred', 'skipped'):
item.status = 'pending'
item.needs_user_retry = False
item.error_message = None
# Add items genuinely not seen before (first iteration, or edge case)
for evt in events:
name = evt.get('file_name', '')
if name not in existing_map:
# Local import avoids module-load circular dependency with manager class.
from jackify.backend.services.manual_download_manager import DownloadItem
new_item = DownloadItem.from_event(evt, loop_iteration)
self._items.append(new_item)
if not new_item.nexus_url:
self._diag(
"MDL-9012",
"Engine manual-download event missing required nexus_url",
level="error",
file_name=new_item.file_name or "missing",
loop_iteration=loop_iteration,
mod_id=new_item.mod_id,
file_id=new_item.file_id,
)
self._active_tabs = 0
total = len(self._items)
pending = sum(1 for i in self._items if i.status == 'pending')
complete = sum(1 for i in self._items if i.status == 'complete')
skipped = sum(1 for i in self._items if i.status == 'skipped')
sample_pending = [i.file_name for i in self._items if i.status == 'pending'][:5]
self._refresh_watcher_pending_items()
self._diag(
"MDL-1001",
"Manual download batch loaded",
loop_iteration=loop_iteration,
batch_size=len(events),
total_items=total,
pending=pending,
complete=complete,
skipped=skipped,
pending_sample=json.dumps(sample_pending, ensure_ascii=True),
)
def start(self) -> None:
with self._lock:
if self._started:
return
self._started = True
self._diag(
"MDL-1002",
"Manual download watcher started",
watch_dir=str(self._watch_dir),
downloads_dir=str(self._dl_dir),
concurrent_limit=self._limit,
)
self._watcher.start()
matched = self._ingest_existing_files()
with self._lock:
self._startup_precheck_pending = matched
if matched:
self._diag("MDL-1003", "Pre-existing archives detected", matched=matched)
self._diag("MDL-1016", "Deferring tab opening until precheck validation completes", pending_precheck=matched)
else:
self._open_next_tabs()
def stop(self) -> None:
self._watcher.stop()
self._validator.shutdown()
with self._lock:
self._started = False
self._startup_precheck_pending = 0
self._diag("MDL-1009", "Manual download manager stopped")
def pause(self) -> None:
with self._lock:
self._paused = True
def resume(self) -> None:
with self._lock:
self._paused = False
self._diag("MDL-1008", "Manual download resumed")
# Explicit user start/resume must open tabs even if startup precheck
# bookkeeping is still in-flight.
self._open_next_tabs(force_user_start=True)
def skip_item(self, file_name: str) -> None:
item_to_notify: Optional[DownloadItem] = None
with self._lock:
for item in self._items:
if item.file_name == file_name and item.status not in ('complete',):
item.status = 'deferred'
if self._active_tabs > 0:
self._active_tabs -= 1
item_to_notify = item
break
if item_to_notify is not None:
self._notify(item_to_notify)
self._open_next_tabs()
self._check_all_done()
def set_concurrent_limit(self, limit: int) -> None:
with self._lock:
self._limit = max(1, min(5, limit))
applied = self._limit
started = self._started
self._diag("MDL-1006", "Manual download concurrency updated", concurrent_limit=applied)
if started:
self._open_next_tabs()
@property
def items(self) -> list[DownloadItem]:
with self._lock:
return list(self._items)
@property
def pending_count(self) -> int:
with self._lock:
return sum(1 for i in self._items if i.status == 'pending')
@property
def complete_count(self) -> int:
with self._lock:
return sum(1 for i in self._items if i.status == 'complete')
@property
def skipped_count(self) -> int:
with self._lock:
return sum(1 for i in self._items if i.status in ('deferred', 'skipped'))

View File

@@ -0,0 +1,479 @@
from __future__ import annotations
"""Internal runtime methods for ManualDownloadManager."""
import json
import logging
import subprocess
from pathlib import Path
from typing import Optional
from jackify.backend.services.file_validator_service import ValidationResult
logger = logging.getLogger(__name__)
class ManualDownloadManagerRuntimeMixin:
"""Mixin containing browser/watcher/validation runtime methods."""
def _open_next_tabs(self, force_user_start: bool = False) -> None:
to_open = []
to_notify = []
with self._lock:
if not self._started or self._paused:
return
if self._startup_precheck_pending > 0 and not force_user_start:
return
while self._active_tabs < self._limit:
item = self._next_pending(include_retry=force_user_start)
if item is None:
break
if force_user_start and item.needs_user_retry:
item.needs_user_retry = False
item.error_message = None
item.status = 'browser_opened'
self._active_tabs += 1
to_notify.append(item)
to_open.append(item)
active_tabs = self._active_tabs
pending_left = sum(1 for i in self._items if i.status == 'pending')
if to_open:
self._diag(
"MDL-1010",
"Opening next manual download tab window",
opening_count=len(to_open),
active_tabs=active_tabs,
pending_after_schedule=pending_left,
)
# Notify outside the lock to prevent GUI callbacks from re-entering manager state.
for item in to_notify:
self._notify(item)
# Open browser tabs outside the lock so Popen/fork doesn't stall lock holders
for item in to_open:
opened, error_message = self._open_browser(item)
if opened:
continue
item_to_notify: Optional[DownloadItem] = None
with self._lock:
# Revert failed launch so the row does not falsely remain "Browser Opened".
current = self._item_by_name(item.file_name)
if current and current.status == 'browser_opened':
current.status = 'pending'
if self._active_tabs > 0:
self._active_tabs -= 1
current.error_message = error_message
if error_message and "No URL available" in error_message:
current.needs_user_retry = True
item_to_notify = current
if item_to_notify is not None:
self._notify(item_to_notify)
self._diag(
"MDL-9001",
"Automatic browser launch failed for manual download item",
level="warning",
file_name=item.file_name,
reason=error_message or "unknown launcher failure",
)
def _next_pending(self, include_retry: bool = False) -> Optional[DownloadItem]:
for item in self._items:
if item.status != 'pending':
continue
if item.needs_user_retry and not include_retry:
continue
return item
return None
def _open_browser(self, item: DownloadItem) -> tuple[bool, Optional[str]]:
url = item.nexus_url
if not url:
msg = "No URL available for manual download item"
logger.warning(f"{msg}: {item.file_name}")
return False, msg
# Linux desktop launch fallbacks. xdg-open should cover most environments,
# but keep alternates for distributions where handlers differ.
launch_cmds = (
['xdg-open', url],
['gio', 'open', url],
['sensible-browser', url],
)
launch_errors: list[str] = []
for cmd in launch_cmds:
try:
proc = subprocess.Popen(
cmd,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
start_new_session=True,
)
except OSError as e:
launch_errors.append(f"{cmd[0]} not available: {e}")
continue
try:
rc = proc.wait(timeout=3)
except subprocess.TimeoutExpired:
# Launcher still running after handoff window; treat as success.
logger.debug(f"Opened browser for: {item.file_name} via {cmd[0]}")
return True, None
if rc == 0:
logger.debug(f"Opened browser for: {item.file_name} via {cmd[0]}")
return True, None
stderr_tail = ""
try:
stderr_tail = (proc.stderr.read() or b"").decode("utf-8", errors="replace").strip()
except Exception:
stderr_tail = ""
launch_errors.append(f"{cmd[0]} exited {rc}{(': ' + stderr_tail) if stderr_tail else ''}")
msg = f"Could not open browser automatically for {item.file_name}"
logger.error(f"{msg}. Launch attempts: {' | '.join(launch_errors)}")
return False, msg
def _on_candidate(self, path: Path, hint: dict, from_startup_precheck: bool = False) -> bool:
"""Called by watcher after debounce when a potential match is found."""
file_name = hint.get('file_name', '')
item_to_notify: Optional[DownloadItem] = None
reject_reason = ""
had_browser_slot = False
with self._lock:
item = self._item_by_name(file_name)
if item is None:
reject_reason = "unknown_item"
elif item.status in ('complete', 'skipped'):
reject_reason = f"terminal_status:{item.status}"
elif item.status == 'validating':
reject_reason = "already_validating"
if reject_reason:
self._diag(
"MDL-1022",
"Candidate ignored",
file_name=file_name or "missing",
source_path=str(path),
from_precheck=from_startup_precheck,
reason=reject_reason,
)
return False
had_browser_slot = item.status == 'browser_opened'
item.status = 'validating'
item_to_notify = item
if item_to_notify is not None:
self._notify(item_to_notify)
self._diag(
"MDL-1020",
"Candidate queued for validation",
file_name=file_name or "missing",
source_path=str(path),
from_precheck=from_startup_precheck,
)
# Pass the engine's canonical filename as dest_name so that if the browser
# stripped a leading dot, the file is renamed correctly on move.
canonical_name = hint.get('file_name') or None
dest_name = canonical_name if canonical_name and canonical_name != path.name else None
self._validator.validate_async(
file_path=path,
expected_hash=hint.get('expected_hash', ''),
modlist_download_dir=self._dl_dir,
on_result=lambda result, dest: self._on_validation_result(
file_name,
result,
dest,
from_startup_precheck=from_startup_precheck,
had_browser_slot=had_browser_slot,
),
dest_name=dest_name,
)
return True
def _on_validation_result(
self,
file_name: str,
result: ValidationResult,
dest: Optional[Path],
from_startup_precheck: bool = False,
had_browser_slot: bool = False,
) -> None:
item_to_notify: Optional[DownloadItem] = None
validation_failed = False
completed_now = False
precheck_ready = False
expected_hash = ""
mod_id = 0
file_id = 0
source_path = str(result.file_path) if getattr(result, "file_path", None) else ""
computed_hash = (result.computed_hash or "").lower() if result.computed_hash else ""
with self._lock:
item = self._item_by_name(file_name)
if item is None:
return
expected_hash = (item.expected_hash or "").lower()
mod_id = item.mod_id
file_id = item.file_id
if result.matches and dest:
item.status = 'complete'
item.local_path = str(dest)
item.needs_user_retry = False
if had_browser_slot and self._active_tabs > 0:
self._active_tabs -= 1
item_to_notify = item
completed_now = True
else:
# Hash mismatch or validation error — revert to pending so the
# sliding window can re-open a browser tab and the watcher can
# re-validate if the user downloads the correct file.
item.status = 'pending'
msg = result.error or f"Hash mismatch (got {result.computed_hash})"
item.error_message = msg
logger.warning(f"Validation failed for {file_name}: {msg}")
if had_browser_slot and self._active_tabs > 0:
self._active_tabs -= 1
item_to_notify = item
validation_failed = True
if from_startup_precheck and self._startup_precheck_pending > 0:
self._startup_precheck_pending -= 1
precheck_ready = self._startup_precheck_pending == 0
if item_to_notify is not None:
self._notify(item_to_notify)
if completed_now:
self._diag(
"MDL-1021",
"Archive validated and accepted",
file_name=file_name,
source_path=source_path or "missing",
destination_path=str(dest) if dest else "missing",
expected_hash=expected_hash or "missing",
computed_hash=computed_hash or "missing",
from_precheck=from_startup_precheck,
mod_id=mod_id,
file_id=file_id,
)
self._maybe_log_progress_summary()
if precheck_ready:
self._diag("MDL-1017", "Startup precheck validation complete; opening tabs")
self._open_next_tabs()
if validation_failed:
self._refresh_watcher_pending_items()
if not from_startup_precheck:
self._open_next_tabs()
self._diag(
"MDL-9002",
"Archive validation failed",
level="warning",
file_name=file_name,
expected_hash=expected_hash or "missing",
computed_hash=computed_hash or "missing",
source_path=source_path or "missing",
mod_id=mod_id,
file_id=file_id,
from_precheck=from_startup_precheck,
reason=result.error or "hash mismatch",
)
return
# Update watcher pending list (remove completed item, keep other in-flight items).
self._refresh_watcher_pending_items()
if not from_startup_precheck:
self._open_next_tabs()
self._check_all_done()
def _check_all_done(self) -> None:
with self._lock:
remaining = [i for i in self._items if i.status not in ('complete', 'deferred', 'skipped', 'error')]
if remaining:
return
completed = sum(1 for i in self._items if i.status == 'complete')
skipped = sum(1 for i in self._items if i.status in ('deferred', 'skipped'))
self._diag("MDL-1011", "Manual download phase completed", completed=completed, skipped=skipped)
if self._on_all_done:
self._on_all_done(completed, skipped)
if self._on_send_continue:
self._diag("MDL-1012", "Sending continue command to engine")
self._on_send_continue()
def _item_by_name(self, file_name: str) -> Optional[DownloadItem]:
for item in self._items:
if item.file_name == file_name:
return item
return None
def _refresh_watcher_pending_items(self) -> None:
"""Keep watcher tracking all non-terminal items, not only pure 'pending' ones."""
with self._lock:
pending_items = [
{'file_name': i.file_name, 'expected_hash': i.expected_hash, 'expected_size': i.expected_size}
for i in self._items
if i.status not in ('complete', 'error')
]
pending_count = len(pending_items)
sample_pending = [i['file_name'] for i in pending_items[:5]]
self._watcher.set_pending_items(pending_items)
self._diag(
"MDL-1019",
"Watcher pending list refreshed",
pending_count=pending_count,
pending_sample=json.dumps(sample_pending, ensure_ascii=True),
)
def _ingest_existing_files(self) -> int:
"""
Pre-check watch/modlist directories for already-present archives so users
do not need to re-download files that already exist.
"""
dirs: list[Path] = []
if self._watch_dir.is_dir():
dirs.append(self._watch_dir)
if self._dl_dir.is_dir() and self._dl_dir != self._watch_dir:
dirs.append(self._dl_dir)
if not dirs:
return 0
existing_files: list[Path] = []
for d in dirs:
try:
for p in d.iterdir():
if p.is_file() and p.suffix not in ('.part', '.crdownload', '.tmp'):
existing_files.append(p)
except OSError as e:
logger.warning(f"[MDL-9021] Precheck scan error: dir={d} reason={e}")
continue
if not existing_files:
self._diag("MDL-1023", "Startup precheck found no candidate files", scan_dirs=len(dirs))
return 0
exact_map: dict[str, Path] = {}
for p in existing_files:
exact_map.setdefault(p.name.lower(), p)
with self._lock:
targets = [
{'file_name': i.file_name, 'expected_hash': i.expected_hash, 'expected_size': i.expected_size}
for i in self._items
if i.status not in ('complete', 'error')
]
self._diag(
"MDL-1024",
"Startup precheck scan summary",
scan_dirs=len(dirs),
discovered_files=len(existing_files),
pending_targets=len(targets),
discovered_sample=json.dumps([p.name for p in existing_files[:5]], ensure_ascii=True),
target_sample=json.dumps([t.get('file_name', '') for t in targets[:5]], ensure_ascii=True),
)
matched = 0
used_paths: set[Path] = set()
for hint in targets:
name = hint['file_name']
exact = exact_map.get(name.lower())
if exact is None:
# Leading-dot normalization: browser may strip a leading dot that
# the engine uses in its canonical filename.
stripped = name.lower().lstrip('.')
if stripped != name.lower():
exact = exact_map.get(stripped)
if exact is None or exact in used_paths:
continue
used_paths.add(exact)
if self._on_candidate(exact, hint, from_startup_precheck=True):
matched += 1
if matched:
logger.info(f"[MDL-1025] Startup precheck queued {matched} archive(s) for validation")
else:
self._diag("MDL-1025", "Startup precheck found zero exact filename matches")
return matched
def reopen_item(self, file_name: str) -> bool:
"""Re-open a specific item's URL (e.g. if user closed browser tab accidentally)."""
notify_item: Optional[DownloadItem] = None
with self._lock:
item = self._item_by_name(file_name)
if item is None:
return False
if item.status in ('complete', 'skipped'):
return False
if item.status != 'browser_opened':
item.status = 'browser_opened'
item.needs_user_retry = False
self._active_tabs += 1
notify_item = item
if notify_item is not None:
self._notify(notify_item)
if item is None:
return False
opened, error = self._open_browser(item)
if not opened:
revert_item: Optional[DownloadItem] = None
with self._lock:
current = self._item_by_name(file_name)
if current is not None and current.status == 'browser_opened':
current.status = 'pending'
current.needs_user_retry = True
if self._active_tabs > 0:
self._active_tabs -= 1
revert_item = current
if revert_item is not None:
self._notify(revert_item)
self._diag(
"MDL-9011",
"Manual reopen failed",
level="warning",
file_name=file_name,
reason=error or "unknown launcher failure",
)
return False
self._diag("MDL-1018", "Manual item URL re-opened by user", file_name=file_name)
return True
def _maybe_log_progress_summary(self) -> None:
with self._lock:
complete = sum(1 for i in self._items if i.status == 'complete')
skipped = sum(1 for i in self._items if i.status in ('deferred', 'skipped'))
pending = sum(1 for i in self._items if i.status == 'pending')
validating = sum(1 for i in self._items if i.status == 'validating')
opened = sum(1 for i in self._items if i.status == 'browser_opened')
total = len(self._items)
if complete == self._last_progress_log_completed:
return
if complete in (1, total) or complete % 10 == 0:
self._last_progress_log_completed = complete
self._diag(
"MDL-1013",
"Manual download progress summary",
total=total,
complete=complete,
browser_opened=opened,
validating=validating,
pending=pending,
skipped=skipped,
needs_retry=sum(1 for i in self._items if i.needs_user_retry),
)
def _diag(self, code: str, message: str, level: str = "info", **ctx) -> None:
details = " ".join(f"{k}={v}" for k, v in ctx.items())
text = f"[{code}] run={self._run_id} {message}"
if details:
text = f"{text} | {details}"
if level == "warning":
logger.warning(text)
elif level == "error":
logger.error(text)
else:
logger.info(text)
def _notify(self, item: DownloadItem) -> None:
if self._on_item_updated:
try:
self._on_item_updated(item)
except Exception as e:
logger.debug(f"on_item_updated callback error: {e}")

View File

@@ -11,6 +11,8 @@ import re
import shutil
import logging
import subprocess
import tempfile
import time
from pathlib import Path
from typing import Callable, Optional, Tuple
@@ -31,10 +33,55 @@ class MO2SetupService:
GITHUB_API = "https://api.github.com/repos/ModOrganizer2/modorganizer/releases/latest"
ASSET_PATTERN = re.compile(r"Mod\.Organizer-\d+\.\d+(\.\d+)?\.7z$")
def _extract_archive(
self,
archive_path: Path,
install_dir: Path,
should_cancel: Optional[Callable[[], bool]] = None,
) -> Tuple[bool, Optional[str]]:
"""Extract the MO2 archive without interactive prompts and honor cancellation."""
process = None
try:
process = subprocess.Popen(
['7z', 'x', '-y', '-aoa', str(archive_path), f'-o{install_dir}'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
while True:
if should_cancel and should_cancel():
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=5)
return False, "MO2 setup cancelled."
returncode = process.poll()
if returncode is not None:
stdout, stderr = process.communicate()
if returncode != 0:
err = (stderr or stdout or "").strip()
return False, f"Extraction failed: {err or '7z returned a non-zero exit code.'}"
return True, None
time.sleep(0.1)
except Exception as e:
if process is not None:
try:
process.kill()
except Exception:
pass
return False, f"Extraction failed: {e}"
def setup_mo2(
self,
install_dir: Path,
shortcut_name: str = "Mod Organizer 2",
existing_appid: Optional[int] = None,
progress_callback: Optional[Callable[[str], None]] = None,
should_cancel: Optional[Callable[[], bool]] = None,
) -> Tuple[bool, Optional[int], Optional[str]]:
@@ -88,16 +135,21 @@ class MO2SetupService:
return False, None, "Could not find main MO2 .7z asset in latest release."
# Download
archive_path = install_dir / asset['name']
_progress(f"Downloading {asset['name']}...")
if _cancel_requested():
return False, None, "MO2 setup cancelled."
try:
with tempfile.NamedTemporaryFile(prefix="jackify-mo2-", suffix=".7z", delete=False) as tmp_file:
archive_path = Path(tmp_file.name)
with requests.get(asset['browser_download_url'], stream=True, timeout=120, verify=True) as r:
r.raise_for_status()
with open(archive_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
if _cancel_requested():
try:
archive_path.unlink(missing_ok=True)
except Exception:
pass
return False, None, "MO2 setup cancelled."
f.write(chunk)
except Exception as e:
@@ -107,18 +159,13 @@ class MO2SetupService:
_progress(f"Extracting to {install_dir}...")
if _cancel_requested():
return False, None, "MO2 setup cancelled."
try:
result = subprocess.run(
['7z', 'x', str(archive_path), f'-o{install_dir}'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=1200,
)
if result.returncode != 0:
err = result.stderr.decode(errors='ignore')
return False, None, f"Extraction failed: {err}"
except Exception as e:
return False, None, f"Extraction failed: {e}"
extract_ok, extract_error = self._extract_archive(archive_path, install_dir, should_cancel)
if not extract_ok:
try:
archive_path.unlink(missing_ok=True)
except Exception:
pass
return False, None, extract_error
# Validate
mo2_exe = install_dir / "ModOrganizer.exe"
@@ -149,12 +196,22 @@ class MO2SetupService:
try:
from .automated_prefix_service import AutomatedPrefixService
svc = AutomatedPrefixService()
success, prefix_path, app_id, _last_ts = svc.run_working_workflow(
shortcut_name=shortcut_name,
modlist_install_dir=str(install_dir),
final_exe_path=str(mo2_exe),
progress_callback=_progress,
)
if existing_appid is not None:
app_id = int(existing_appid)
_progress(f"Reusing existing Steam shortcut with AppID: {app_id}")
prefix_path = svc.get_prefix_path(app_id)
if prefix_path is None:
if not svc.create_prefix_with_proton_wrapper(app_id):
return False, None, "Failed to create Proton prefix for existing shortcut."
prefix_path = svc.get_prefix_path(app_id)
success = True
else:
success, prefix_path, app_id, _last_ts = svc.run_working_workflow(
shortcut_name=shortcut_name,
modlist_install_dir=str(install_dir),
final_exe_path=str(mo2_exe),
progress_callback=_progress,
)
except Exception as e:
logger.error(f"AutomatedPrefixService failed: {e}")
return False, None, f"Prefix setup failed: {e}"

View File

@@ -334,9 +334,9 @@ class ModlistService(ModlistServiceInstallationMixin):
if completion_callback:
if success:
debug_callback("Configuration completed successfully, calling completion callback")
debug_callback("Core configuration complete, calling completion callback")
# Pass ENB detection status through callback
completion_callback(True, "Configuration completed successfully!", context.name, enb_detected)
completion_callback(True, "Core configuration complete", context.name, enb_detected)
else:
debug_callback("Configuration failed, calling completion callback with failure")
completion_callback(False, "Configuration failed", context.name, False)
@@ -439,7 +439,7 @@ class ModlistService(ModlistServiceInstallationMixin):
if success:
logger.info("Modlist configuration completed successfully")
if completion_callback:
completion_callback(True, "Configuration completed successfully", context.name, False)
completion_callback(True, "Core configuration complete", context.name, False)
else:
logger.warning("Modlist configuration had issues")
if completion_callback:

View File

@@ -186,10 +186,23 @@ class ModlistServiceInstallationMixin:
clean_env = get_clean_subprocess_env()
proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=False, env=clean_env, cwd=engine_dir
)
def _write_stdin(line: str) -> bool:
try:
payload = line if line.endswith('\n') else line + '\n'
proc.stdin.write(payload.encode())
proc.stdin.flush()
return True
except (OSError, BrokenPipeError):
return False
from jackify.backend.utils.cc_content_detector import is_cc_content_error, extract_cc_filename
import json as _json
_cc_filename = None
_pending_manual: list = []
buffer = b''
while True:
chunk = proc.stdout.read(1)
@@ -197,26 +210,81 @@ class ModlistServiceInstallationMixin:
break
buffer += chunk
if chunk == b'\n':
if chunk in (b'\n', b'\r'):
line = buffer.decode('utf-8', errors='replace')
if output_callback:
output_callback(line.rstrip())
decoded = line.rstrip()
buffer = b''
elif chunk == b'\r':
line = buffer.decode('utf-8', errors='replace')
# JSON engine events - handle silently, don't pass to output_callback
if decoded.strip().startswith('{'):
try:
obj = _json.loads(decoded.strip())
event = obj.get('event')
if event == 'manual_download_required':
_pending_manual.append(obj)
continue
if event == 'manual_download_list_complete':
loop_iter = obj.get('loop_iteration', 1)
for item in _pending_manual:
item['loop_iteration'] = loop_iter
items_batch = list(_pending_manual)
_pending_manual.clear()
from jackify.backend.handlers.config_handler import ConfigHandler
raw_limit = ConfigHandler().get('manual_download_concurrent_limit', 2)
try:
manual_limit = int(raw_limit)
except (TypeError, ValueError):
manual_limit = 2
manual_limit = max(1, min(5, manual_limit))
from jackify.frontends.cli.commands.manual_download_flow import run_cli_manual_download_phase
completed = run_cli_manual_download_phase(
events=items_batch,
loop_iteration=loop_iter,
download_dir=actual_download_path,
stdin_write=_write_stdin,
output_callback=output_callback,
concurrent_limit=manual_limit,
)
if not completed:
if proc.poll() is None:
proc.terminate()
break
continue
if event == 'manual_download_phase_complete':
if output_callback:
found = obj.get('total_found', 0)
required = obj.get('total_required', 0)
output_callback(f"All manual downloads confirmed ({found}/{required}). Resuming installation...")
continue
except (_json.JSONDecodeError, ValueError):
pass
if output_callback:
output_callback(line.rstrip())
buffer = b''
output_callback(decoded)
if _cc_filename is None and is_cc_content_error(decoded):
_cc_filename = extract_cc_filename(decoded) or ""
if buffer:
line = buffer.decode('utf-8', errors='replace')
decoded = line.rstrip()
if output_callback:
output_callback(line.rstrip())
output_callback(decoded)
if _cc_filename is None and is_cc_content_error(decoded):
_cc_filename = extract_cc_filename(decoded) or ""
proc.wait()
if proc.returncode != 0:
if output_callback:
output_callback(f"Jackify Install Engine exited with code {proc.returncode}.")
if _cc_filename is not None and output_callback:
fname_note = f" ({_cc_filename})" if _cc_filename else ""
output_callback("")
output_callback(f"[WARN] Anniversary Edition Content Missing{fname_note}")
output_callback(" - Open Vanilla Skyrim SE/AE and let it run until all Creation Club content has downloaded.")
output_callback(" - From the Skyrim main menu, go into Creations and select 'Download All'.")
output_callback(" - If specific files are still missing, search for and download them from the Creations menu.")
output_callback(" - If problems persist, uninstall and reinstall Skyrim, then launch once to trigger the AE download.")
output_callback(" - Note: Skyrim AE via Steam Family Sharing does not transfer DLC content.")
return False
if output_callback:
output_callback("Installation completed successfully")

View File

@@ -16,6 +16,7 @@ from pathlib import Path
from typing import Optional, Tuple, Dict, Any, List
from ..handlers.vdf_handler import VDFHandler
from jackify.shared.steam_utils import get_ordered_steam_roots, STEAM_PREFERENCE_AUTO
logger = logging.getLogger(__name__)
@@ -30,13 +31,14 @@ class NativeSteamService:
"""
def __init__(self):
self.steam_paths = [
Path.home() / ".steam" / "steam",
Path.home() / ".local" / "share" / "Steam",
Path.home() / ".var" / "app" / "com.valvesoftware.Steam" / "data" / "Steam",
Path.home() / ".var" / "app" / "com.valvesoftware.Steam" / ".local" / "share" / "Steam",
Path.home() / ".var" / "app" / "com.valvesoftware.Steam" / "home" / ".local" / "share" / "Steam"
]
preference = STEAM_PREFERENCE_AUTO
try:
from jackify.backend.handlers.config_handler import ConfigHandler
preference = ConfigHandler().get("steam_install_preference", STEAM_PREFERENCE_AUTO)
except Exception:
pass
self.steam_paths = get_ordered_steam_roots(preference=preference)
self.steam_path = None
self.userdata_path = None
self.user_id = None
@@ -620,4 +622,4 @@ class NativeSteamService:
except Exception as e:
logger.error(f"Error creating symlink: {e}")
return False
return False

View File

@@ -26,6 +26,7 @@ class NexusOAuthProtocolMixin:
'APPIMAGE' in env or 'APPDIR' in env or
(sys.argv[0] and sys.argv[0].endswith('.AppImage'))
)
exec_path_reliable = True
if is_appimage:
if 'APPIMAGE' in env:
exec_path = env['APPIMAGE']
@@ -35,34 +36,27 @@ class NexusOAuthProtocolMixin:
logger.info("Using resolved sys.argv[0]: %s", exec_path)
else:
exec_path = sys.argv[0]
exec_path_reliable = False
logger.warning("Using sys.argv[0] as fallback: %s", exec_path)
else:
src_dir = Path(__file__).resolve().parent.parent.parent.parent
exec_path = f'bash -c \'cd "{src_dir}" && "{sys.executable}" -m jackify.frontends.gui "$@"\' --'
logger.info("DEV mode exec path: %s", exec_path)
logger.info("Source directory: %s", src_dir)
needs_update = False
if not desktop_file.exists():
needs_update = True
logger.info("Creating desktop file for protocol handler")
else:
expected_exec = f'Exec="{exec_path}" %u' if is_appimage else f'Exec={exec_path} %u'
needs_write = not desktop_file.exists()
if not needs_write and exec_path_reliable:
current_content = desktop_file.read_text()
if is_appimage:
expected_exec = f'Exec="{exec_path}" %u'
else:
expected_exec = f"Exec={exec_path} %u"
if expected_exec not in current_content:
needs_update = True
logger.info("Updating desktop file with new Exec path: %s", exec_path)
if is_appimage and ' ' in exec_path:
import re
if re.search(r'Exec=[^"]\S*\s+\S*\.AppImage', current_content):
needs_update = True
logger.info("Fixing malformed desktop file (unquoted path with spaces)")
if needs_update:
desktop_file.parent.mkdir(parents=True, exist_ok=True)
if is_appimage:
desktop_content = f"""[Desktop Entry]
needs_write = True
logger.info("Desktop file Exec path outdated, updating: %s", exec_path)
elif not needs_write and not exec_path_reliable:
logger.warning("Could not reliably determine AppImage path, keeping existing desktop file")
desktop_file.parent.mkdir(parents=True, exist_ok=True)
if needs_write and is_appimage:
desktop_content = f"""[Desktop Entry]
Type=Application
Name=Jackify
Comment=Wabbajack modlist manager for Linux
@@ -72,9 +66,9 @@ Terminal=false
Categories=Game;Utility;
MimeType=x-scheme-handler/jackify;
"""
else:
src_dir = Path(__file__).resolve().parent.parent.parent.parent
desktop_content = f"""[Desktop Entry]
elif needs_write:
src_dir = Path(__file__).resolve().parent.parent.parent.parent
desktop_content = f"""[Desktop Entry]
Type=Application
Name=Jackify
Comment=Wabbajack modlist manager for Linux
@@ -85,10 +79,14 @@ Categories=Game;Utility;
MimeType=x-scheme-handler/jackify;
Path={src_dir}
"""
if needs_write:
desktop_file.write_text(desktop_content)
logger.info("Desktop file written: %s", desktop_file)
logger.info("Exec path: %s", exec_path)
logger.info("AppImage mode: %s", is_appimage)
else:
logger.debug("Desktop file up to date, skipping write")
logger.info("Registering jackify:// protocol handler")
apps_dir = Path.home() / ".local" / "share" / "applications"
subprocess.run(['update-desktop-database', str(apps_dir)], capture_output=True, timeout=10)

View File

@@ -673,6 +673,7 @@ def robust_steam_restart(progress_callback: Optional[Callable[[str], None]] = No
final_check = subprocess.run(['pgrep', '-f', 'steamwebhelper'], capture_output=True, timeout=10, env=start_env)
if final_check.returncode == 0:
report("Steam started successfully.")
report("[Jackify] Steam restart complete")
logger.info(f"Steam confirmed running after {elapsed_wait}s wait.")
return True
else:

View File

@@ -0,0 +1,62 @@
"""Shared backend helpers for locating and installing TTW_Linux_Installer."""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Callable, Optional, Tuple
from jackify.backend.handlers.config_handler import ConfigHandler
from jackify.backend.handlers.filesystem_handler import FileSystemHandler
from jackify.backend.handlers.ttw_installer_handler import TTWInstallerHandler
logger = logging.getLogger(__name__)
def _build_handler() -> TTWInstallerHandler:
return TTWInstallerHandler(
steamdeck=False,
verbose=False,
filesystem_handler=FileSystemHandler(),
config_handler=ConfigHandler(),
)
def get_ttw_installer_path() -> Optional[Path]:
"""Return the resolved TTW_Linux_Installer executable path, if available."""
handler = _build_handler()
path = handler.ttw_installer_executable_path
if path and path.exists():
return path
return None
def ensure_ttw_installer_available(
progress_callback: Optional[Callable[[str], None]] = None,
) -> Tuple[Optional[Path], str]:
"""
Ensure TTW_Linux_Installer is installed and return its executable path.
Returns:
(path, message)
"""
existing = get_ttw_installer_path()
if existing:
return existing, "TTW_Linux_Installer ready"
if progress_callback:
progress_callback("TTW_Linux_Installer not found, installing...")
handler = _build_handler()
success, message = handler.install_ttw_installer()
if not success:
logger.error("Failed to install TTW_Linux_Installer: %s", message)
return None, message
path = handler.ttw_installer_executable_path
if path and path.exists():
if progress_callback:
progress_callback("TTW_Linux_Installer installed successfully")
return path, message
return None, "TTW_Linux_Installer install completed but executable was not found"

View File

@@ -101,13 +101,15 @@ class UpdateService:
break
if download_url:
# Prefer Nexus CDN for Premium users when release embeds nexus_file_id
release_body = release_data.get('body', '')
nexus_url = self._try_nexus_download_url(release_body)
# Prefer Nexus CDN for Premium users if this version is available there
nexus_url = self._try_nexus_download_url(latest_version)
update_source = "github"
if nexus_url:
download_url = nexus_url
update_source = "nexus"
logger.debug(f"UPD-1001 update_source_selected source=nexus version={latest_version}")
else:
logger.debug(f"UPD-1001 update_source_selected source=github version={latest_version}")
# Determine if this is a delta update
is_delta = '.delta' in download_url or 'delta' in download_url.lower()
@@ -152,54 +154,69 @@ class UpdateService:
logger.error(f"Unexpected error checking for updates: {e}")
return None
def _try_nexus_download_url(self, release_body: str) -> Optional[str]:
"""
If the user is Nexus Premium and the release body embeds nexus_file_id,
return a Nexus CDN download URL. Returns None on any failure.
_NEXUS_MOD_ID = 1427
Release body format expected:
nexus_mod_id: 12345
nexus_file_id: 67890
def _try_nexus_download_url(self, target_version: str) -> Optional[str]:
"""
If the user is Nexus Premium, query the Nexus files list for the mod
and return a CDN download URL for the file matching target_version.
Returns None on any failure or if the version is not yet on Nexus.
"""
import re
try:
mod_match = re.search(r'nexus_mod_id:\s*(\d+)', release_body, re.IGNORECASE)
file_match = re.search(r'nexus_file_id:\s*(\d+)', release_body, re.IGNORECASE)
if not file_match:
return None
nexus_file_id = int(file_match.group(1))
nexus_mod_id = int(mod_match.group(1)) if mod_match else None
from jackify.backend.services.nexus_auth_service import NexusAuthService
auth_service = NexusAuthService()
token = auth_service.get_auth_token()
if not token:
logger.debug("UPD-1002 nexus_lookup_skipped reason=missing_auth_token")
return None
auth_method = auth_service.get_auth_method()
is_oauth = auth_method == "oauth"
from jackify.backend.services.nexus_premium_service import NexusPremiumService
is_premium, _ = NexusPremiumService().check_premium_status(token)
is_premium, _ = NexusPremiumService().check_premium_status(token, is_oauth=is_oauth)
if not is_premium:
logger.debug("Nexus download skipped: user is not Premium")
logger.debug("UPD-1002 nexus_lookup_skipped reason=not_premium")
return None
if nexus_mod_id is None:
auth_headers = {"Accept": "application/json"}
if is_oauth:
auth_headers["Authorization"] = f"Bearer {token}"
else:
auth_headers["apikey"] = token
files_url = f"https://api.nexusmods.com/v1/games/site/mods/{self._NEXUS_MOD_ID}/files.json"
resp = requests.get(files_url, headers=auth_headers, timeout=8)
resp.raise_for_status()
files = resp.json().get("files", [])
# Prefer MAIN category; accept any non-archived/removed file matching the version.
match = None
for f in files:
if f.get("version") != target_version:
continue
if f.get("category_name") == "MAIN":
match = f
break
if f.get("category_name") not in ("ARCHIVED", "REMOVED"):
match = match or f
if match is None:
logger.debug(f"UPD-1002 nexus_lookup_skipped reason=version_not_on_nexus version={target_version}")
return None
api_url = f"https://api.nexusmods.com/v1/games/site/mods/{nexus_mod_id}/files/{nexus_file_id}/download_link.json"
resp = requests.get(
api_url,
headers={"apikey": token, "Accept": "application/json"},
timeout=8,
)
nexus_file_id = match["file_id"]
dl_url = f"https://api.nexusmods.com/v1/games/site/mods/{self._NEXUS_MOD_ID}/files/{nexus_file_id}/download_link.json"
resp = requests.get(dl_url, headers=auth_headers, timeout=8)
resp.raise_for_status()
links = resp.json()
if isinstance(links, list) and links:
cdn_url = links[0].get("URI")
if cdn_url:
logger.debug(f"Using Nexus CDN URL for update")
logger.debug(f"UPD-1003 nexus_lookup_success file_id={nexus_file_id} version={target_version}")
return cdn_url
logger.debug("UPD-1002 nexus_lookup_skipped reason=empty_download_links")
except Exception as e:
logger.debug(f"Nexus download URL lookup failed: {e}")
logger.debug(f"UPD-1004 nexus_lookup_failed error={e}")
return None
def _is_newer_version(self, version: str) -> bool:
@@ -494,4 +511,4 @@ rm -f "{helper_script}"
except Exception as e:
logger.error(f"Failed to create update helper script: {e}")
return None
return None

View File

@@ -13,6 +13,7 @@ Uses native Linux tools (no Wine required) by downloading from Nexus with OAuth.
import logging
import os
import json
import shutil
import subprocess
import stat
@@ -83,6 +84,110 @@ class VNVPostInstallService:
self.download_service = NexusDownloadService(auth_token)
return True
def _ensure_download_service(self, progress_callback: Optional[Callable[[str], None]] = None) -> bool:
if self.download_service is not None:
return True
return self._ensure_auth(progress_callback)
def _find_cached_4gb_patcher(self) -> Optional[Path]:
for path in self.cache_dir.iterdir():
if path.is_file() and path.suffix.lower() == ".zip" and "4gb" in path.name.lower():
return path
for path in self.cache_dir.iterdir():
if path.is_dir() and path.name.lower().endswith("_extracted") and "4gb" in path.name.lower():
for child in path.iterdir():
if child.is_file():
return child
return None
def _find_cached_bsa_mpi(self) -> Optional[Path]:
for path in self.cache_dir.iterdir():
if path.is_file() and path.suffix.lower() == ".mpi" and "bsa" in path.name.lower():
return path
for path in self.cache_dir.iterdir():
if path.is_dir() and path.name.lower().endswith("_extracted") and "bsa" in path.name.lower():
for child in path.rglob("*.mpi"):
if child.is_file():
return child
return None
def _find_cached_bsa_package(self) -> Optional[Path]:
preferred = []
fallback = []
for path in self.cache_dir.iterdir():
if not path.is_file():
continue
lower = path.name.lower()
if "bsa" not in lower or path.suffix.lower() not in {".zip", ".7z"}:
continue
if path.suffix.lower() == ".zip":
preferred.append(path)
else:
fallback.append(path)
candidates = sorted(preferred) or sorted(fallback)
return candidates[0] if candidates else None
def _extract_bsa_package(self, archive_path: Path) -> tuple[bool, Optional[Path], str]:
extract_dir = self.cache_dir / f"{archive_path.stem}_extracted"
mpi_path = next((p for p in extract_dir.rglob("*.mpi") if p.is_file()), None) if extract_dir.exists() else None
if mpi_path:
return True, mpi_path, f"Using extracted BSA package from {archive_path.name}"
extract_dir.mkdir(parents=True, exist_ok=True)
try:
suffix = archive_path.suffix.lower()
if suffix == ".zip":
with zipfile.ZipFile(archive_path, 'r') as zip_ref:
zip_ref.extractall(extract_dir)
elif suffix == ".7z":
result = subprocess.run(
["7z", "x", "-y", f"-o{extract_dir}", str(archive_path)],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False, None, (result.stderr or result.stdout or "7z extraction failed").strip()
else:
return False, None, f"Unsupported BSA package format: {archive_path.name}"
except Exception as e:
return False, None, str(e)
mpi_path = next((p for p in extract_dir.rglob("*.mpi") if p.is_file()), None)
if not mpi_path:
return False, None, f"No .mpi file found in BSA package: {archive_path.name}"
return True, mpi_path, f"Extracted BSA package {archive_path.name}"
@staticmethod
def _select_manual_download_file(files: list[dict], mod_id: int) -> Optional[dict]:
def _active(entries: list[dict]) -> list[dict]:
return [f for f in entries if f.get("category_name") not in ("ARCHIVED", "REMOVED")]
active_files = _active(files)
if mod_id == VNVPostInstallService.LINUX_4GB_PATCHER_MOD_ID:
proton_files = [
f for f in active_files
if "proton" in f.get("file_name", "").lower() and f.get("file_name", "").lower().endswith(".zip")
]
if proton_files:
proton_files.sort(key=lambda f: f.get("uploaded_timestamp", 0), reverse=True)
return proton_files[0]
if mod_id == VNVPostInstallService.FNV_BSA_DECOMPRESSOR_MOD_ID:
zip_files = [f for f in active_files if f.get("file_name", "").lower().endswith(".zip")]
if zip_files:
zip_files.sort(key=lambda f: f.get("uploaded_timestamp", 0), reverse=True)
return zip_files[0]
main_files = [f for f in active_files if f.get("category_name") == "MAIN"]
if main_files:
main_files.sort(key=lambda f: f.get("uploaded_timestamp", 0), reverse=True)
return main_files[0]
if active_files:
active_files.sort(key=lambda f: f.get("uploaded_timestamp", 0), reverse=True)
return active_files[0]
return None
def should_run_automation(self, modlist_name: str) -> bool:
"""
Check if this modlist should trigger VNV automation.
@@ -108,11 +213,60 @@ class VNVPostInstallService:
"1. Copy root mods to game directory\n"
"2. Download and run Linux 4GB patcher\n"
"3. Download and run BSA decompressor (reduces loading times)\n\n"
"Premium users: Downloads happen automatically\n"
"Non-Premium users: You'll be prompted to download files manually\n\n"
"Jackify will download the required tools automatically where possible.\n"
"If you are not a Nexus Premium member, you will be prompted to\n"
"manually download any tools that cannot be fetched automatically.\n\n"
"Would you like Jackify to automate these steps?"
)
def get_manual_download_items(self, include_bsa: bool = False) -> list:
"""
Query Nexus for the current MAIN file of each required VNV tool and return
a list of DownloadItem-compatible event dicts for use with ManualDownloadManager.
Works with any Nexus auth (not Premium-only).
Returns an empty list if auth is unavailable or queries fail.
"""
import requests as _requests
token = self.auth_service.get_auth_token()
if not token:
return []
auth_method = self.auth_service.get_auth_method()
headers = {"Accept": "application/json"}
if auth_method == "oauth":
headers["Authorization"] = f"Bearer {token}"
else:
headers["apikey"] = token
tools = [(self.LINUX_4GB_PATCHER_MOD_ID, "4GB Patcher")]
if include_bsa:
tools.append((self.FNV_BSA_DECOMPRESSOR_MOD_ID, "BSA Decompressor"))
items = []
for mod_id, label in tools:
try:
resp = _requests.get(
f"https://api.nexusmods.com/v1/games/newvegas/mods/{mod_id}/files.json",
headers=headers, timeout=8,
)
resp.raise_for_status()
files = resp.json().get("files", [])
match = self._select_manual_download_file(files, mod_id)
if match is None:
logger.warning(f"VNV tool lookup: no suitable file found for mod {mod_id} ({label})")
continue
file_id = match["file_id"]
items.append({
"file_name": match["file_name"],
"mod_name": label,
"nexus_url": f"https://www.nexusmods.com/newvegas/mods/{mod_id}?tab=files&file_id={file_id}",
"expected_hash": "",
"expected_size": match.get("size_kb", 0) * 1024,
"mod_id": mod_id,
"file_id": file_id,
})
except Exception as e:
logger.warning(f"VNV tool lookup failed for mod {mod_id} ({label}): {e}")
return items
def check_already_completed(self) -> dict:
"""
Check which VNV automation steps have already been completed.
@@ -158,11 +312,6 @@ class VNVPostInstallService:
logger.info(msg)
try:
# Ensure authentication
update_progress("Checking Nexus authentication...")
if not self._ensure_auth(progress_callback):
return False, "Nexus authentication required. Please authenticate in Settings."
# Step 1: Copy root mods
update_progress("Step 1/3: Copying root mods to game directory...")
success, msg = self.copy_root_mods()
@@ -253,25 +402,15 @@ class VNVPostInstallService:
return True, "Game already patched (backup exists)"
# Check cache first - look for extracted executable or zip
patcher_path = None
cached_extracted = list(self.cache_dir.glob("*4gb*_extracted/*"))
if cached_extracted:
# Use already extracted executable
for f in cached_extracted:
if f.is_file():
patcher_path = f
logger.info(f"Using cached extracted 4GB patcher: {patcher_path}")
break
if not patcher_path:
cached_files = list(self.cache_dir.glob("*4gb*.zip"))
if cached_files:
patcher_path = cached_files[0]
logger.info(f"Using cached 4GB patcher zip: {patcher_path}")
patcher_path = self._find_cached_4gb_patcher()
if patcher_path:
logger.info(f"Using cached 4GB patcher: {patcher_path}")
if not patcher_path:
# Try to download from Nexus
# Linux version is named "FNV4GB for Proton", not "linux"
if not self._ensure_download_service(progress_callback):
return False, "Nexus authentication required to download the 4GB patcher."
success, patcher_path, msg = self.download_service.download_latest_file(
self.GAME_DOMAIN,
self.LINUX_4GB_PATCHER_MOD_ID,
@@ -394,60 +533,58 @@ class VNVPostInstallService:
return True, "BSA decompression already completed"
if not self.ttw_installer_path or not self.ttw_installer_path.exists():
logger.warning("TTW_Linux_Installer not found, skipping BSA decompression")
return True, "BSA decompression skipped (TTW_Linux_Installer not available)"
from .ttw_installer_service import ensure_ttw_installer_available
# Check cache first
cached_files = list(self.cache_dir.glob("*BSA*.mpi"))
if cached_files:
mpi_path = cached_files[0]
self.ttw_installer_path, message = ensure_ttw_installer_available(progress_callback)
if not self.ttw_installer_path:
return False, f"TTW_Linux_Installer is required for BSA decompression: {message}"
mpi_path = self._find_cached_bsa_mpi()
if mpi_path:
logger.info(f"Using cached BSA Decompressor MPI: {mpi_path}")
else:
# Also check for exact filename match (handles spaces in filename)
exact_path = self.cache_dir / "FNV BSA Decompressor.mpi"
if exact_path.exists():
mpi_path = exact_path
logger.info(f"Using cached BSA Decompressor MPI: {mpi_path}")
else:
# Try to download from Nexus
# Look for files with .mpi extension (TTW installer format)
success, mpi_path, msg = self.download_service.download_latest_file(
package_path = self._find_cached_bsa_package()
if not package_path:
if not self._ensure_download_service(progress_callback):
return False, "Nexus authentication required to download the BSA Decompressor."
success, package_path, msg = self.download_service.download_latest_file(
self.GAME_DOMAIN,
self.FNV_BSA_DECOMPRESSOR_MOD_ID,
self.cache_dir,
file_name_filter=".mpi",
file_name_filter=".zip",
progress_callback=progress_callback
)
if not success:
# Download failed - offer manual download
logger.warning(f"Automatic download failed: {msg}")
if not manual_file_callback:
return False, f"Failed to download BSA Decompressor MPI: {msg}\n\nPlease download manually from: https://www.nexusmods.com/newvegas/mods/65854"
return False, f"Failed to download BSA Decompressor package: {msg}\n\nPlease download manually from: https://www.nexusmods.com/newvegas/mods/65854"
instructions = (
"Automatic download failed (requires Nexus Premium).\n\n"
"Please download the FNV BSA Decompressor manually:\n"
"Please download the FNV BSA Decompressor package manually:\n"
"1. Visit: https://www.nexusmods.com/newvegas/mods/65854\n"
"2. Download the .mpi file\n"
"3. Select the downloaded file below"
"2. Download the zip package\n"
"3. Select the downloaded archive below"
)
selected_path = manual_file_callback("BSA Decompressor Required", instructions)
if not selected_path or not selected_path.exists():
return False, "BSA Decompressor package not provided"
if selected_path.suffix.lower() not in {'.zip', '.7z', '.mpi'}:
return False, f"Selected file is not a supported BSA package: {selected_path}"
cached_path = self.cache_dir / selected_path.name
shutil.copy2(selected_path, cached_path)
package_path = cached_path
logger.info(f"Using manually selected BSA package: {package_path}")
mpi_path = manual_file_callback("BSA Decompressor Required", instructions)
if not mpi_path or not mpi_path.exists():
return False, "BSA Decompressor MPI file not provided"
# Validate it's an MPI file
if not mpi_path.suffix.lower() == '.mpi':
return False, f"Selected file is not an MPI file: {mpi_path}"
# Copy to cache for future use
cached_path = self.cache_dir / mpi_path.name
shutil.copy2(mpi_path, cached_path)
mpi_path = cached_path
logger.info(f"Using manually selected BSA Decompressor MPI: {mpi_path}")
if package_path.suffix.lower() == ".mpi":
mpi_path = package_path
else:
if progress_callback:
progress_callback("Preparing BSA decompressor package...")
success, mpi_path, msg = self._extract_bsa_package(package_path)
if not success or not mpi_path:
return False, f"Failed to prepare BSA Decompressor package: {msg}"
logger.info(msg)
# Create temp output directory
with tempfile.TemporaryDirectory() as temp_output:
@@ -455,7 +592,6 @@ class VNVPostInstallService:
# Create config file for TTW_Linux_Installer (handles spaces in paths better)
config_file = self.ttw_installer_path.parent / "ttw-config.json"
import json
config_data = {
"FalloutNVRoot": str(self.game_root),
"MpiPackagePath": str(mpi_path),
@@ -467,6 +603,7 @@ class VNVPostInstallService:
# Run via TTW_Linux_Installer
if progress_callback:
progress_callback("Ensuring TTW_Linux_Installer is available...")
progress_callback("Running BSA decompressor...")
cmd = [

View File

@@ -63,6 +63,7 @@ class WabbajackInstallerService:
install_folder: Path,
shortcut_name: str = "Wabbajack",
enable_gog: bool = True,
existing_appid: Optional[int] = None,
progress_callback: Optional[Callable[[str, int], None]] = None,
log_callback: Optional[Callable[[str], None]] = None
) -> Tuple[bool, Optional[int], Optional[str], Optional[int], Optional[str], Optional[str]]:
@@ -128,34 +129,6 @@ class WabbajackInstallerService:
self.handler.create_dotnet_cache(install_folder)
update_progress(".NET cache created", 3, 20)
# Step 4: Stop Steam briefly (required to safely modify shortcuts.vdf)
# We'll do a full restart after creating the shortcut
update_progress("Stopping Steam to modify shortcuts...", 4, 25)
try:
shutdown_env = _get_clean_subprocess_env()
if _is_steam_deck:
subprocess.run(['systemctl', '--user', 'stop', 'app-steam@autostart.service'],
timeout=15, check=False, capture_output=True, env=shutdown_env)
elif _is_flatpak:
subprocess.run(['flatpak', 'kill', 'com.valvesoftware.Steam'],
timeout=15, check=False, capture_output=True, env=shutdown_env)
subprocess.run(['pkill', 'steam'], timeout=15, check=False, capture_output=True, env=shutdown_env)
time.sleep(2)
check_result = subprocess.run(['pgrep', '-f', 'steamwebhelper'], capture_output=True, timeout=10, env=shutdown_env)
if check_result.returncode == 0:
subprocess.run(['pkill', '-9', 'steam'], timeout=15, check=False, capture_output=True, env=shutdown_env)
time.sleep(2)
update_progress("Steam stopped", 4, 25)
except Exception as e:
update_progress(f"Warning: Steam shutdown had issues: {e}. Proceeding anyway...", 4, 25)
# Step 5: Create Steam shortcut using NativeSteamService
update_progress("Adding Wabbajack to Steam shortcuts...", 5, 30)
# Generate launch options with STEAM_COMPAT_MOUNTS
launch_options = ""
try:
@@ -170,27 +143,58 @@ class WabbajackInstallerService:
except Exception as e:
update_progress(f"Could not generate STEAM_COMPAT_MOUNTS (non-critical): {e}", 5, 30)
success, app_id = self.steam_service.create_shortcut_with_proton(
app_name=shortcut_name,
exe_path=str(wabbajack_exe),
start_dir=str(wabbajack_exe.parent),
launch_options=launch_options,
tags=["Jackify"],
proton_version=proton_compat_name
)
if not success or app_id is None:
return False, None, None, None, None, "Failed to create Steam shortcut"
update_progress(f"Created Steam shortcut with AppID: {app_id}", 5, 30)
if existing_appid is None:
# Step 4: Stop Steam briefly (required to safely modify shortcuts.vdf)
# We'll do a full restart after creating the shortcut
update_progress("Stopping Steam to modify shortcuts...", 4, 25)
try:
shutdown_env = _get_clean_subprocess_env()
# Step 5b: Restart Steam (same pattern as modlist workflows)
update_progress("Restarting Steam...", 5, 35)
def restart_callback(msg):
update_progress(msg, 5, 35)
if _is_steam_deck:
subprocess.run(['systemctl', '--user', 'stop', 'app-steam@autostart.service'],
timeout=15, check=False, capture_output=True, env=shutdown_env)
elif _is_flatpak:
subprocess.run(['flatpak', 'kill', 'com.valvesoftware.Steam'],
timeout=15, check=False, capture_output=True, env=shutdown_env)
if not robust_steam_restart(progress_callback=restart_callback):
update_progress("Warning: Steam restart had issues, continuing anyway...", 5, 35)
subprocess.run(['pkill', 'steam'], timeout=15, check=False, capture_output=True, env=shutdown_env)
time.sleep(2)
check_result = subprocess.run(['pgrep', '-f', 'steamwebhelper'], capture_output=True, timeout=10, env=shutdown_env)
if check_result.returncode == 0:
subprocess.run(['pkill', '-9', 'steam'], timeout=15, check=False, capture_output=True, env=shutdown_env)
time.sleep(2)
update_progress("Steam stopped", 4, 25)
except Exception as e:
update_progress(f"Warning: Steam shutdown had issues: {e}. Proceeding anyway...", 4, 25)
# Step 5: Create Steam shortcut using NativeSteamService
update_progress("Adding Wabbajack to Steam shortcuts...", 5, 30)
success, app_id = self.steam_service.create_shortcut_with_proton(
app_name=shortcut_name,
exe_path=str(wabbajack_exe),
start_dir=str(wabbajack_exe.parent),
launch_options=launch_options,
tags=["Jackify"],
proton_version=proton_compat_name
)
if not success or app_id is None:
return False, None, None, None, None, "Failed to create Steam shortcut"
update_progress(f"Created Steam shortcut with AppID: {app_id}", 5, 30)
# Step 5b: Restart Steam (same pattern as modlist workflows)
update_progress("Restarting Steam...", 5, 35)
def restart_callback(msg):
update_progress(msg, 5, 35)
if not robust_steam_restart(progress_callback=restart_callback):
update_progress("Warning: Steam restart had issues, continuing anyway...", 5, 35)
else:
update_progress("Steam restarted successfully", 5, 40)
else:
update_progress("Steam restarted successfully", 5, 40)
app_id = int(existing_appid)
update_progress(f"Reusing existing Steam shortcut with AppID: {app_id}", 5, 30)
# Step 6: Initialize Wine prefix (using same method as modlist workflows)
update_progress("Creating Proton prefix...", 6, 45)
@@ -277,4 +281,3 @@ class WabbajackInstallerService:
if log_callback:
log_callback(f"ERROR: {error_msg}")
return False, None, None, None, None, error_msg

View File

@@ -0,0 +1,34 @@
"""
Detects Creation Club / Anniversary Edition content missing errors in engine output.
"""
import re
from typing import Optional
# Matches CC content file names: ccXXXsse001-name.bsa/esm/esl/esp, ccXXXfo4001-name.ba2, etc.
# No leading \b — filenames often appear with a Data_ prefix (Data_ccbgssse019-...)
# where _ is a word char and would prevent \b from matching.
_CC_FILE_RE = re.compile(
r'cc[a-z]{2,8}\d{3,4}[-\w]*\.(?:bsa|esm|esl|esp|ba2)',
re.IGNORECASE
)
_ERROR_WORDS = frozenset((
'missing', 'required', 'failed', 'unable', 'cannot', 'error', 'not found',
))
def is_cc_content_error(line: str) -> bool:
"""Return True if line indicates a missing CC/AE content file in an error context."""
if not line:
return False
normalized = line.strip().lower()
if not _CC_FILE_RE.search(normalized):
return False
return any(w in normalized for w in _ERROR_WORDS)
def extract_cc_filename(line: str) -> Optional[str]:
"""Return the CC filename from a line, or None if not found."""
m = _CC_FILE_RE.search(line)
return m.group(0) if m else None

View File

@@ -1,8 +1,10 @@
import json
import re
from typing import Optional
from jackify.shared.errors import (
JackifyError, InstallError, OAuthError,
oauth_expired, wabbajack_install_failed, format_technical_context,
game_not_found_for_modlist,
)
@@ -12,7 +14,29 @@ def _ctx_detail(ctx: dict) -> Optional[str]:
return format_technical_context(context=ctx)
def _engine_error(msg: str, ctx: dict) -> InstallError:
"""Map generic engine_error payloads to user-visible, actionable InstallError variants."""
text = (msg or "").strip()
match = re.search(r"can't find game\s+([A-Za-z0-9_:-]+)", text, flags=re.IGNORECASE)
if match:
game_name = match.group(1)
return game_not_found_for_modlist(game_name, detail=text)
return InstallError(
"Install Engine Error",
text or "An install engine error occurred.",
suggestion="Review the error message and retry after correcting the reported issue.",
solutions=[
"Check the exact error message shown above and fix the prerequisite it mentions.",
"Retry the install after restarting Steam.",
"If this persists, check Modlist_Install_workflow.log for the same error text.",
],
technical=_ctx_detail(ctx),
)
_TYPE_MAP = {
"engine_error": _engine_error,
"auth_failed": lambda msg, ctx: oauth_expired(),
"premium_required": lambda msg, ctx: InstallError(
"Nexus Premium Required",

Binary file not shown.

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More