SSH2 Spider: Installation, Configuration, and Best Practices

Troubleshooting SSH2 Spider: Common Errors and Fixes

1. Connection refused / cannot reach host

• Cause: SSH server not running, wrong host/port, or network blocked.
• Fixes:
  1. Verify host and port: ssh -p user@host (replace values).
  2. Ensure SSH service is running on target (e.g., systemctl status sshd).
  3. Check firewall/NAT and open port 22 (or custom port).
  4. Use telnet host port or nc -vz host port to test reachability.

2. Authentication failures (password or key rejected)

• Cause: Wrong credentials, key not authorized, key format mismatch, or permissions too open.
• Fixes:
  1. Confirm username/password.
  2. For key auth: ensure public key is in /.ssh/authorized_keys on target and file permissions are 700 for /.ssh and 600 for authorized_keys.
  3. Convert key formats if needed (OpenSSH vs. PEM) using ssh-keygen -i / ssh-keygen -p.
  4. Enable verbose logging: run the spider with -vvv (or enable SSH client ssh -vvv) to inspect auth negotiation.

3. Host key verification failed / mismatched host key

• Cause: Target host key changed (possible MITM) or cached key differs.
• Fixes:
  1. Inspect ~/.ssh/known_hosts entry for the host and remove the offending line: ssh-keygen -R hostname.
  2. Verify the new host key out-of-band (admin or console) before accepting.
  3. In automated testing only, allow new keys temporarily with strictness disabled — but avoid in production.

4. Timeout during commands or SFTP transfers

• Cause: Network latency, server load, aggressive timeouts, or large transfers.
• Fixes:
  1. Increase SSH timeout/retry settings in spider configuration (e.g., connection and command timeouts).
  2. Use compression (-C) for transfers or limit concurrency.
  3. Monitor server load and IO; reduce parallel sessions.

5. Partial or corrupted file downloads/uploads

• Cause: Interrupted transfers, encoding issues, or SFTP subsystem problems.
• Fixes:
  1. Use checksums (e.g., sha256sum) before and after transfer to verify integrity.
  2. Retry with resume support if available, or switch to SCP/SFTP clients that support resumption.
  3. Ensure binary mode used for non-text files.

6. Permission denied when running remote commands

• Cause: Insufficient user privileges or restricted shell.
• Fixes:
  1. Confirm the account has required permissions or use sudo with proper configuration (/etc/sudoers).
  2. Check the shell assigned to the user and any forced commands in authorized_keys.
  3. Use an account with needed privileges or adjust remote command invocation.

7. Host environment differences causing failures (PATH, shell, missing deps)

• Cause: Remote environment lacks required tools or has different PATH.
• Fixes:
  1. Execute commands with full paths (e.g., /usr/bin/rsync).
  2. Source profile files in the command (. ~/.profile) or start an interactive shell when needed.
  3. Install required dependencies on target.

8. High resource use or too many concurrent sessions

• Cause: Spider configured with aggressive concurrency causing server overload.
• Fixes:
  1. Lower concurrency and rate limits in spider config.
  2. Add exponential backoff for retries.
  3. Monitor and throttle based on server responses.

9. Cipher/kex algorithm mismatch

• Cause: Client and server have no overlapping ciphers/kex or server disables weak algorithms.
• Fixes:
  1. Update client/server to support modern algorithms (e.g., AES-GCM, ECDH).
  2. Temporarily enable compatible algorithms on the client or server with caution.
  3. Review SSH logs for negotiated algorithms.

10. Logs show “broken pipe” or connection resets

• Cause: Idle timeouts, NAT timeouts, or abrupt server disconnects.
• Fixes:
  1. Enable keepalives: set ServerAliveInterval/ClientAliveInterval.
  2. Adjust TCP keepalive settings or increase timeout on intermediate devices.
  3. Investigate server logs for reasons of disconnect.

Troubleshooting workflow (quick checklist)

1. Reproduce the issue with -vvv/detailed logging.
2. Confirm basic network reachability.
3. Verify credentials and permissions.
4. Check server-side logs (/var/log/auth.log or journalctl -u sshd).
5. Adjust timeouts, concurrency, and ciphers as needed.
6. Validate transfers with checksums and retry.

Useful commands

• Connection test: nc -vz host port
• Verbose SSH client: ssh -vvv -p port user@host
• Remove known host: ssh-keygen -R hostname
• Check service: systemctl status sshd
• Verify file integrity: sha256sum file

If you want, I can produce a concise checklist tailored to your SSH2 Spider config—provide the spider config file or its key settings.