147 Commits
Author SHA1 Message Date
ZacharyZcR d7dbccab76 merge dev into main for v2.2.0-rc.1 re-release
发布 / release (push) Has been cancelled
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-15 12:46:01 +08:00
ZacharyZcR 6eff1d5ccf fix: 外部审查 8 项修复 + 国密 TLS 按需回退
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- UserAgent 默认值回退 + 注册 -ua flag (#2)
- README 编译命令 main.go → . (#3)
- README 版本号同步 rc.1 (#4)
- Client.go gmtls stdout 劫持删除 (#5)
- ms17010 smb1GetResponse size<32 越界 panic (#6)
- SSH 拨号超时统一 ModuleTimeout (#8)
- AddPorts 死字段删除 (#9)
- 国密 TLS 按需回退:标准 TLS 握手失败时仅在错误为
  cipher/protocol 不兼容时尝试国密,跳过超时/拒绝等连接级错误
2026-06-15 04:46:25 +08:00
ZacharyZcR 2f7d2d49c6 fix: redis exploit 超时改用配置值 & 清理死代码
- redis exploit 硬编码 30s deadline 改为 config.ModuleTimeout(),与同文件其他超时一致
- 删除 BaseScanStrategy.LogPluginInfo 残留死代码(全是空操作)
- .gitignore 补充 fscan_cli/fscan_web/embed-agent 构建产物
2026-06-14 23:58:56 +08:00
ZacharyZcR 35f3cf1960 docs: release notes 移除实测验证章节
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-14 23:01:39 +08:00
ZacharyZcR 065ba6fae7 docs: 更新 v2.2.0-rc.1 release notes,覆盖全部 29 个 commit 的变更 2026-06-14 23:00:26 +08:00
ZacharyZcR 6d61b661f4 fix: 修复实机测试发现的可靠性问题 (v2.2.0-rc.1)
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- UDP 插件在 -p 指定端口时被跳过
- Redis exploit 无超时保护 / readReply 吞没非超时错误
- service_probe 连接丢失后静默成功
- SNMP 探测成功但终端无输出
- SSH 爆破不稳定 (并发过高 + 自适应超时过短 + 限流误判)
- 进度条 isActive 竞态

新增 Config.ModuleTimeout() 协议级超时下限 (≥3s)
新增 ErrorTypeThrottle 限流错误分类
2026-06-14 22:23:52 +08:00
ZacharyZcR 3babff6863 fix: -hash 支持 LM:NT 格式 & -debug 日志文件修复
1. -hash 支持标准的 LM:NT 格式 (如 aad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0)
   之前只接受纯 32 字符 NTLM hash,LM:NT 格式报 invalid hash length

2. -debug 日志文件写入修复(已在上一个 commit 中)
2026-06-14 22:23:51 +08:00
ZacharyZcR d7071b7b8e fix: -debug 日志文件写入失败,applyLogLevel 重建 Logger 时丢失 DebugLogFile 配置 2026-06-14 22:23:51 +08:00
ZacharyZcR 6ae37b8892 fix: -nopoc 禁用POC时不再输出错误日志 2026-06-14 22:23:51 +08:00
ZacharyZcR 2b202aa298 fix: 让 -gt 全局超时参数真正生效
-gt 参数之前是死代码:flag 定义了 GlobalTimeout 但从未被使用。
现在在 RunScan 中用它创建带 deadline 的 context,
超时后所有扫描任务(端口扫描、插件执行)被取消。
2026-06-14 22:23:51 +08:00
ZacharyZcR 6d7e6cd394 fix: 修复3个实测输出问题
1. 静默模式NDJSON banner截断至200字符
   Redis INFO响应~5KB导致JSONL单行过长,截断后加...后缀

2. CSV漏洞Type列补全
   ResultTypeVuln的fillDetail未设type字段,CSV Vulns列为空
   统一设为"vulnerability"

3. ICMP权限不足告警精简
   4行告警(listen失败/连接失败/权限不足/切换ping)合并为1行
2026-06-14 22:23:51 +08:00
ZacharyZcR 626d8f79bb fix: 修复3个实测发现的问题
1. -pwd 支持逗号分隔多个密码
   之前 -pwd "123,456,root" 被当作单个密码,SSH root:123 无法匹配
   现在逗号分隔为独立密码,空格保留(可能是密码的一部分)

2. -nobr 禁用爆破时仍检测 Redis 未授权访问
   未授权访问是服务探测不是爆破,不应被 -nobr 跳过
   将未授权检测移到 DisableBrute 判断之前

3. 指定端口时跳过 UDP 插件调度
   -p 80 只扫 HTTP 时不需要 SNMP/BACnet/DNS 等 UDP 探测
   仅在默认端口扫描时才分发 UDP 插件
   效果: -p 80 从 9 秒降到 3 秒
2026-06-14 22:23:51 +08:00
ZacharyZcR a52e93e84c fix: 非终端输出时禁用进度条,防止ANSI控制码覆盖扫描结果 2026-06-14 22:23:50 +08:00
ZacharyZcR 04cae2e42d fix: 彻底解决UDP插件阻塞导致扫描无法结束的问题
根因分析(通过 goroutine dump 定位):
1. UDP conn.Write 在 WSL2 上可能永久阻塞(SetDeadline 对 Write 不生效)
2. SNMP community 爆破混入通用密码字典(57个),串行 × 10s超时 = 10分钟

修复:
- 提取 udpProbe() 公共函数,用 context timeout + conn.Close 双保险
  超时后强制关闭连接,中断阻塞的 Write/Read
- BACnet/DNS/IPMI/TFTP 统一使用 udpProbe()
- SNMP probe 使用 goroutine + context select 保护
- SNMP community 列表不再混入通用密码字典(8个专用 community 足够)
- SNMP 后续 community 爆破用 3s 短超时 + 连续失败 3 次快速退出

效果: 同样的扫描从无限卡死 → 9秒完成
2026-06-14 22:23:50 +08:00
ZacharyZcR 4169eb6ee0 fix: UDP插件改用 conn.SetDeadline 替代类型断言 2026-06-14 22:23:50 +08:00
ZacharyZcR 63631d6cdf fix: 所有UDP插件添加ReadDeadline防止无限阻塞
SNMP/BACnet/DNS/IPMI/TFTP 的 conn.Read() 在目标不响应时
无限阻塞(goroutine 泄漏),导致整个扫描无法结束。

在 Write 前设置 ReadDeadline 确保超时后返回。
2026-06-14 22:23:50 +08:00
ZacharyZcR 4f6bb28138 fix: 修复6个运行时问题
1. 抑制 gmtls 库的 handshake error stdout 噪声
   gmtls/conn.go:1304 硬编码了 fmt.Println,在调用时临时重定向 os.Stdout

2. MySQL 3306 服务名误识别为 genetec-5400
   nmap 指纹库将 MySQL 握手包的随机 salt 误匹配,通过 banner 特征校正

3. 管道输出时自动禁用 ANSI 控制码
   检测 stdout 是否为终端,非终端时自动启用 NoColor

4. 进度条完成消息措辞精确化
   去掉冗余冒号,保持信息简洁一致

5. URL 模式跳过不必要的 TLS 探测
   用户已通过 -u 显式指定 http:// 协议时直接使用,不再做 TLS 握手

6. 无网络探测数据时降低默认重试次数
   -np 跳过存活探测后,将默认重试从 3 降到 2,加速不可达主机的超时
2026-06-14 22:23:49 +08:00
ZacharyZcR 46a6d812a4 fix: DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式
有 transport 字段但 rules 是数组的 POC(如 apache-httpd-cve-2021-40438)
属于 fscan 格式,不应被 xray 分支兜底。移除错误的 fallback return,
让这类 POC 正确落入 fscan 格式检测分支。

修复前: 388个POC成功380个,失败8个
修复后: 388个POC成功388个,失败0个
2026-06-14 22:23:49 +08:00
ZacharyZcR d4f4e65dec refactor: 4项架构优化 — CEL缓存/POC隔离/服务缓存/结果统一
1. CEL 表达式编译缓存
   - 新增 CelProgCache,同一 POC 的所有规则/参数组合共享编译后的 Program
   - clusterpoc 热路径上消除重复的 Compile+Program 调用

2. POC 全局状态消除
   - allPocs/pocLoaded 全局变量改为 pocStore 按 PocPath 缓存
   - 不同 PocPath 的扫描独立加载,Web API 并发场景不再互相覆盖

3. serviceCache 下沉到 per-session State
   - 服务识别缓存从包级全局 map 迁移到 State.serviceCache (sync.Map)
   - BaseScanStrategy 通过 SetState 注入 session state
   - 消除多个并发扫描之间的服务识别缓存串台

4. POC 结果输出路径统一
   - 提取 buildVulnDetails/buildVulnLogMsg/saveVulnResult 三个公共函数
   - CheckMultiPoc 和 recordVulnerabilityResult 共用统一的结果构造逻辑
   - 消除 details 字段名不一致和日志格式差异
2026-06-14 22:23:49 +08:00
ZacharyZcR a115499793 fix+perf: 修复10个bug & 10项性能优化
Bug修复:
- clustersend CEL结果判断从字符串比较改为类型断言
- Nuclei DSL matcher安全降级为false避免误报
- clusterpoc发现漏洞后返回true修正语义
- reverseCheck加10s超时防止ceye API阻塞
- doSearch/bmatches正则编译结果缓存到sync.Map
- evalset CEL求值失败时存空字符串而非原始表达式
- CEL wait()函数加nil Reverse指针检查防panic
- MongoDB readMongoMsg应用timeout参数设置读超时
- TXTWriter.Close确保Sync失败后仍调用file.Close

性能优化:
- 指纹regex缓存从RWMutex+map改为sync.Map消除锁竞争
- CaseInsensitive指纹词加载时预小写化避免匹配时分配
- 版本提取FindAllStringSubmatch限制返回数量
- i18n.Tr用strconv.Itoa替代Sprintf减少分配
- POC加载用atomic.Bool+DCLP消除热路径锁
- 结果缓冲map预分配容量减少rehash
- HTTP连接池参数随并发数动态调整
- getRuleHash去除反射+Headers排序保证确定性dedup
- POC并发加载用channel替代Mutex收集结果
2026-06-14 22:23:49 +08:00
ZacharyZcR 8402be98e3 优化自适应扫描系统 & 修复 POC 调度问题
自适应扫描优化:
- target/ceiling 分离,自适应池可向上探索而非锁死在 target
- assessHealth 阈值按网络环境区分(LAN 收紧 / Internet 放宽)
- RTT 漂移时动态压低 target,配合 AIMD 双重降速
- 去掉 semaphore 双层流控,由 ants pool 统一反压
- 探测端口从 3 个扩充到 8 个,减少 RTT 采样偏差
- computeRetries 按环境调整目标概率和上限

Bug 修复:
- AdaptivePool.Wait() 加 10 分钟超时,防止 goroutine 卡死时永久挂起
- CEL 环境初始化失败后允许重试(sync.Once → sync.Mutex + 标志位)
- CAS 自旋加 runtime.Gosched() 退避,减少高并发下 CPU 空转
- -full 模式下 web 插件跳过 IsMarkedWebService 检查 #588
- 不确定服务补做 HTTP 回退探测,覆盖自定义框架漏网场景
- POC sets 纯字面量值跳过 CEL 编译,消除大量误报错误日志
2026-06-14 22:23:48 +08:00
ZacharyZcR c49c23c7f0 Harden scan robustness and tests 2026-06-14 22:23:48 +08:00
ZacharyZcR 5ad914a1bb feat: 统一服务缓存 + 指纹驱动插件匹配
将 webServiceCache 扩展为通用 serviceCache,所有指纹识别结果
统一缓存,插件匹配时端口不命中则回退到服务名称匹配。

删除多余的 service_cache.go,复用已有的 ServiceInfo 体系。
补充 nil 防御、Explicit 标记、大量单元/集成/回归测试。
2026-06-14 22:23:47 +08:00
ZacharyZcR 2ab7c4d9b2 fix: 非标准端口的服务无法匹配对应插件 #588
端口扫描识别到 8881 上运行 SSH,但 SSH 插件只注册了 [22,2222,2200,22222],
端口不匹配导致插件不执行。

新增服务名称缓存:端口扫描阶段记录 host:port → serviceName,
插件匹配时端口不命中则回退到服务名称匹配。
2026-06-14 22:23:46 +08:00
ZacharyZcR 5c251b123d fix: 移除误导性的"无可用插件"日志 #588
扫描开始前的插件预检基于端口列表静态匹配,不代表实际扫描中插件
不会执行。移除"无可用插件"提示,避免用户误以为插件未工作。
2026-06-14 22:23:46 +08:00
ZacharyZcR 800cc30794 Fix credential cleanup and explicit tuning flags 2026-06-14 22:23:46 +08:00
ZacharyZcR 8e3cac303d fix: webtitle HTTP 请求失败时重试,修复批量扫描 POC 缺失 #587
批量扫描(-hf)时并发压力导致 HTTP 请求瞬时失败,getWebTitle
直接返回 error,跳过指纹识别和 POC 触发。

加入指数退避重试(200ms→400ms),最多 3 次,复用 config.MaxRetries。
2026-06-14 22:23:45 +08:00
ZacharyZcR 4b79cb7a18 ci: 升级 CI Go 版本到 1.25 2026-06-14 22:23:45 +08:00
ZacharyZcR 9d38874a03 fix: 降级 modernc.org/sqlite 到 v1.39.0 适配 CI 2026-06-14 22:23:45 +08:00
ZacharyZcR 88c7e4f2be feat: 自适应并发调度 — 网络探测 + AIMD + 参数智能推导
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据
推导 6 个关键参数,替代硬编码默认值:

- Timeout: median_RTT + 4σ(覆盖 99.9% 正常连接)
- ModuleThreadNum: target_concurrency / 30
- MaxRetries: ceil(log(0.01)/log(loss_rate))(全失败概率 <1%)
- ICMPRate: 环境基准 × fd 系数
- PocNum: 跟随 ModuleThreadNum
- DisablePing: 已有 ICMP 权限降级机制

线程池从单信号(资源耗尽率)升级为 AIMD + 慢启动:
- 慢启动:target/4 起步,500ms 翻倍
- 稳态 AIMD:健康 +5%,拥塞 ×0.5
- 双信号:资源耗尽率 + RTT 趋势(双 EMA)

用户 -t 显式指定时作为 ceiling,探测仍调整其他参数。

测试:单元 + 边界 + 集成 + 真实网络,core 包 580+ 用例全通过。
2026-06-14 22:23:45 +08:00
ZacharyZcR e0468ecd35 feat: Web 版独立入口 + SQLite 持久化存储
- 拆分 main.go 为 main_cli.go 和 main_web.go,Web 版不再包含 CLI 参数解析
- Web 版直接启动 HTTP 服务,通过 -port/-lang 控制,无需 -web flag
- 结果存储从内存 map 替换为 SQLite(modernc.org/sqlite,纯 Go 零 CGO)
- 数据库文件 ~/.fscan/results.db,进程重启后结果不丢失
- 修复结果分布面板跟随 tab 筛选联动的问题
2026-06-14 22:23:44 +08:00
ZacharyZcR d0295dcb92 fix: 修复 SSH 扫描 goroutine 泄漏
ssh.NewClientConn 不接受 context,context 取消后底层 TCP 连接未关闭,
导致 readLoop goroutine 永久阻塞在 conn.Read 上。大规模扫描时泄漏数万
goroutine。

- doSSHAuth 新增 goroutine 监听 context 取消并关闭底层连接
- TestSingleCredential 移除 5 秒超时放弃逻辑,改为持续等待清理
2026-06-14 22:23:44 +08:00
ZacharyZcR ade9cd1bff 修复默认扫描 POC 结果缺失 #586 2026-06-14 22:23:44 +08:00
ZacharyZcR 6d91b544de 显示 Web 服务识别 URL 2026-06-14 22:23:44 +08:00
ZacharyZcR 42092d8664 fix: 修复实机测试发现的可靠性问题 (v2.2.0-rc.1)
- UDP 插件在 -p 指定端口时被跳过
- Redis exploit 无超时保护 / readReply 吞没非超时错误
- service_probe 连接丢失后静默成功
- SNMP 探测成功但终端无输出
- SSH 爆破不稳定 (并发过高 + 自适应超时过短 + 限流误判)
- 进度条 isActive 竞态

新增 Config.ModuleTimeout() 协议级超时下限 (≥3s)
新增 ErrorTypeThrottle 限流错误分类
2026-06-14 22:14:02 +08:00
ZacharyZcR bc46e90d89 fix: -hash 支持 LM:NT 格式 & -debug 日志文件修复
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
1. -hash 支持标准的 LM:NT 格式 (如 aad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0)
   之前只接受纯 32 字符 NTLM hash,LM:NT 格式报 invalid hash length

2. -debug 日志文件写入修复(已在上一个 commit 中)
2026-06-14 09:50:10 +08:00
ZacharyZcR 67f2251da3 fix: -debug 日志文件写入失败,applyLogLevel 重建 Logger 时丢失 DebugLogFile 配置 2026-06-14 09:48:03 +08:00
ZacharyZcR e6c5e5a9a8 fix: -nopoc 禁用POC时不再输出错误日志
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-13 23:09:47 +08:00
ZacharyZcR 0356485595 fix: 让 -gt 全局超时参数真正生效
-gt 参数之前是死代码:flag 定义了 GlobalTimeout 但从未被使用。
现在在 RunScan 中用它创建带 deadline 的 context,
超时后所有扫描任务(端口扫描、插件执行)被取消。
2026-06-13 23:04:22 +08:00
ZacharyZcR b085df1878 fix: 修复3个实测输出问题
1. 静默模式NDJSON banner截断至200字符
   Redis INFO响应~5KB导致JSONL单行过长,截断后加...后缀

2. CSV漏洞Type列补全
   ResultTypeVuln的fillDetail未设type字段,CSV Vulns列为空
   统一设为"vulnerability"

3. ICMP权限不足告警精简
   4行告警(listen失败/连接失败/权限不足/切换ping)合并为1行
2026-06-13 22:55:02 +08:00
ZacharyZcR 8fb66e8e2f fix: 修复3个实测发现的问题
1. -pwd 支持逗号分隔多个密码
   之前 -pwd "123,456,root" 被当作单个密码,SSH root:123 无法匹配
   现在逗号分隔为独立密码,空格保留(可能是密码的一部分)

2. -nobr 禁用爆破时仍检测 Redis 未授权访问
   未授权访问是服务探测不是爆破,不应被 -nobr 跳过
   将未授权检测移到 DisableBrute 判断之前

3. 指定端口时跳过 UDP 插件调度
   -p 80 只扫 HTTP 时不需要 SNMP/BACnet/DNS 等 UDP 探测
   仅在默认端口扫描时才分发 UDP 插件
   效果: -p 80 从 9 秒降到 3 秒
2026-06-13 22:35:48 +08:00
ZacharyZcR 272b0e28c8 fix: 非终端输出时禁用进度条,防止ANSI控制码覆盖扫描结果 2026-06-13 22:21:03 +08:00
ZacharyZcR 28686f845d fix: 彻底解决UDP插件阻塞导致扫描无法结束的问题
根因分析(通过 goroutine dump 定位):
1. UDP conn.Write 在 WSL2 上可能永久阻塞(SetDeadline 对 Write 不生效)
2. SNMP community 爆破混入通用密码字典(57个),串行 × 10s超时 = 10分钟

修复:
- 提取 udpProbe() 公共函数,用 context timeout + conn.Close 双保险
  超时后强制关闭连接,中断阻塞的 Write/Read
- BACnet/DNS/IPMI/TFTP 统一使用 udpProbe()
- SNMP probe 使用 goroutine + context select 保护
- SNMP community 列表不再混入通用密码字典(8个专用 community 足够)
- SNMP 后续 community 爆破用 3s 短超时 + 连续失败 3 次快速退出

效果: 同样的扫描从无限卡死 → 9秒完成
2026-06-13 22:07:03 +08:00
ZacharyZcR 06ba595e32 fix: UDP插件改用 conn.SetDeadline 替代类型断言 2026-06-13 21:52:26 +08:00
ZacharyZcR 1a7770530d fix: 所有UDP插件添加ReadDeadline防止无限阻塞
SNMP/BACnet/DNS/IPMI/TFTP 的 conn.Read() 在目标不响应时
无限阻塞(goroutine 泄漏),导致整个扫描无法结束。

在 Write 前设置 ReadDeadline 确保超时后返回。
2026-06-13 21:42:49 +08:00
ZacharyZcR 2a9a3c36e2 fix: 修复6个运行时问题
1. 抑制 gmtls 库的 handshake error stdout 噪声
   gmtls/conn.go:1304 硬编码了 fmt.Println,在调用时临时重定向 os.Stdout

2. MySQL 3306 服务名误识别为 genetec-5400
   nmap 指纹库将 MySQL 握手包的随机 salt 误匹配,通过 banner 特征校正

3. 管道输出时自动禁用 ANSI 控制码
   检测 stdout 是否为终端,非终端时自动启用 NoColor

4. 进度条完成消息措辞精确化
   去掉冗余冒号,保持信息简洁一致

5. URL 模式跳过不必要的 TLS 探测
   用户已通过 -u 显式指定 http:// 协议时直接使用,不再做 TLS 握手

6. 无网络探测数据时降低默认重试次数
   -np 跳过存活探测后,将默认重试从 3 降到 2,加速不可达主机的超时
2026-06-13 19:53:28 +08:00
ZacharyZcR 70cce742e1 fix: DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式
有 transport 字段但 rules 是数组的 POC(如 apache-httpd-cve-2021-40438)
属于 fscan 格式,不应被 xray 分支兜底。移除错误的 fallback return,
让这类 POC 正确落入 fscan 格式检测分支。

修复前: 388个POC成功380个,失败8个
修复后: 388个POC成功388个,失败0个
2026-06-13 19:31:16 +08:00
ZacharyZcR 02ad8f5334 refactor: 4项架构优化 — CEL缓存/POC隔离/服务缓存/结果统一
1. CEL 表达式编译缓存
   - 新增 CelProgCache,同一 POC 的所有规则/参数组合共享编译后的 Program
   - clusterpoc 热路径上消除重复的 Compile+Program 调用

2. POC 全局状态消除
   - allPocs/pocLoaded 全局变量改为 pocStore 按 PocPath 缓存
   - 不同 PocPath 的扫描独立加载,Web API 并发场景不再互相覆盖

3. serviceCache 下沉到 per-session State
   - 服务识别缓存从包级全局 map 迁移到 State.serviceCache (sync.Map)
   - BaseScanStrategy 通过 SetState 注入 session state
   - 消除多个并发扫描之间的服务识别缓存串台

4. POC 结果输出路径统一
   - 提取 buildVulnDetails/buildVulnLogMsg/saveVulnResult 三个公共函数
   - CheckMultiPoc 和 recordVulnerabilityResult 共用统一的结果构造逻辑
   - 消除 details 字段名不一致和日志格式差异
2026-06-13 19:24:57 +08:00
ZacharyZcR 6a1636112f fix+perf: 修复10个bug & 10项性能优化
Bug修复:
- clustersend CEL结果判断从字符串比较改为类型断言
- Nuclei DSL matcher安全降级为false避免误报
- clusterpoc发现漏洞后返回true修正语义
- reverseCheck加10s超时防止ceye API阻塞
- doSearch/bmatches正则编译结果缓存到sync.Map
- evalset CEL求值失败时存空字符串而非原始表达式
- CEL wait()函数加nil Reverse指针检查防panic
- MongoDB readMongoMsg应用timeout参数设置读超时
- TXTWriter.Close确保Sync失败后仍调用file.Close

性能优化:
- 指纹regex缓存从RWMutex+map改为sync.Map消除锁竞争
- CaseInsensitive指纹词加载时预小写化避免匹配时分配
- 版本提取FindAllStringSubmatch限制返回数量
- i18n.Tr用strconv.Itoa替代Sprintf减少分配
- POC加载用atomic.Bool+DCLP消除热路径锁
- 结果缓冲map预分配容量减少rehash
- HTTP连接池参数随并发数动态调整
- getRuleHash去除反射+Headers排序保证确定性dedup
- POC并发加载用channel替代Mutex收集结果
2026-06-13 18:46:14 +08:00
ZacharyZcR 45ebe7040e 优化自适应扫描系统 & 修复 POC 调度问题
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
自适应扫描优化:
- target/ceiling 分离,自适应池可向上探索而非锁死在 target
- assessHealth 阈值按网络环境区分(LAN 收紧 / Internet 放宽)
- RTT 漂移时动态压低 target,配合 AIMD 双重降速
- 去掉 semaphore 双层流控,由 ants pool 统一反压
- 探测端口从 3 个扩充到 8 个,减少 RTT 采样偏差
- computeRetries 按环境调整目标概率和上限

Bug 修复:
- AdaptivePool.Wait() 加 10 分钟超时,防止 goroutine 卡死时永久挂起
- CEL 环境初始化失败后允许重试(sync.Once → sync.Mutex + 标志位)
- CAS 自旋加 runtime.Gosched() 退避,减少高并发下 CPU 空转
- -full 模式下 web 插件跳过 IsMarkedWebService 检查 #588
- 不确定服务补做 HTTP 回退探测,覆盖自定义框架漏网场景
- POC sets 纯字面量值跳过 CEL 编译,消除大量误报错误日志
2026-06-13 12:39:24 +08:00
ZacharyZcR 15a7670ba2 Harden scan robustness and tests 2026-06-13 07:55:37 +08:00
ZacharyZcR 1595c92aed feat: 统一服务缓存 + 指纹驱动插件匹配
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
将 webServiceCache 扩展为通用 serviceCache,所有指纹识别结果
统一缓存,插件匹配时端口不命中则回退到服务名称匹配。

删除多余的 service_cache.go,复用已有的 ServiceInfo 体系。
补充 nil 防御、Explicit 标记、大量单元/集成/回归测试。
2026-06-12 19:49:07 +08:00
ZacharyZcR 517133f72f fix: 非标准端口的服务无法匹配对应插件 #588
端口扫描识别到 8881 上运行 SSH,但 SSH 插件只注册了 [22,2222,2200,22222],
端口不匹配导致插件不执行。

新增服务名称缓存:端口扫描阶段记录 host:port → serviceName,
插件匹配时端口不命中则回退到服务名称匹配。
2026-06-12 19:31:56 +08:00
ZacharyZcR 0918eb38a6 fix: 移除误导性的"无可用插件"日志 #588
扫描开始前的插件预检基于端口列表静态匹配,不代表实际扫描中插件
不会执行。移除"无可用插件"提示,避免用户误以为插件未工作。
2026-06-12 16:36:27 +08:00
ZacharyZcR 5b7e72e56e Fix credential cleanup and explicit tuning flags 2026-06-12 15:30:44 +08:00
ZacharyZcR 52f872b8d1 fix: webtitle HTTP 请求失败时重试,修复批量扫描 POC 缺失 #587
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
批量扫描(-hf)时并发压力导致 HTTP 请求瞬时失败,getWebTitle
直接返回 error,跳过指纹识别和 POC 触发。

加入指数退避重试(200ms→400ms),最多 3 次,复用 config.MaxRetries。
2026-06-12 12:23:54 +08:00
ZacharyZcR 66b175d623 ci: 升级 CI Go 版本到 1.25 2026-06-12 11:41:49 +08:00
ZacharyZcR 6b5bc191ca fix: 降级 modernc.org/sqlite 到 v1.39.0 适配 CI 2026-06-12 10:11:34 +08:00
ZacharyZcR f883944b2b feat: 自适应并发调度 — 网络探测 + AIMD + 参数智能推导
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据
推导 6 个关键参数,替代硬编码默认值:

- Timeout: median_RTT + 4σ(覆盖 99.9% 正常连接)
- ModuleThreadNum: target_concurrency / 30
- MaxRetries: ceil(log(0.01)/log(loss_rate))(全失败概率 <1%)
- ICMPRate: 环境基准 × fd 系数
- PocNum: 跟随 ModuleThreadNum
- DisablePing: 已有 ICMP 权限降级机制

线程池从单信号(资源耗尽率)升级为 AIMD + 慢启动:
- 慢启动:target/4 起步,500ms 翻倍
- 稳态 AIMD:健康 +5%,拥塞 ×0.5
- 双信号:资源耗尽率 + RTT 趋势(双 EMA)

用户 -t 显式指定时作为 ceiling,探测仍调整其他参数。

测试:单元 + 边界 + 集成 + 真实网络,core 包 580+ 用例全通过。
2026-06-12 09:46:02 +08:00
ZacharyZcR 683707fcd4 feat: Web 版独立入口 + SQLite 持久化存储
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- 拆分 main.go 为 main_cli.go 和 main_web.go,Web 版不再包含 CLI 参数解析
- Web 版直接启动 HTTP 服务,通过 -port/-lang 控制,无需 -web flag
- 结果存储从内存 map 替换为 SQLite(modernc.org/sqlite,纯 Go 零 CGO)
- 数据库文件 ~/.fscan/results.db,进程重启后结果不丢失
- 修复结果分布面板跟随 tab 筛选联动的问题
2026-06-12 05:17:26 +08:00
ZacharyZcR b94e8bc4ca fix: 修复 SSH 扫描 goroutine 泄漏
ssh.NewClientConn 不接受 context,context 取消后底层 TCP 连接未关闭,
导致 readLoop goroutine 永久阻塞在 conn.Read 上。大规模扫描时泄漏数万
goroutine。

- doSSHAuth 新增 goroutine 监听 context 取消并关闭底层连接
- TestSingleCredential 移除 5 秒超时放弃逻辑,改为持续等待清理
2026-06-12 03:59:02 +08:00
ZacharyZcR 4198c1abc8 修复默认扫描 POC 结果缺失 #586
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-04 14:41:49 +08:00
ZacharyZcR 68f990d20b 显示 Web 服务识别 URL 2026-06-04 14:26:20 +08:00
ZacharyZcR 72042111c6 Merge pull request #580 from shadow1ng/dev
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
v2.2.0-rc Release
2026-06-01 08:54:24 +08:00
ZacharyZcR 08a87b3128 ci: 完善发版流程
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- release workflow 自动读取 .github/release-notes/<tag>.md 覆盖 body
- 移除手动 prerelease input(goreleaser auto 模式根据 tag 后缀判断)
- 新增 RELEASE.md 发版流程文档
2026-06-01 08:42:17 +08:00
ZacharyZcR cc1af631f0 fix: CI 测试排除路径 mylib → libs 2026-06-01 08:40:14 +08:00
ZacharyZcR 5078fbc975 fix: 更新 lint 排除路径 mylib → libs 2026-06-01 08:39:49 +08:00
ZacharyZcR 354ad2e24b chore: 移除 fscan-lite (C 版本)
功能过于单一(仅 TCP 端口扫描),与 fscan 品牌定位不符。
fscan-nolocal 已是精简版的正确方案。

- 删除 fscan-lite 源码和构建脚本
- goreleaser 移除 lite 构建步骤
- CI 不再安装 mingw 交叉编译工具
2026-06-01 08:37:04 +08:00
ZacharyZcR 942f0fa6c3 fix: web API 版本号改为动态获取,不再硬编码 2026-06-01 08:34:42 +08:00
ZacharyZcR 37b77f9c86 docs: 添加 v2.2.0-rc Release Notes 草稿
用于发版时 gh release create --notes-file 引用。
2026-06-01 08:28:59 +08:00
ZacharyZcR a02eed0518 refactor: mylib 重命名为 libs
更清晰的目录命名,libs/ 存放内嵌的独立协议实现库。
2026-06-01 08:22:13 +08:00
ZacharyZcR 019d13c9d4 chore: 版本号更新为 2.2.0-rc
- README/README_EN 版本标记更新
- globals.go version 常量更新
- 移除 Issue 模板中不可用的安全报告链接
2026-06-01 08:20:01 +08:00
ZacharyZcR c43982f834 chore: 更新 Issue 模板
- 版本号更新至 2.2.0-rc / 2.1.3 / 2.1.2
- 移除 placeholder 预填充内容,避免用户不删不改直接提交
- 新增 SDK、代理、mips 等选项
- macOS/Ubuntu 版本更新
- 新增安全漏洞报告入口指引
- 模板顶部增加"未填写有效信息会被关闭"提示
2026-06-01 08:17:00 +08:00
ZacharyZcR d6d323854a refactor: 完成全局状态到 session 的完整迁移
将 plugins/services、plugins/local、plugins/web、webscan 层的日志输出、
漏洞结果保存和 TCP 计数器从全局 common.Log*/GetGlobalState() 迁移到
session 实例方法,确保 SDK 并发扫描时各实例完全隔离。

- 50 个文件,所有插件日志走 session.Log*
- DoRequest 加入 session 参数,计数器走 session.State
- POC 执行器通过 POCContext.Session 传递
- 仅保留 init() 和 CEL runtime 等无 session 场景的全局回退
2026-06-01 08:13:23 +08:00
ZacharyZcR 569d21a8bc fix: harden address parsing edge cases 2026-06-01 04:03:46 +08:00
ZacharyZcR 8ec96bfe6d fix: harden scan edge cases 2026-06-01 03:32:13 +08:00
ZacharyZcR 8b558b4f12 fix: align atomic counters on arm 2026-06-01 03:03:40 +08:00
ZacharyZcR 3e4e2db722 feat: stream large host scans 2026-06-01 02:55:28 +08:00
ZacharyZcR ac755a9b4a test: avoid expanding massive CIDRs 2026-06-01 02:33:10 +08:00
ZacharyZcR ea17b5b1bc feat: allow multiple added passwords
Fixes #584
2026-06-01 02:06:15 +08:00
ZacharyZcR bfc0e0fb52 fix: support SOCKS5 proxy auth
Fixes #585
2026-06-01 01:59:08 +08:00
ZacharyZcR 40b0eee654 Remove SimpleMaxHosts limit to fix large CIDR scanning
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
Closes #583
2026-05-27 16:16:42 +08:00
ZacharyZcR 8d30ee334c Add native protocol service plugins
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-23 15:51:50 +08:00
ZacharyZcR 73cbe803c4 Expand i18n coverage 2026-05-23 15:18:40 +08:00
ZacharyZcR 9ed6cc95b6 replace string builder sprintf with fprintf
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-23 07:53:28 +08:00
ZacharyZcR a0f0379c6c silence ci warnings and staticcheck suggestions 2026-05-23 07:50:49 +08:00
ZacharyZcR 3c7823355d tighten scan session and HTTP paths 2026-05-23 07:46:53 +08:00
ZacharyZcR 1a714f6a0c reduce scanner client fingerprints 2026-05-23 07:20:24 +08:00
ZacharyZcR 0046817c2e fix mssql login client metadata 2026-05-23 07:17:55 +08:00
ZacharyZcR e0080555b4 fix: 跳过snmp社区字符串暴力破解中的冗余public探测
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-19 21:00:47 +08:00
ZacharyZcR bc2a8f653a fix: 修复新服务扫描插件的健壮性问题
- ipmi: 删除未使用的 encoding/binary 导入
- rmi: TCP读取改用 io.ReadFull 避免分片导致的解析错误
- jdwp: handshake响应读取改用 io.ReadFull 避免分片误判
- nfs: v4协议回退时使用新连接避免残留数据污染
- snmp: 修正timeout计算与其他插件保持一致
2026-05-19 20:46:22 +08:00
ZacharyZcR a92df59dfe fix: NFS plugin RPC probe + UDP dispatch in auto mode
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
NFS: use RPC NULL call to detect NFS service before MOUNT EXPORT
UDP dispatch: query plugin registry directly in auto mode instead of
GetPlugins which excludes UDP from FilterService
2026-05-19 00:37:35 +08:00
ZacharyZcR 0a28db7371 feat: add NFS, RMI and IPMI plugins
NFS (2049/TCP): Sun RPC EXPORT call, lists shared directories
RMI (1099/TCP): Java RMI handshake, detects exposed JMX/RMI endpoints
IPMI (623/UDP): RMCP ping + channel auth capabilities probe

All pure stdlib, zero new dependencies.
2026-05-19 00:01:58 +08:00
ZacharyZcR 778ba019d5 feat: add IMAP and POP3 plugins for mail server detection
IMAP (143/993): banner grab + LOGIN brute force
POP3 (110/995): banner grab + USER/PASS brute force
Both registered as safe default plugins with auth-check capability.
2026-05-18 23:29:52 +08:00
ZacharyZcR f43e3e3d0a feat: add JDWP plugin for Java debug port detection
Detects exposed JDWP (Java Debug Wire Protocol) services on ports
5005/8000/8787/5050. Performs protocol handshake and reports as VULN
since an exposed debug port allows remote code execution.
2026-05-18 23:22:15 +08:00
ZacharyZcR a1588a321f feat: SDK agent integration + UDP plugin framework + SNMP plugin
SDK enhancements for endpoint agent embedding:
- ScanWithController for pause/resume and live stats
- OnProgress callback for periodic progress reporting
- TaskID injection into every scan result
- ScanController with goroutine-safe pause/resume/stats
- Multi-target stats aggregation (race-free)

UDP plugin infrastructure:
- PluginTypeUDP registry with dedicated dispatch path
- DialUDP on ScanSession with rate limiting and packet counting
- UDP plugins bypass TCP port scan, probe targets directly
- FilterService excludes UDP plugins from TCP port matching

SNMP plugin (first UDP plugin):
- SNMPv2c GetRequest probe for sysDescr detection
- Community string brute force (public/private/community/etc)
- Pure stdlib implementation (encoding/asn1)
- Registered as safe default plugin on port 161/UDP

Tests: 95.7% SDK coverage, race-free, 50+ new test cases
2026-05-18 23:11:39 +08:00
ZacharyZcR eb4fa38fea merge sdk architecture polish 2026-05-18 21:41:01 +08:00
ZacharyZcR c15aa3488b polish embedded scanner SDK contracts 2026-05-18 21:40:52 +08:00
ZacharyZcR 218811c38d Merge pull request #582 from shadow1ng/feature/sdk-library
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
Add embeddable scanner SDK
2026-05-18 18:03:12 +08:00
ZacharyZcR 5a884ca6ad fix lint issues before merge 2026-05-18 17:59:00 +08:00
ZacharyZcR 5942d3bbcb preserve sdk host language state 2026-05-18 17:43:00 +08:00
ZacharyZcR c0a9cfd8f5 use config proxy state for credential prechecks 2026-05-18 17:13:19 +08:00
ZacharyZcR c16aa04e28 isolate session network checks 2026-05-18 17:11:24 +08:00
ZacharyZcR 856eeccd78 respect per-call session dial timeouts 2026-05-18 16:35:58 +08:00
ZacharyZcR 13f7997d16 isolate scan strategy runtime state 2026-05-18 16:33:47 +08:00
ZacharyZcR adb3ac5b74 add session-aware scan logging 2026-05-18 16:19:34 +08:00
ZacharyZcR 8de7570268 allow concurrent embedded scans 2026-05-18 16:11:41 +08:00
ZacharyZcR 3dde0c6a8e move plugin safety metadata to registry 2026-05-18 15:57:18 +08:00
ZacharyZcR d4ed0867c9 polish scanner SDK API and docs 2026-05-18 15:49:57 +08:00
ZacharyZcR 6605c93dd9 improve embedded scanner runtime 2026-05-18 14:53:00 +08:00
ZacharyZcR 6bfa05cb45 add embeddable scanner SDK 2026-05-18 14:41:15 +08:00
ZacharyZcR bce0a718ea slim mssql credential check 2026-05-18 14:00:45 +08:00
ZacharyZcR 40c3f1423e slim oracle credential check 2026-05-18 13:13:40 +08:00
ZacharyZcR bfd64f78ed 瘦身: Kafka/MongoDB/Cassandra用raw TCP替代重型依赖
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- kafka: 移除IBM/sarama(45MB), 自实现SASL PLAIN+ApiVersions协议(~150行)
- mongodb: 移除mongo-driver(25MB), 自实现OP_MSG+saslStart认证(~180行)
- cassandra: 移除gocql(1.2MB), 自实现CQLv4 STARTUP+SASL PLAIN(~130行)
- 同时移除间接依赖: pierrec/lz4, klauspost/compress, eapache/snappy等

二进制: 47MB → 40MB (-15%), 移除~55MB压缩依赖
全部13个测试包通过
2026-05-18 06:37:05 +08:00
ZacharyZcR 7063027acf 性能优化: 热路径零分配, 自适应池CAS无锁化, 锁外计算
- port_scan: fmt.Sprintf→JoinHostPort+fmtPort零分配地址格式化 (2.5x)
- port_scan: strings.ToLower→containsFold零分配大小写不敏感匹配 (2.5x)
- port_scan: slidingWindowSchedule修复semaphore泄漏bug
- service_probe: readFromConn预分配4KB缓冲区消除扩容
- adaptive_pool: maybeAdjust用atomic CAS代替持锁检查, 99%免锁
- adaptive_timeout: Timeout锁外计算均值/标准差, 只锁缓存更新
- 新增perf_bench_test.go基准测试验证所有优化
2026-05-18 06:11:10 +08:00
ZacharyZcR 639298b7c8 修复全部SA1006: fmt.Errorf(i18n)统一使用"%s"前缀 2026-05-18 05:17:40 +08:00
ZacharyZcR f427f04d35 修复SA1006: fmt.Errorf(i18n.Tr)改用fmt.Errorf("%s", ...) 2026-05-18 05:08:34 +08:00
ZacharyZcR 05e383d8bf 修复lint: errcheck返回值检查, SA1006动态格式化字符串
- scanner_core: 3处InitializeGlobalVScan()返回值用_显式丢弃
- flag/initialize/network: fmt.Errorf(i18n.Tr(...))改为fmt.Errorf("%s", i18n.Tr(...))
2026-05-18 05:07:02 +08:00
ZacharyZcR a560531e70 修复CI: 覆盖率门禁改用awk整数比较, lint过滤简化
- 覆盖率检查移除bc依赖,改用awk int()整数比较
- lint过滤用单条grep -vE替代管道链,更健壮
- 测试命令预计算包列表避免子shell问题
2026-05-18 04:57:39 +08:00
ZacharyZcR 77827bef66 修复凭证测试器计数器、消除panic、补齐i18n
- credential_tester: testCredentialWithRetry返回ErrorType,修复网络错误计数器永久不递增的bug
- scanner: os.Exit(1)改为return,defer Cleanup可正常执行
- probe_parser: 5处panic改为error返回,调用链透传到init()
- common库: parsers/initialize/network/session共17处硬编码中文改用i18n
- services插件: 18个文件115处硬编码中文改用i18n
- locale: 补齐service/parser/network相关~25个中英文键
2026-05-18 04:55:20 +08:00
ZacharyZcR c266912dcb 修复10个代码缺陷: panic防护, 死锁, 密钥泄漏, 错误吞没
- eval_random: randomInt参数max<=min时不再panic,返回CEL错误
- scanner: 长驻插件nil/panic时兜底发送ready通道,消除死锁
- poc_executor: Ceye API密钥改为环境变量CEYE_API/CEYE_DOMAIN
- Eval: ParseResponse加入oResp.Request nil检查
- Eval: reverseCheck中http.NewRequest错误不再忽略
- poc_executor: clusterpoc中CEL表达式求值错误记录日志
- winwmi: PowerShell执行失败完整记录错误信息
- sshkey: authorized_keys读取失败处理错误
- minidump: Scan结束后释放系统DLL句柄
- Windows插件: PE文件错误消息改用i18n
2026-05-18 03:47:42 +08:00
ZacharyZcR 2f2b30763c 修复硬编码中文,迁移废弃的 uuid 依赖
- flag.go: -local/-perf 参数改用 i18n.GetText,checkParameterConflicts 改用 i18n.Tr
- 补齐中英文 locale 中 flag_local_plugin/flag_perf_stats/param_local_multi_plugin
- satori/go.uuid 迁移为 google/uuid (satori 已废弃)
- CI 补充 lint Go 版本说明
2026-05-18 02:43:45 +08:00
ZacharyZcR 72403ebb32 Merge remote-tracking branch 'origin/dev' into dev
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-16 06:57:16 +08:00
ZacharyZcR 4fa8fdc30e chore: 砍掉 downloader/shellenv,新增 sshkey 插件
- 删除 downloader(curl/certutil 可替代)
- 删除 shellenv(劫持 ls 别名动静太大,实用性差)
- 新增 sshkey:生成 ed25519 密钥对,注入 authorized_keys,
  私钥保存到当前目录,支持多用户(root 权限下自动注入 root)
2026-05-16 06:56:12 +08:00
ZacharyZcR bf210dd94d refactor: minidump 实现降级链凭据提取
不再检测到杀软就直接退出,改为三级降级:
1. 直接 MiniDumpWriteDump(无杀软时尝试,15秒超时)
2. comsvcs.dll(系统签名DLL,部分杀软不拦截)
3. reg save SAM/SECURITY/SYSTEM(离线方式,不碰 LSASS)
全部失败才退出,0.2 秒完成不会卡死系统
2026-05-16 06:51:42 +08:00
ZacharyZcR cb1079598d feat: cleaner 支持清理所有持久化痕迹,minidump 增加杀软前置检测
cleaner 新增 Windows 持久化完整清理:
- Winlogon Shell/Userinit 恢复默认值
- IFEO 映像劫持清理
- 注册表 Run/RunOnce 清理
- 计划任务、服务、启动文件夹清理
- BITS 任务、WMI 事件订阅清理
- Prefetch 记录清理

minidump 增加杀软前置检测,发现 Defender/EDR 直接退出,
避免 MiniDumpWriteDump hang 导致系统崩溃
2026-05-16 06:20:00 +08:00
ZacharyZcR ec10097e76 fix: minidump 增加杀软检测和缩短超时,防止 hang 导致系统崩溃
- 新增 isAVBlocking 检测 Defender/EDR 进程,发现活跃杀软直接跳过
- dump 超时从 120 秒缩短到 15 秒(正常 dump 几秒完成)
- 三层防护:杀软检测 → 权限检测 → 超时兜底
2026-05-16 06:13:47 +08:00
ZacharyZcR 231563e82b fix: 修复 socks5proxy/cleaner/minidump 三个插件问题
- socks5proxy: 监听地址从 127.0.0.1 改为 0.0.0.0,允许外部连接
- cleaner: 重写清理逻辑,精准匹配 fscan 产物,修复 glob 遍历大目录卡死问题,
  history 清理改为真正删除 fscan 相关行
- minidump: SeDebugPrivilege 提升失败时直接退出,不再卡 120 秒超时
2026-05-16 03:17:44 +08:00
ZacharyZcR 5af8682d22 fix: 修复长驻插件启动竞态,删除重复的 bashrc 插件
- 修复 forwardshell/reverseshell/socks5proxy 因竞态导致主进程提前退出
- 删除与 shellenv 功能重复的 bashrc 插件
2026-05-16 00:12:21 +08:00
ZacharyZcR a42f074b00 feat: 新增 4 种持久化插件
- winifeo: IFEO 映像劫持 (sethc/utilman/narrator)
- winbits: BITS 后台任务持久化,通过 GUID 操作避免同名冲突
- winlogon: Winlogon Userinit/Shell 追加
- bashrc: Linux bashrc/profile 注入
2026-05-15 23:48:30 +08:00
ZacharyZcR 1ebcd6dc79 fix: 补全 Windows 持久化插件的执行逻辑
5 个 Windows 持久化插件原先只拼接命令字符串不执行,现全部补上真实执行逻辑:
- winschtask: schtasks /create 创建计划任务
- winservice: sc create 创建系统服务
- winstartup: 复制 PE 到启动文件夹
- winregistry: reg add 写入 Run/RunOnce 注册表键
- winwmi: PowerShell 创建 WMI 事件订阅(单次调用,1.2s 完成)
2026-05-15 23:35:43 +08:00
ZacharyZcR 150e48ba85 refactor: 合并 fileinfo 和 dcinfo 到 systeminfo
- fileinfo 的敏感文件扫描合并到 systeminfo
- dcinfo 的域控信息收集合并到 systeminfo,通过 build tag 隔离平台差异
- 检测到域环境时自动触发域信息收集,WORKGROUP 环境自动跳过
- 新增 systeminfo_dc_windows.go 和 systeminfo_dc_other.go 处理跨平台编译
2026-05-15 23:16:35 +08:00
ZacharyZcR 76a4331fae refactor: 合并 avdetect 到 systeminfo,修复杀软检测误报
- 将 avdetect 的进程匹配逻辑合并到 systeminfo 插件
- 修复进程匹配使用 Contains 导致大量误报,改为精确匹配
- 修正 auto.json 中 Microsoft Security Essentials 为 Microsoft Defender
- 使用 map 索引优化进程匹配性能
- 清理废弃的 envinfo/avdetect i18n key
2026-05-15 22:52:11 +08:00
ZacharyZcR ca4e1ffad3 refactor: 增强 systeminfo 插件并合并 envinfo
- systeminfo 新增网卡信息、权限检测、补丁数量、杀软检测、
  防火墙状态、敏感环境变量扫描等功能
- 合并 envinfo 到 systeminfo,删除独立的 envinfo 插件
- 修复本地插件通过 -m 指定时仍需 -h 参数的问题
- 通过回调机制解决 common/plugins 循环依赖
2026-05-15 22:47:39 +08:00
ZacharyZcR a0fc7881f2 fix: 修复 -m 指定本地插件时误走服务扫描策略的问题
当通过 -m 参数指定本地插件(如 systeminfo)时,由于 LocalMode 未被设置,
会错误地进入服务扫描流程,导致不必要的端口扫描和插件重复执行。
现在 determineScanMode 会自动检测 -m 指定的插件类型,
全部为 local 类型时自动切换到本地扫描模式。
2026-05-15 22:40:52 +08:00
ZacharyZcR bc28df2ed1 Merge branch 'main' into dev 2026-05-15 22:16:07 +08:00
ZacharyZcR 7459da2b44 fix: 恢复 plugins/local 中实际使用的 config 变量
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-15 19:54:36 +08:00
ZacharyZcR 9d191889d0 fix: 移除 composite action 中不支持的 timeout-minutes 2026-05-15 18:43:12 +08:00
ZacharyZcR 2e277a51ed fix: 修复 plugins/local 未使用变量导致编译失败 2026-05-15 18:41:36 +08:00
Ruihang ZhangandZacharyZcR 73b0b0b822 fix: 修复大网段扫描不设置上限的问题 (#581)
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
* fix: 修复大网段扫描不设置上限的问题

修复三处关键Bug:

1. parseIPFullRange() - 修复逻辑错误
   - 问题:当maxTargets=-1时,count >= -1 导致只返回1个IP
   - 修复:添加maxTargets > 0检查条件

2. parseIPShortRange() - 添加maxTargets参数支持
   - 问题:短格式范围(如192.168.1.1-254)无法被限制
   - 修复:添加maxTargets参数并在循环中检查

3. parseIPRangeString() - 修复调用传参遗漏
   - 问题:调用parseIPShortRange()未传递maxTargets
   - 修复:传入maxTargets参数

改进点:
- 大网段解析(如10.0.0.0/8)现在正确处理不超限的情况
- IP范围解析完整性验证
- 添加详细的bug报告和修复文档

* fix parser range limit handling

---------

Co-authored-by: ZacharyZcR <[email protected]>
2026-05-14 21:45:42 +08:00
ZacharyZcR d412786228 feat: 支持国密 TLS 网站扫描
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- 添加 tjfoc/gmsm 依赖,提供 gmtls 国密 TLS 支持
- 新增国密 HTTP 客户端 ClientGM/ClientNoRedirectGM
- DetectHTTPScheme 增加国密 TLS 回退检测,返回 https-gm 协议标识
- webtitle 插件识别并路由到国密客户端
- DoRequest 标准 TLS 失败时自动回退国密客户端
2026-05-13 19:10:37 +08:00
ZacharyZcR b2e91d9fc0 fix: 修复 POC Cookie/变量提取的两个问题
- poc_adapter: xray/afrog 的 output.search 转换为 Search 字段,多步POC变量传递不再丢失
- poc_executor: Set-Cookie 提取优化不再要求捕获组名含 cookie,sessid/token等命名均生效
2026-05-13 18:37:33 +08:00
ZacharyZcR 3436d6ad02 chore: 版本号更新为 2.1.4 2026-05-13 17:32:28 +08:00
ZacharyZcR a0f323a891 docs: 更新README版本号为 2.1.3 2026-05-13 17:32:27 +08:00
ZacharyZcR 46e50a021f ci: 修复发布超时 — 增加 timeout-minutes 以及步骤级超时
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-13 17:32:09 +08:00
ZacharyZcR b32ce5dec8 ci: 添加 main 分支 push 触发 CI 验证 2026-05-13 17:32:09 +08:00
ZacharyZcRandr00t 2c2ca6ace3 v2.1.3 Release (#572)
* add CVE-2026-24061 detect logic  (#562)

* add CVE-2026-24061 detect logic

* fix(telnet): 修复 errcheck 警告,统一错误处理风格

---------

Co-authored-by: ZacharyZcR <[email protected]>

* fix: 修复 Hub 广播 data race 和端口扫描潜在死锁,清理死代码

- hub.go: broadcast 路径 RLock 改 Lock,修复并发 delete/close 竞争
- port_scan.go: pool.Invoke 失败时释放 wg 和 semaphore,防止死锁
- web_scanner.go: 删除只写不读的 fingerprintCache
- webtitle.go: 移除对已删除 SetFingerprints 的调用
- keylogger.go: 删除未使用的 stopChan 和 isRunning 字段

* refactor: context 穿透扫描生命周期,修复长驻插件阻塞和 Web Stop 无效

- RunScan 接受 context.Context,创建可取消上下文并穿透到所有策略和插件
- 长驻插件(forwardshell/socks5proxy/reverseshell)不再进入 scan WaitGroup,
  通过 ctx.Done() 管理生命周期,解除 wg.Wait() 死锁
- Web Stop API 从 stopChan 改为 context.CancelFunc,取消信号真正传播到扫描链路
- ExecuteScanTasks 和 executeScanTask 支持 context 取消检查,停止分发新任务
- CLI 模式传 context.Background(),行为完全不变

* fix: 修复 Web Stop 信号等待阻塞和 SMB 响应解析越界 panic

- scanner.go: 长驻插件等待信号时同时监听 ctx.Done(),Web Stop 可正常返回
- smb_protocol.go: 响应长度检查修正为 47,远端偏移量全部做边界校验

* fix: POC 扫描接入调用方 context,修复 cachedPocPath 竞争和 ProxyStats data race

- webscan/web_scan.go: WebScan 接受 ctx 参数,替换 context.Background();
  sync.Once 改为 sync.Mutex 保护 POC 加载,消除 cachedPocPath 并发写竞争
- webtitle.go: ctx 从 Scan 穿透到 identifyFingerprintsMulti → triggerPocScan → WebScan
- webpoc.go: 传递 ctx 到 WebScan
- proxy/types.go: ProxyStats 增加 sync.Mutex
- proxy/manager.go: LastConnectTime/LastError/AverageConnectTime 读写加锁

* fix: 修复 ProxyStats 含 mutex 导致的 copylocks 告警

Stats() 方法改为手动构造副本,避免值拷贝复制 sync.Mutex

* fix: 补全 HTTP/TLS proxy stats 加锁,修复 RPC/SMB 解析越界和 POC 加载逻辑

- httpdialer.go/tlsdialer.go: LastError/LastConnectTime/AverageConnectTime 加 mutex
- findnet.go: RPC 响应结束标记位置 < 4 时跳过截断,防止负数切片 panic
- ms17010.go: SMB 会话响应最小长度改为 45,sessionSetupResponse 加长度校验
- web_scan.go: POC 加载失败时不标记 pocLoaded,允许后续重试
- Eval.go: DNSLog 配置去掉 sync.Once,允许多次扫描更新配置

* fix: Web 全局状态同步、字典文件错误提示、长驻插件连接可取消

- scan.go: Web API 构建 config/state 后同步到全局实例
- config_builder.go: 用户名/密码/URL 文件读取失败时输出错误日志
- reverseshell.go: 读命令设 1s 超时,超时后检查 ctx 实现可取消
- forwardshell.go: handleClient 接受 ctx,取消时关闭连接解除阻塞
- socks5proxy.go: handleClient 接受 ctx,取消时关闭连接解除 IO 阻塞

* refactor: 引入 ScanSession,替代全局状态穿透扫描管道 (Phase 1-3)

- 新增 common/session.go: ScanSession 结构体封装 Config/State/Params/Dialer
- RunScan/Strategy/ExecuteScanTasks/executeScanTask 全部接收 session
- Plugin 接口从 Scan(ctx, info, config, state) 改为 Scan(ctx, info, session)
- 48 个插件实现统一更新签名
- Web API 构建 ScanSession 传给 RunScan
- CLI 模式通过 Initialize() 创建 session

* refactor: 全量替换 WrapperTcpWithTimeout 为 session.DialTCP (Phase 4)

- core/port_scan.go: EnhancedPortScan/connectWithRetry/scanSinglePort 接入 session
- core/service_probe.go: SmartPortInfoScanner 持有 session,重连走 session.DialTCP
- core/icmp.go: CheckLive/tcpProbeAlive 接入 session
- 17 个 service 插件: 内部 helper 函数全部穿透 ctx+session
- 移除插件中冗余的手动 TCP 计数(DialTCP 内部已处理)
- plugins/core 下已无 WrapperTcpWithTimeout/SafeTCPDial 调用残留

* refactor: 清除 core/plugins 全局状态依赖,ProgressManager 缓存引用 (Phase 5)

- core/alive_scanner.go: GetFlagVars() → session.Params
- core/service_scanner.go: GetFlagVars() → session.Params 和 config.Target.Ports
- common/progress_manager.go: 缓存 State 和 NoColor 到字段,不再运行时读全局
- common/output_api.go: SaveResult 改用 GetGlobalConfig().Output.DisableSave
- common/network.go: WrapperTcpWithTimeout 标记 Deprecated
- core/ 和 plugins/ 下已无全局状态调用残留

* fix: 修复 dialer timeout 锁死、CVE 检测绕过 session 和误报问题

* fix: 修复 pocDNSLog data race,穿透 ctx 到全链路,消除残余 net.DialTimeout 绕过

* perf: CVE-2026-24061 检测改并发执行,消除硬 sleep 用 deadline 替代

* feat: 项目缓存系统,跨扫描合并资产,缓存 host:port 避免漏报

* perf: 三阶段性能优化,ICMP 并发提升+TCP 并行探测,端口扫描退避调整,服务探测超时减半

* fix: 修复凭据测试清理 goroutine 无限阻塞导致的 goroutine 泄漏

* fix: 凭据测试连续网络错误短路、resultChan 缓冲防阻塞、timer 泄漏修复

* perf: 大规模扫描网段预筛,按 /24 探活跳过空子网,B 段扫描从 2h+ 降至 2min

* fix: 网段预筛从抽样改全覆盖,每台主机发 1 个探测包,消除漏报

* perf: 网段预筛增加网关启发式,.1/.254 多端口优先探测,命中即跳过逐主机兜底

* fix: MSSQL 连接加 encrypt=disable 修复无 TLS 环境扫描失败,Web API 参数校验负数

* feat: Release 增加 armv5 架构支持

* chore: bump version to 2.1.3

* fix: 锁定 golangci-lint 版本为 v2.12.1 修复 CI checksum 校验失败

* fix: golangci-lint 改用 go install 安装,绕过上游安装脚本 checksum 校验问题

* feat: -silent 模式输出 NDJSON 到 stdout,支持 AI agent 管道消费

- 新增 StdoutNDJSONWriter,silent 模式下每条扫描结果实时输出一行 JSON
- LogWithProgress 层拦截人类可读日志,绕过 logger sync.Once 初始化时序问题
- 支持 fscan -h xxx -silent | jq 管道用法

* fix rdp invalid random panic (#573)

* restore ms17010 legacy detection and exploit (#574)

* fix ms17010 legacy packet decoding (#574)

* fix csv web title output (#575)

* fix web result protocol output (#577)

* feat: add -ntp flag to disable TCP supplementary probe

* fix: skip TCP supplementary probe in icmp mode

* feat: add -debug flag with file logging to fscan_debug.log

* fix: resolve golangci-lint errcheck and staticcheck warnings

* fix: skip proxy deep verification for SOCKS5 connections (#579)

SOCKS5 protocol validates connection reachability at protocol level,
deep verification was incorrectly rejecting non-banner services like
SMB(445), RPC(139) and Kerberos(88).

* fix: exclude timeout from scan failure rate calculation (#578)

Timeout is a normal scan result when firewalls drop packets, not a
scan failure. Only resource exhaustion errors count toward failure rate.

* feat: flatten NDJSON output for AI agent consumption and add SKILL.md

* perf: 端口扫描自适应超时,基于 RTT 采样动态调整连接超时

* perf: 四项扫描性能优化

- SO_LINGER=0 快速释放连接,减少 TIME_WAIT 堆积
- 服务探测超时自适应,RTT 采样约束读超时上限
- 端口扫描结果流式传递,pipeline 并行端口扫描和插件执行
- ICMP 批量预构建包和地址,减少发送循环开销

* perf: 六项性能优化

- DNS 解析缓存:sync.Map 缓存避免重复系统调用
- 凭据测试 TCP 预检:不可达目标直接跳过全部凭据
- Web 探测 HTTP Client 复用:全局共享连接池
- 端口扫描 Bloom Filter 去重:替代 map 降低内存
- 进度条 atomic 累加 + 50ms 节流渲染:消除锁竞争
- 服务探针预解码:Init 时预编译,运行时零解码开销

* refactor: replace bloom filter with map for deduplication

Bloom filter has false positive risk which can silently drop valid
scan results. Map provides exact deduplication with negligible memory
overhead at the scale of open ports (typically thousands, not millions).

* fix: credential TCP precheck bypass proxy and pipeline goroutine leak

- Skip TCP precheck when proxy is enabled, net.DialTimeout cannot
  reach targets behind SOCKS5/HTTP proxy
- Drain stream channel on ctx cancellation to prevent EnhancedPortScan
  goroutine from blocking on a full channel

* fix: stream channel 提前返回未关闭导致 goroutine 泄漏,服务探测超时下限 500ms

* fix: resolve golangci-lint errcheck and staticcheck warnings

---------

Co-authored-by: r00t <[email protected]>
2026-05-13 14:41:23 +08:00
322 changed files with 32305 additions and 8276 deletions
+17 -18
View File
@@ -1,6 +1,6 @@
name: 🐛 Bug 报告
description: 报告扫描异常、崩溃或错误行为
title: "[Bug] 简要描述问题"
title: "[Bug] "
labels: ["bug"]
body:
@@ -8,6 +8,7 @@ body:
attributes:
value: |
感谢您提交 Bug 报告!请尽可能详细地填写以下信息,这将帮助我们更快定位和修复问题。
**请勿直接提交未修改的模板**,未填写有效信息的 Issue 会被直接关闭。
- type: dropdown
id: module
@@ -21,8 +22,10 @@ body:
- 弱口令爆破 (Brute Force)
- POC/漏洞扫描 (POC Scan)
- Web指纹识别 (Web Fingerprint)
- 代理功能 (Proxy)
- 输出/日志 (Output/Logging)
- 命令行参数 (CLI Arguments)
- 嵌入式 SDK (Embedded SDK)
- 其他 (Other)
validations:
required: true
@@ -32,7 +35,7 @@ body:
attributes:
label: 严重程度
options:
- 崩溃/无法使用 (Crash)
- 崩溃/Panic (Crash)
- 功能异常 (Malfunction)
- 结果不准确 (Inaccurate)
- 性能问题 (Performance)
@@ -44,10 +47,7 @@ body:
id: description
attributes:
label: 问题描述
description: 清晰描述遇到的问题
placeholder: |
发生了什么?
预期的行为是什么?
description: 清晰描述遇到的问题,包括预期行为和实际行为
validations:
required: true
@@ -55,10 +55,7 @@ body:
id: reproduce
attributes:
label: 复现步骤
description: 提供可以复现问题的命令或步骤
placeholder: |
1. 执行命令: fscan -h xxx -p xxx
2. 观察到...
description: 提供可以复现问题的完整命令(请脱敏 IP/密码)
render: shell
validations:
required: true
@@ -75,10 +72,12 @@ body:
attributes:
label: fscan 版本
options:
- 2.2.0-rc (dev)
- 2.1.3
- 2.1.2
- 2.1.0
- 2.0.1
- 2.0.0
- 1.8.4
- 2.0.x
- 1.8.x
- 其他/自编译
validations:
required: true
@@ -92,14 +91,13 @@ body:
- Windows 10
- Windows Server 2022
- Windows Server 2019
- Ubuntu 24.04
- Ubuntu 22.04
- Ubuntu 20.04
- CentOS 7
- CentOS 8/Stream
- CentOS 7/8/Stream
- Debian 11/12
- Kali Linux
- macOS 15 (Sequoia)
- macOS 14 (Sonoma)
- macOS 13 (Ventura)
- 其他 Linux
- 其他
validations:
@@ -114,6 +112,7 @@ body:
- arm64 (aarch64)
- 386 (x86)
- arm
- mips/mipsle
validations:
required: true
@@ -121,4 +120,4 @@ body:
id: additional
attributes:
label: 补充信息
description: 其他可能有助于排查问题的信息(自编译请注明 Go 版本)
description: 其他有助于排查的信息(网络环境、是否使用代理、自编译请注明 Go 版本
+4 -5
View File
@@ -1,11 +1,10 @@
# Issue 模板配置
# 禁止空白 issue,强制用户选择模板
blank_issues_enabled: false
contact_links:
- name: 📖 使用文档
- name: 📖 使用文档 / README
url: https://github.com/shadow1ng/fscan/blob/main/README.md
about: 提交 Issue 前请先查阅文档
- name: 💬 讨论区
about: 提交 Issue 前请先查阅文档,确认不是用法问题
- name: 💬 讨论区 / Discussions
url: https://github.com/shadow1ng/fscan/discussions
about: 一般性问题和讨论请使用 Discussions
about: 使用疑问、经验分享、一般性讨论请使用 Discussions
+29 -35
View File
@@ -1,13 +1,14 @@
name: 🎯 误报/漏报
description: 报告扫描结果不准确的问题
title: "[Accuracy] 服务名 - 误报/漏报描述"
title: "[Accuracy] "
labels: ["accuracy"]
body:
- type: markdown
attributes:
value: |
感谢您帮助提高 fscan 的准确性!误报和漏报都是需要优化的问题。
感谢您帮助提高 fscan 的准确性!
**请提供对比验证结果**(如 nmap、手动连接等),仅描述"感觉不对"的报告无法处理。
- type: dropdown
id: type
@@ -34,11 +35,20 @@ body:
validations:
required: true
- type: textarea
id: command
attributes:
label: 使用的命令
description: 执行的 fscan 命令(请脱敏真实 IP/密码)
render: shell
validations:
required: true
- type: textarea
id: fscan-output
attributes:
label: fscan 输出结果
description: 粘贴相关的扫描输出(请脱敏敏感信息如真实IP、密码等
description: 粘贴相关的扫描输出(请脱敏敏感信息)
render: shell
validations:
required: true
@@ -46,11 +56,8 @@ body:
- type: textarea
id: actual
attributes:
label: 实际情况
description: 描述目标的真实状态
placeholder: |
实际上这个端口是关闭的 / 服务版本是 xxx / 密码不是 xxx...
验证方式: 通过 nmap/手动连接/其他工具 确认...
label: 实际情况及验证方式
description: 描述目标的真实状态,以及你通过什么方式验证的(nmap、手动连接、其他工具等)
validations:
required: true
@@ -59,15 +66,13 @@ body:
attributes:
label: 目标操作系统
options:
- Windows Server 2022
- Windows Server 2019
- Windows Server 2016
- Windows 10/11
- Ubuntu
- Windows Server
- Windows Desktop
- Ubuntu/Debian
- CentOS/RHEL
- Debian
- 其他 Linux
- 网络设备
- 容器/Docker
- 未知
validations:
required: true
@@ -77,20 +82,10 @@ body:
attributes:
label: 网络环境
options:
- 直连
- 通过代理
- 直连(同网段)
- 跨网段/路由
- 通过代理 (SOCKS5/HTTP)
- VPN
- 跨网段
validations:
required: true
- type: textarea
id: command
attributes:
label: 使用的命令
description: 执行的 fscan 命令
placeholder: "fscan -h x.x.x.x -p 1-65535 -pwdf pass.txt"
render: shell
validations:
required: true
@@ -99,10 +94,12 @@ body:
attributes:
label: fscan 版本
options:
- 2.2.0-rc (dev)
- 2.1.3
- 2.1.2
- 2.1.0
- 2.0.1
- 2.0.0
- 1.8.4
- 2.0.x
- 1.8.x
- 其他/自编译
validations:
required: true
@@ -110,8 +107,5 @@ body:
- type: textarea
id: suggestion
attributes:
label: 改进建议
description: 如果您有改进的想法,请分享
placeholder: |
建议增加 xxx 判断条件...
或者调整 xxx 检测逻辑...
label: 改进建议(可选)
description: 如果您有改进判断逻辑的想法
+9 -15
View File
@@ -1,13 +1,14 @@
name: ✨ 功能请求
description: 提议新功能或改进现有功能
title: "[Feature] 一句话描述功能"
title: "[Feature] "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
感谢您的功能建议!请详细描述您的需求,这将帮助我们评估和实现。
感谢您的功能建议!请描述清楚使用场景和期望行为。
纯 "+1" 类请求请用 Reaction 表达,不要开新 Issue。
- type: dropdown
id: category
@@ -19,6 +20,7 @@ body:
- 用户体验 (UX/CLI)
- 输出格式 (Output Format)
- 配置选项 (Configuration)
- 嵌入式 SDK (Embedded SDK)
- 集成/API (Integration/API)
- 其他 (Other)
validations:
@@ -28,10 +30,7 @@ body:
id: problem
attributes:
label: 解决什么问题?
description: 描述您遇到的痛点或使用场景
placeholder: |
在进行 xxx 操作时,我希望能够...
目前的问题是...
description: 描述您的实际使用场景和痛点
validations:
required: true
@@ -39,20 +38,15 @@ body:
id: solution
attributes:
label: 期望的解决方案
description: 描述您希望的功能行为
placeholder: |
希望能够通过 -xxx 参数来...
或者增加一个新的模块来...
description: 描述您希望的功能行为,最好包含预期的命令行用法示例
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: 替代方案
description: 考虑过的其他解决方案或变通方法
placeholder: |
目前我通过 xxx 方式来解决,但是...
label: 替代方案(可选)
description: 目前的变通方法,或考虑过的其他方案
- type: dropdown
id: priority
@@ -60,7 +54,7 @@ body:
label: 优先级建议
description: 您认为这个功能的重要程度
options:
- 高 - 核心功能缺失
- 高 - 核心场景缺失,严重影响使用
- 中 - 明显改善体验
- 低 - 锦上添花
validations:
+12 -29
View File
@@ -1,6 +1,6 @@
name: 🔌 新插件/协议支持
description: 请求支持新的服务、协议或漏洞检测
title: "[Plugin] 协议/服务名称"
title: "[Plugin] "
labels: ["plugin", "enhancement"]
body:
@@ -8,6 +8,7 @@ body:
attributes:
value: |
感谢您的插件请求!fscan 持续扩展对各种服务和协议的支持。
提交前请先确认该服务尚未被支持(查看 README 的支持列表)。
- type: dropdown
id: type
@@ -25,7 +26,7 @@ body:
id: service
attributes:
label: 服务/协议名称
placeholder: "如: Kafka, ClickHouse, etcd, Consul"
description: 例如 ClickHouseetcdConsul、Zookeeper
validations:
required: true
@@ -33,49 +34,31 @@ body:
id: port
attributes:
label: 默认端口
placeholder: "如: 9092, 8123, 2379"
description: 例如 9000、2379、8500
- type: textarea
id: description
attributes:
label: 服务描述
description: 简要介绍这个服务/协议
placeholder: |
这是一个用于 xxx 的服务...
在内网环境中常见于...
label: 服务描述与检测思路
description: 简要介绍服务用途、在内网中的常见程度,以及如何识别(Banner 特征、认证方式等)
validations:
required: true
- type: textarea
id: detection
attributes:
label: 识别方法
description: 如何识别/检测这个服务(如有了解)
placeholder: |
Banner 特征: xxx
默认响应: xxx
认证方式: xxx
- type: textarea
id: reference
attributes:
label: 参考资料
description: 相关文档、其他工具实现、漏洞详情
placeholder: |
- 官方文档: https://...
- 其他工具实现: https://...
- CVE编号: CVE-xxxx-xxxx
label: 参考资料(可选)
description: 官方文档链接、协议规范、其他工具实现、CVE 编号
- type: dropdown
id: prevalence
attributes:
label: 使用普遍程度
description: 这个服务在目标环境中的常见程度
label: 内网常见程度
options:
- 非常常见 (企业环境标配)
- 较为常见 (经常遇到)
- 非常常见企业环境标配
- 较为常见经常遇到
- 偶尔遇到
- 较少见但重要
- 较少见但高价值
validations:
required: true
+64
View File
@@ -0,0 +1,64 @@
# 发版流程
## 预检查
```bash
# 1. 确认 CI 通过
gh run list --branch dev --limit 3
# 2. 全平台 dry-run(手动触发 snapshot 模式)
gh workflow run release.yml -f snapshot=true
# 3. 确认版本号一致
grep "version" common/globals.go
grep "版本" README.md
```
## 发版
```bash
# 1. 确认 release notes 已就绪
cat .github/release-notes/v<VERSION>.md
# 2. 打 tag(在 dev 分支打 RC,在 main 分支打正式版)
git tag v<VERSION>
git push origin v<VERSION>
# CI 自动执行:
# - goreleaser 全平台构建 + UPX 压缩
# - 创建 GitHub ReleaseRC 自动标记 pre-release
# - 用 .github/release-notes/ 下的文件覆盖 release body
```
## 版本号规范
| 场景 | 格式 | 分支 | 示例 |
|------|------|------|------|
| 正式版 | `vX.Y.Z` | main | `v2.2.0` |
| 预发布 | `vX.Y.Z-rc` | dev | `v2.2.0-rc` |
| 热修复 | `vX.Y.Z` | main | `v2.2.1` |
## Release Notes 模板
放在 `.github/release-notes/<tag>.md`,格式参考 `v2.2.0-rc.md`
如果文件不存在,goreleaser 会自动生成基于 commit 的 changelog。
## 正式版发布(RC → 正式)
```bash
# 1. 合并 dev 到 main
git checkout main
git merge dev
git push
# 2. 更新版本号去掉 -rc
# common/globals.go, README.md, README_EN.md
# 3. 准备正式版 release notes
# .github/release-notes/v2.2.0.md
# 4. 打 tag
git tag v2.2.0
git push origin v2.2.0
```
+5 -17
View File
@@ -24,15 +24,12 @@ runs:
steps:
- name: 设置 Go 环境
uses: actions/setup-go@v5
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
with:
go-version: ${{ inputs.go-version }}
cache: true
- name: 安装 C 编译工具
shell: bash
run: |
sudo apt-get update -qq
sudo apt-get install -y gcc make mingw-w64 gcc-multilib g++-multilib
- name: 下载依赖
shell: bash
@@ -59,12 +56,12 @@ runs:
- name: 上传产物
uses: actions/upload-artifact@v4
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
if: always()
with:
name: build-${{ inputs.mode }}-${{ github.run_id }}
path: |
dist/
dist-lite/
path: dist/
retention-days: ${{ inputs.retention-days }}
- name: 生成报告
@@ -83,7 +80,6 @@ runs:
## 构建产物
### fscan (Go 版本)
$(if [ -d "dist" ]; then
echo "- 文件数: $(find dist -type f 2>/dev/null | wc -l)"
echo "- 大小: $(du -sh dist 2>/dev/null | cut -f1)"
@@ -91,13 +87,5 @@ runs:
echo "- 无产物"
fi)
### fscan-lite (C 版本)
$(if [ -d "dist-lite" ]; then
echo "- 文件数: $(find dist-lite -type f 2>/dev/null | wc -l)"
echo "- 大小: $(du -sh dist-lite 2>/dev/null | cut -f1)"
else
echo "- 无产物"
fi)
[查看产物](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
EOF
+4 -8
View File
@@ -4,8 +4,6 @@ before:
hooks:
- go mod tidy
- go mod download
- chmod +x .github/scripts/build-lite.sh
- bash .github/scripts/build-lite.sh {{ .Version }}
builds:
# 标准版 - 全部插件(全架构)
@@ -16,7 +14,7 @@ builds:
- CGO_ENABLED=0
goos: [windows, linux, darwin, freebsd, solaris]
goarch: [amd64, arm64, "386", arm, mips, mips64, mipsle]
goarm: ["6", "7"]
goarm: ["5", "6", "7"]
gomips: [softfloat]
ignore:
- goos: darwin
@@ -69,7 +67,7 @@ builds:
- CGO_ENABLED=0
goos: [windows, linux, darwin, freebsd, solaris]
goarch: [amd64, arm64, "386", arm, mips, mips64, mipsle]
goarm: ["6", "7"]
goarm: ["5", "6", "7"]
gomips: [softfloat]
ignore:
- goos: darwin
@@ -231,15 +229,13 @@ release:
| 平台 | 架构 |
|------|------|
| Linux | x64, x32, arm64, armv6, armv7, mips, mips64, mipsle |
| Linux | x64, x32, arm64, armv5, armv6, armv7, mips, mips64, mipsle |
| Windows | x64, x32 |
| macOS | x64, arm64 |
| FreeBSD | x64, x32, arm64, armv6, armv7 |
| FreeBSD | x64, x32, arm64, armv5, armv6, armv7 |
| Solaris | x64 |
footer: |
**完整更新日志**: https://github.com/{{ .Env.GITHUB_OWNER }}/{{ .Env.GITHUB_REPO }}/compare/{{ .PreviousTag }}...{{ .Tag }}
extra_files:
- glob: ./dist-lite/*
snapshot:
name_template: "{{ incpatch .Version }}-dev-{{ .ShortCommit }}"
+117
View File
@@ -0,0 +1,117 @@
# fscan v2.2.0-rc.1
> ⚠️ **这是预发布版本 (Release Candidate)**,可能存在未发现的问题。
> 如果你在使用中遇到任何异常,请积极通过 [Issue](https://github.com/shadow1ng/fscan/issues/new/choose) 反馈,帮助我们尽快稳定正式版。
> 生产环境建议继续使用 [v2.1.3](https://github.com/shadow1ng/fscan/releases/tag/v2.1.3)。
---
## 与 v2.2.0-rc 的变更
本版本基于大量实机测试反馈,**修复 30+ 个问题,新增自适应扫描系统**。183 个文件变更。
---
### 🚀 新功能
#### 自适应并发调度
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据推导关键参数,替代硬编码默认值:
- **Timeout**: `median_RTT + 4σ`(覆盖 99.9% 正常连接),下限 1s,上限 10s
- **ModuleThreadNum**: `ThreadNum / 30`,下限 5,上限 50
- **MaxRetries**: 基于丢包率推导,保证全失败概率 <1%
- **ICMPRate / PocNum**: 跟随环境和并发自动调整
线程池升级为 **AIMD + 慢启动**:慢启动阶段 500ms 翻倍,稳态 AIMD(健康 +5%,拥塞 ×0.5),双信号(资源耗尽率 + RTT 趋势)驱动。
#### 协议级超时下限(ModuleTimeout
新增 `Config.ModuleTimeout()` 方法,保证插件级交互超时不低于 3s。自适应系统将端口扫描超时压到 1s 时,SSH 握手、SNMP 探测、数据库认证等多轮交互协议不再受影响。全部 44 个服务插件已迁移。
#### 限流错误分类(ErrorTypeThrottle
新增 `ErrorTypeThrottle` 错误类型,区分服务端限流(SSH MaxStartups 等)和真正的网络不可达。限流错误不计入连续失败计数,触发 500ms 退避后继续,避免误判目标不可达而提前放弃。
#### Web 版独立入口
- 拆分 `main.go``main_cli.go``main_web.go`
- Web 版结果存储从内存替换为 SQLite 持久化(纯 Go 零 CGO
---
### 🐛 Bug 修复
#### 插件调度(#586 #587 #588
- **非标准端口服务无法匹配插件** — SSH 在 8881 端口,端口匹配失败导致插件不执行。新增服务名称缓存 + 指纹驱动回退匹配 (#588)
- **移除误导性的"无可用插件"日志** — 预检基于静态端口匹配,不代表实际不执行 (#588)
- **默认扫描 POC 结果缺失** — `executeRules` 返回空 `vulName` 导致检测结果被丢弃 (#586)
- **批量扫描(-hf)POC 缺失** — 并发压力下 HTTP 请求瞬时失败未重试,跳过指纹识别和 POC 触发。加入指数退避重试 (#587)
- **UDP 插件在 `-p` 指定端口时被跳过** — 现在按用户指定端口过滤并正确调度
- **`-full` 模式下 Web 插件跳过 `IsMarkedWebService` 检查**
- **不确定服务补做 HTTP 回退探测**,覆盖自定义 HTTP 框架漏网场景
#### UDP 插件
- **UDP 插件阻塞导致扫描无法结束** — `conn.Read()` 在目标不响应时无限阻塞。所有 UDP 插件(SNMP/BACnet/DNS/IPMI/TFTP)统一使用 context timeout + conn.Close 双保险
- **SNMP community 爆破混入通用密码字典** — 57 个通用密码串行探测导致 10 分钟阻塞,精简为 8 个专用 community
#### SSH
- **SSH goroutine 泄漏** — `ssh.NewClientConn` 不接受 contextcontext 取消后底层 TCP 连接未关闭,大规模扫描时泄漏数万 goroutine
- **SSH 握手无 TCP deadline 兜底** — 在 `NewClientConn` 前设置 deadline,握手成功后清除
- **SSH 爆破并发过高** — 从 30 降至 3,避免触发 OpenSSH MaxStartups 限流
#### Redis
- **Redis exploit 无超时保护** — exploit 阶段移除了全部 deadline,改为 30s 超时
- **Redis readReply 吞没非超时错误** — 现在仅对 timeout 类型错误做容忍
#### POC 引擎
- **DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式** — 修复后 388 个 POC 全部正确加载(之前 8 个失败)
- **CEL clustersend 结果判断错误** — 从字符串比较改为类型断言
- **CEL wait() 函数 nil Reverse 指针 panic**
- **reverseCheck 无超时** — 加 10s 超时防止 ceye API 阻塞
- **正则编译结果未缓存** — `doSearch`/`bmatches` 缓存到 `sync.Map`
#### 参数与输出
- **`-gt` 全局超时参数是死代码** — 现在真正生效,超时后取消所有扫描任务
- **`-nopoc` 禁用 POC 时仍输出错误日志** — 已修复
- **`-debug` 日志文件写入失败** — `applyLogLevel` 重建 Logger 时丢失 `DebugLogFile` 配置
- **`-hash` 不支持 LM:NT 格式** — 现在支持 `aad3b435b51404ee:31d6cfe0d16ae931...` 标准格式
- **`-pwd` 不支持逗号分隔多个密码** — 现在 `-pwd "123,456,root"` 正确拆分
- **`-nobr` 跳过了 Redis 未授权检测** — 未授权是服务探测不是爆破,不受 `-nobr` 影响
- **非终端输出时 ANSI 控制码覆盖扫描结果** — 管道/重定向时自动禁用进度条和颜色
- **静默模式 NDJSON banner 过长** — Redis INFO ~5KB 截断至 200 字符
- **CSV 漏洞 Type 列为空** — 补全 `type` 字段
- **SNMP 探测成功但终端无输出** — 补充 `session.LogVuln` 调用
#### 其他
- **service_probe 连接丢失后静默成功** — `Write`/`Read``Conn=nil` 时返回明确错误
- **MongoDB readMongoMsg 未设置读超时**
- **TXTWriter.Close Sync 失败后未关闭文件**
- **MySQL 3306 服务名误识别为 genetec-5400** — nmap 指纹库误匹配,通过 banner 特征校正
- **gmtls stdout 竞态** — 移除 `os.Stdout` 非同步重定向
---
### 🏗️ 架构优化
- **统一服务缓存** — `webServiceCache` 扩展为通用 `serviceCache`,下沉到 per-session State,消除多实例缓存串台
- **CEL 表达式编译缓存** — 同一 POC 的所有规则共享编译后的 Program
- **POC 全局状态消除** — `allPocs` 全局变量改为 `pocStore` 按 PocPath 缓存,并发场景不再互相覆盖
- **进度条竞态修复** — `isActive` 改为 `atomic.Bool`
- **Lint 全量修复** — cassandra/ipmi/mongodb/webscan 的 ineffassign、unused、errcheck
---
## 反馈与贡献
- 🐛 发现 Bug → [提交 Bug 报告](https://github.com/shadow1ng/fscan/issues/new?template=bug_report.yml)
- 🎯 结果不准 → [提交误报/漏报](https://github.com/shadow1ng/fscan/issues/new?template=false_positive.yml)
- ✨ 功能建议 → [提交功能请求](https://github.com/shadow1ng/fscan/issues/new?template=feature_request.yml)
- 💬 使用疑问 → [Discussions](https://github.com/shadow1ng/fscan/discussions)
+106
View File
@@ -0,0 +1,106 @@
# fscan v2.2.0-rc
> ⚠️ **这是预发布版本 (Release Candidate)**,可能存在未发现的问题。
> 如果你在使用中遇到任何异常,请积极通过 [Issue](https://github.com/shadow1ng/fscan/issues/new/choose) 反馈,帮助我们尽快稳定正式版。
> 生产环境建议继续使用 [v2.1.3](https://github.com/shadow1ng/fscan/releases/tag/v2.1.3)。
---
## 与 v2.1.3 的主要变更
### 🏗️ 架构升级:嵌入式 Scanner SDK
新增 `pkg/fscan` 包,fscan 从纯 CLI 工具进化为**可嵌入的扫描引擎**。
- 支持从其他 Go 程序直接调用 fscan 扫描能力
- 每个 Scanner 实例拥有独立的 config/state/session
- 支持多实例并发扫描,日志和结果完全隔离
- 全局状态已完整迁移到 session 实例
### 🚀 大规模扫描支持
- **流式 Host Iterator** — /8 级别 CIDR 不再一次性展开到内存
- **移除 MaxHosts 硬限制** — 大网段不再被静默截断
- **ARM 原子计数器对齐** — 修复 32 位 ARM 平台 panic
### 🔌 新增协议插件(+9
| 插件 | 用途 |
|------|------|
| IMAP / POP3 | 邮件服务器检测 |
| JDWP | Java Debug 端口检测 |
| NFS / RMI | 文件共享 / Java 远程调用 |
| IPMI | 服务器带外管理 |
| SNMP (UDP) | 网络设备管理 |
| DNS / BACnet / Modbus | DNS 服务 / 工控协议 |
全部为**原生协议实现**,零外部依赖。
### ⚡ 性能与体积优化
- Kafka/MongoDB/Cassandra 从重型客户端库改为 raw TCP 实现,**二进制体积减小**
- 热路径零分配、自适应连接池 CAS 无锁化
- Scanner 客户端指纹收敛(降低被检测概率)
### 🌐 网络能力增强
- **SOCKS5 代理认证**支持(用户名/密码)
- **国密 TLS**SM2/SM3/SM4)网站扫描
- 支持 `-add-password` 追加多个自定义密码
### 🔧 本地后渗透插件
- 新增 sshkey、4 种持久化插件(crontask/systemd/winregistry/winschtask
- minidump 降级链凭据提取 + 杀软前置检测
- cleaner 清理所有持久化痕迹
- systeminfo 整合合并(fileinfo/dcinfo/avdetect/envinfo
### 📦 工程改进
- 目录重命名:`WebScan/``webscan/``mylib/``libs/`
- i18n 覆盖扩展(中/英双语)
- 全量 lint/errcheck 修复
- 补充大量单元测试
---
## ⚠️ 已知问题 / 注意事项
| 问题 | 说明 |
|------|------|
| SDK 并发限速共享 | `CanSendPacket()` 全局限速器在多 Scanner 实例间共享(按进程限速,非按实例) |
| CEL 回连验证走全局 | DNSLog 反连检测的 HTTP 请求计数走全局 state(影响可忽略) |
| 本地插件仅限授权场景 | 持久化/minidump/shell 类插件仅用于授权渗透测试 |
| WebUI 为实验性功能 | `fscan-web` 版本的 Web 界面尚未完善 |
| `common` 包未完全拆分 | 内部架构的 common 包职责仍较重,后续版本持续优化 |
---
## 反馈与贡献
这是一个 RC 版本,我们非常需要你的反馈:
- 🐛 发现 Bug → [提交 Bug 报告](https://github.com/shadow1ng/fscan/issues/new?template=bug_report.yml)
- 🎯 结果不准 → [提交误报/漏报](https://github.com/shadow1ng/fscan/issues/new?template=false_positive.yml)
- ✨ 功能建议 → [提交功能请求](https://github.com/shadow1ng/fscan/issues/new?template=feature_request.yml)
- 💬 使用疑问 → [Discussions](https://github.com/shadow1ng/fscan/discussions)
---
## 版本说明
| 版本 | 说明 |
|------|------|
| **fscan** | 标准版,包含全部插件(推荐) |
| **fscan-nolocal** | 精简版,不含本地后渗透模块(体积更小) |
| **fscan-web** | WebUI版,带 Web 管理界面(主流平台) |
## 平台支持
| 平台 | 架构 |
|------|------|
| Linux | x64, x32, arm64, armv5/6/7, mips, mips64, mipsle |
| Windows | x64, x32 |
| macOS | x64 (Intel), arm64 (Apple Silicon) |
| FreeBSD | x64, x32, arm64, armv5/6/7 |
| Solaris | X64n |
-81
View File
@@ -1,81 +0,0 @@
#!/bin/bash
# 构建 fscan-lite 并准备发布产物
set -e
VERSION="${1:-dev}"
LITE_DIR="fscan-lite"
OUTPUT_DIR="dist-lite"
echo "==> 构建 fscan-lite (版本: $VERSION)"
# 清理旧产物
rm -rf "$OUTPUT_DIR"
mkdir -p "$OUTPUT_DIR"
# 进入 lite 目录
cd "$LITE_DIR"
# 源文件
SOURCES="src/main.c src/scanner.c src/platform.c"
INCLUDE="-Iinclude"
CFLAGS_BASE="-std=c89 -Wall -O2"
# 构建 Linux 版本
echo "==> 构建 Linux 版本..."
# Linux x64
echo " - Linux x64"
mkdir -p bin
gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite $SOURCES -lpthread
cp bin/fscan-lite "../$OUTPUT_DIR/fscan-lite_${VERSION}_linux_x64"
rm -rf bin
# Linux x32
echo " - Linux x32"
mkdir -p bin
gcc $CFLAGS_BASE -m32 $INCLUDE -o bin/fscan-lite $SOURCES -lpthread 2>/dev/null || echo " (跳过: 缺少 32-bit 支持)"
if [ -f bin/fscan-lite ]; then
cp bin/fscan-lite "../$OUTPUT_DIR/fscan-lite_${VERSION}_linux_x32"
fi
rm -rf bin
# 构建 Windows 版本
echo "==> 构建 Windows 版本..."
# Windows x64
echo " - Windows x64"
mkdir -p bin
x86_64-w64-mingw32-gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite.exe $SOURCES -lws2_32 -static
if [ -f bin/fscan-lite.exe ]; then
cp bin/fscan-lite.exe "../$OUTPUT_DIR/fscan-lite_${VERSION}_windows_x64.exe"
echo " ✓ 编译成功"
else
echo " ✗ 编译失败"
fi
rm -rf bin
# Windows x32
echo " - Windows x32"
mkdir -p bin
i686-w64-mingw32-gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite.exe $SOURCES -lws2_32 -static
if [ -f bin/fscan-lite.exe ]; then
cp bin/fscan-lite.exe "../$OUTPUT_DIR/fscan-lite_${VERSION}_windows_x32.exe"
echo " ✓ 编译成功"
else
echo " ✗ 编译失败"
fi
rm -rf bin
cd ..
# 统计产物
echo ""
echo "==> 构建完成!"
echo "产物列表:"
if [ -d "$OUTPUT_DIR" ]; then
ls -lh "$OUTPUT_DIR" 2>/dev/null || echo " (无产物)"
echo ""
FILECOUNT=$(ls "$OUTPUT_DIR" 2>/dev/null | wc -l)
echo "总计: $FILECOUNT 个文件"
fi
+37 -7
View File
@@ -14,18 +14,17 @@ on:
description: '创建草稿发布'
type: boolean
default: false
prerelease:
description: '标记为预发布'
type: boolean
default: false
permissions:
contents: write
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
release:
runs-on: ubuntu-latest
timeout-minutes: 45
timeout-minutes: 90
steps:
- name: 检出代码
@@ -33,10 +32,41 @@ jobs:
with:
fetch-depth: 0
- name: 准备 Release Notes
if: ${{ !inputs.snapshot }}
run: |
TAG="${GITHUB_REF_NAME}"
NOTES_FILE=".github/release-notes/${TAG}.md"
if [ -f "$NOTES_FILE" ]; then
echo "📝 找到版本 Release Notes: $NOTES_FILE"
cp "$NOTES_FILE" /tmp/release-notes.md
else
echo "⚠️ 未找到 $NOTES_FILE,使用自动生成的 changelog"
echo "" > /tmp/release-notes.md
fi
echo "RELEASE_NOTES_FILE=/tmp/release-notes.md" >> $GITHUB_ENV
- name: 构建和发布
uses: ./.github/actions/build-release
with:
mode: ${{ inputs.snapshot && 'snapshot' || 'release' }}
go-version: '1.20'
go-version: '1.25'
retention-days: '90'
release-args: ${{ inputs.draft && '--draft' || '' }} ${{ inputs.prerelease && '--prerelease' || '' }}
release-args: ${{ inputs.draft && '--draft' || '' }}
- name: 更新 Release Notes
if: ${{ !inputs.snapshot && env.RELEASE_NOTES_FILE != '' }}
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${GITHUB_REF_NAME}"
NOTES_FILE="${RELEASE_NOTES_FILE}"
if [ -s "$NOTES_FILE" ]; then
echo "📝 更新 Release Notes..."
# 用版本特定的 notes 替换 goreleaser 生成的 body
gh release edit "$TAG" --notes-file "$NOTES_FILE"
echo "✅ Release Notes 已更新"
fi
+25 -34
View File
@@ -3,6 +3,7 @@ name: 测试构建
on:
push:
branches:
- main
- dev
- dev-*
- develop
@@ -37,6 +38,9 @@ on:
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
lint:
name: 代码检查
@@ -50,37 +54,27 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.23'
go-version: '1.25'
cache: true
- name: 运行 golangci-lint
run: |
# 下载 golangci-lint v2
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin latest
# 安装 golangci-lint v2 (需 Go 1.22+,仅用于静态分析,不影响二进制兼容性)
go install github.com/golangci/golangci-lint/v2/cmd/[email protected]
# 运行检查并灵活处理结果
set +e
golangci-lint run --timeout=5m > lint_output.txt 2>&1
LINT_EXIT_CODE=$?
# 运行检查
golangci-lint run --timeout=5m > lint_output.txt 2>&1 || true
cat lint_output.txt
set -e
# 只关注真正的 bug,忽略代码质量建议
# 过滤规则:
# - gocognit/gocyclo: 复杂度警告(阈值已在配置中设置)
# - QF/S/ST: staticcheck 的代码质量改进建议(非bug)
if [ $LINT_EXIT_CODE -ne 0 ]; then
CRITICAL_ISSUES=$(grep -E "\.go:[0-9]+:[0-9]+:" lint_output.txt | grep -v "gocognit" | grep -v "gocyclo" | grep -v "QF[0-9]" | grep -v " S[0-9]" | grep -v "ST[0-9]" || true)
if [ -n "$CRITICAL_ISSUES" ]; then
echo "❌ Linting failed with critical issues:"
echo "$CRITICAL_ISSUES" | head -20
exit 1
else
echo "⚠️ Only quality suggestions - PASSING"
exit 0
fi
# 过滤非关键问题(复杂度建议和 staticcheck 风格建议不阻塞 CI
CRITICAL=$(grep -E "\.go:[0-9]+:[0-9]+:" lint_output.txt \
| grep -vE "gocognit|gocyclo|QF[0-9]+| S[0-9]+|ST[0-9]+" || true)
if [ -n "$CRITICAL" ]; then
echo "❌ Linting failed with critical issues:"
echo "$CRITICAL" | head -20
exit 1
fi
echo "✅ No lint issues found"
echo "✅ Lint 检查通过"
- name: 检查代码复杂度(质量门禁)
run: |
@@ -120,7 +114,7 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.20'
go-version: '1.25'
cache: true
- name: 下载依赖
@@ -131,7 +125,8 @@ jobs:
- name: 运行测试
run: |
# 排除第三方grdp库测试(存在环境依赖问题)
go test -vet=off -race -coverprofile=coverage.out -covermode=atomic $(go list ./... | grep -v '/mylib/grdp/')
PKGS=$(go list ./... | grep -v '/libs/grdp/')
go test -vet=off -race -coverprofile=coverage.out -covermode=atomic $PKGS
- name: 上传覆盖率
uses: actions/upload-artifact@v4
@@ -152,18 +147,14 @@ jobs:
echo "### 🚦 覆盖率质量门禁" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
# 提取总体覆盖率
TOTAL_COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
# 提取总体覆盖率awk 整数比较,不需要 bc
TOTAL_COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print int($3)}')
echo "总体覆盖率: ${TOTAL_COVERAGE}%" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
# 检查核心模块覆盖率(core, common/parsers必须>50%
CORE_COVERAGE=$(go tool cover -func=coverage.out | grep "^github.com/shadow1ng/fscan/core/" | grep -v "_test.go" | awk '{sum+=$3; count++} END {if(count>0) print sum/count; else print 0}')
PARSERS_COVERAGE=$(go tool cover -func=coverage.out | grep "^github.com/shadow1ng/fscan/common/parsers/" | grep -v "_test.go" | awk '{sum+=$3; count++} END {if(count>0) print sum/count; else print 0}')
# 警告阈值:总体<40%, 核心模块<50%
if (( $(echo "$TOTAL_COVERAGE < 40" | bc -l) )); then
# 警告阈值:总体<40%
if [ "$TOTAL_COVERAGE" -lt 40 ]; then
echo "⚠️ **警告**: 总体覆盖率 ${TOTAL_COVERAGE}% < 40%,建议补充测试" >> $GITHUB_STEP_SUMMARY
fi
@@ -190,7 +181,7 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.20'
go-version: '1.25'
cache: true
- name: 构建验证
+5 -2
View File
@@ -1,9 +1,9 @@
result.txt
result.json
main
/main
.idea
fscan.exe
fscan
/fscan
fscanapi.csv
# IDE files / IDE 文件
@@ -58,6 +58,9 @@ bin/
*.dll
*.so
*.dylib
/fscan_cli
/fscan_web
/embed-agent
# Web UI build / Web前端构建
web-ui/node_modules/
+1 -1
View File
@@ -45,7 +45,7 @@ linters:
- vendor
- testdocker
- image
- mylib/grdp
- libs/grdp
issues:
max-issues-per-linter: 50
+1 -1
View File
@@ -64,7 +64,7 @@ build-web: build-ui
@echo "$(BLUE)构建Web版本...$(NC)"
$(GO) build -tags web -ldflags="-s -w" -trimpath -o $(BINARY_NAME)-web .
@echo "$(GREEN)✓ 构建完成: $(BINARY_NAME)-web$(NC)"
@echo "$(BLUE)提示: 运行 ./$(BINARY_NAME)-web -web 启动Web界面$(NC)"
@echo "$(BLUE)提示: 运行 ./$(BINARY_NAME)-web 启动Web界面(默认端口 10240$(NC)"
## build-ui: 构建前端(需要Node.js和npm
build-ui:
+13 -3
View File
@@ -4,7 +4,7 @@
内网综合扫描工具,一键自动化漏扫。
**版本**: 2.1.2
**版本**: 2.2.0-rc.1
## 功能特性
@@ -52,6 +52,7 @@
- **并发控制** - 端口扫描线程、服务扫描线程独立配置
### 扩展功能
- **SDK嵌入** - `pkg/fscan`提供Go SDK,可嵌入Agent或安全平台,支持任务控制(Pause/Resume)、实时进度回调、TaskID追溯
- **Web管理界面** - 可视化扫描任务管理(条件编译 -tags web)
- **Lab靶场环境** - 内置Docker靶场用于测试学习
- **插件化架构** - 服务插件/Web插件/本地插件分离,易于扩展
@@ -185,10 +186,10 @@
```bash
# 标准编译
go build -ldflags="-s -w" -trimpath -o fscan main.go
go build -ldflags="-s -w" -trimpath -o fscan .
# 带Web管理界面
go build -tags web -ldflags="-s -w" -trimpath -o fscan main.go
go build -tags web -ldflags="-s -w" -trimpath -o fscan-web .
```
## 安装
@@ -211,9 +212,13 @@ yay -S fscan-git
`fscan.exe -h 192.168.x.x -m ssh -user root -pwd password`
![](image/3.png)
`fscan.exe -h 192.168.x.x -m ssh -user root -pwda pass1 pass2 pass3` (追加多个密码)
`fscan.exe -h 192.168.x.x -p80 -proxy http://127.0.0.1:8080`
![](image/2020-12-12-13-34-44.png)
`fscan.exe -h 192.168.x.x -socks5 socks5://user:[email protected]:1080` (SOCKS5认证代理)
`fscan.exe -h 192.168.x.x -p 139 -m netbios`
![](image/netbios.png)
@@ -234,6 +239,11 @@ yay -S fscan-git
- **后两周** - Bug修复与代码整合
- **欢迎PR** - 期待您的贡献!
### SDK & Agent 集成
- 扩展SDK能力,完善端侧Agent嵌入支持
- 断点续扫、带宽级限速、内存水位控制
- 更多Agent场景的集成示例
### 插件生态
- 持续扩展服务插件覆盖范围
- 为每个服务插件开发更多漏洞检测和利用能力
+7 -3
View File
@@ -4,7 +4,7 @@
Comprehensive intranet scanning tool for automated vulnerability assessment.
**Version**: 2.1.2
**Version**: 2.2.0-rc.1.1
## Features
@@ -185,10 +185,10 @@ Comprehensive intranet scanning tool for automated vulnerability assessment.
```bash
# Standard build
go build -ldflags="-s -w" -trimpath -o fscan main.go
go build -ldflags="-s -w" -trimpath -o fscan .
# With Web UI
go build -tags web -ldflags="-s -w" -trimpath -o fscan main.go
go build -tags web -ldflags="-s -w" -trimpath -o fscan-web .
```
## Install
@@ -211,9 +211,13 @@ yay -S fscan-git
`fscan.exe -h 192.168.x.x -m ssh -user root -pwd password`
![](image/3.png)
`fscan.exe -h 192.168.x.x -m ssh -user root -pwda pass1 pass2 pass3` (add multiple passwords)
`fscan.exe -h 192.168.x.x -p80 -proxy http://127.0.0.1:8080`
![](image/2020-12-12-13-34-44.png)
`fscan.exe -h 192.168.x.x -socks5 socks5://user:[email protected]:1080` (SOCKS5 proxy with authentication)
`fscan.exe -h 192.168.x.x -p 139 -m netbios`
![](image/netbios.png)
+304
View File
@@ -0,0 +1,304 @@
---
name: fscan-agent
description: 使用 fscan 进行网络扫描和安全评估。当用户要求扫描网段、探测主机存活、发现开放端口、识别服务、检测漏洞或弱口令时使用。支持 NDJSON 结构化输出,适合 AI agent 管道消费。
argument-hint: <目标IP/网段> [附加参数]
allowed-tools: Bash, Read, Agent
---
# Fscan AI Agent Skill
## 工具概述
Fscan 是一款内网综合扫描工具,功能包括:
- 主机存活探测(ICMP / TCP
- 端口扫描与服务识别
- 漏洞检测(MS17-010、Redis 未授权等)
- 弱口令爆破(SSH、SMB、MySQL、MSSQL、FTP、RDP 等)
- Web 指纹识别与 POC 扫描
- NetBIOS / SMB 信息收集
- 本地信息收集(杀软检测、系统信息等)
二进制路径:当前项目编译产物 `fscan_cli`,或系统 PATH 中的 `fscan`
## 调用格式
```bash
# AI agent 标准用法:NDJSON 输出,无人类日志干扰
fscan -h <目标> -silent [其他参数]
# 解析输出
fscan -h 192.168.1.0/24 -silent | jq 'select(.type=="VULN")'
```
## 核心参数
### 目标指定
| 参数 | 说明 | 示例 |
|------|------|------|
| `-h` | 目标主机(IP / CIDR / 范围) | `-h 192.168.1.0/24` `-h 10.0.0.1-10.0.0.100` |
| `-hf` | 从文件读取目标 | `-hf targets.txt` |
| `-p` | 指定端口(逗号/范围) | `-p 22,80,443,445,3306` `-p 1-1000` |
| `-ep` | 排除端口 | `-ep 25,110` |
| `-eh` | 排除主机 | `-eh 192.168.1.1` |
| `-u` | 指定 URLWeb 扫描) | `-u https://example.com` |
| `-uf` | URL 文件 | `-uf urls.txt` |
### 扫描控制
| 参数 | 说明 | 默认值 |
|------|------|--------|
| `-m` | 扫描模式 | `all` |
| `-t` | 端口扫描线程数 | `600` |
| `-mt` | 模块线程数 | `20` |
| `-time` | 连接超时(秒) | `3` |
| `-gt` | 全局超时(秒) | `180` |
| `-np` | 跳过存活检测 | `false` |
| `-ntp` | 禁用 TCP 补充探测 | `false` |
| `-ao` | 仅存活检测 | `false` |
| `-nobr` | 禁用暴力破解 | `false` |
| `-full` | 全量 POC 扫描 | `false` |
| `-max-retries` | 最大重试次数 | `1` |
### 认证
| 参数 | 说明 |
|------|------|
| `-user` | 用户名 |
| `-pwd` | 密码 |
| `-usera` | 追加用户名 |
| `-pwda` | 追加密码,支持逗号或空格分隔多个值 |
| `-userf` | 用户名字典文件 |
| `-pwdf` | 密码字典文件 |
| `-domain` | 域名(SMB/WMI |
| `-sshkey` | SSH 私钥文件 |
| `-hash` / `-hashf` | NTLM Hash / Hash 文件 |
### 代理
| 参数 | 说明 |
|------|------|
| `-socks5` | SOCKS5 代理 (`127.0.0.1:1080`) |
| `-proxy` | HTTP 代理 (`http://127.0.0.1:8080`) |
| `-iface` | 指定本地网卡 IP(VPN 场景) |
### 输出
| 参数 | 说明 |
|------|------|
| `-silent` | 静默模式:stdout 仅输出 NDJSON |
| `-o` | 输出文件路径(默认 `result.txt` |
| `-f` | 输出格式:`txt` / `json` / `csv` |
| `-no` | 禁用文件保存 |
| `-debug` | 调试模式:日志写入 `fscan_debug.log` |
| `-log` | 日志级别(`debug` / `info` / `base` / `error` |
### 扫描模式 `-m` 的取值
| 值 | 说明 |
|------|------|
| `all` | 全部扫描(默认) |
| `icmp` | 仅 ICMP 存活检测 |
| 插件名 | 仅运行指定插件(如 `ssh``smb``ms17010``webtitle` |
## 服务插件列表
| 插件 | 默认端口 | 功能 |
|------|----------|------|
| `ftp` | 21 | FTP 弱口令 |
| `ssh` | 22 | SSH 弱口令 |
| `telnet` | 23 | Telnet 弱口令 |
| `smtp` | 25 | SMTP 弱口令 |
| `findnet` | 135 | RPC 网络信息发现(NetInfo |
| `netbios` | 139 | NetBIOS 信息收集 |
| `smb` | 445 | SMB 弱口令 |
| `ms17010` | 445 | MS17-010 永恒之蓝检测 |
| `ldap` | 389 | LDAP 弱口令 |
| `mssql` | 1433 | MSSQL 弱口令 |
| `oracle` | 1521 | Oracle 弱口令 |
| `mysql` | 3306 | MySQL 弱口令 |
| `rdp` | 3389 | RDP 弱口令 + 系统信息 |
| `postgresql` | 5432 | PostgreSQL 弱口令 |
| `vnc` | 5900 | VNC 弱口令 |
| `redis` | 6379 | Redis 未授权 + 弱口令 |
| `elasticsearch` | 9200 | ES 未授权 |
| `mongodb` | 27017 | MongoDB 未授权 + 弱口令 |
| `memcached` | 11211 | Memcached 未授权 |
| `kafka` | 9092 | Kafka 未授权 |
| `activemq` | 61616 | ActiveMQ 弱口令 |
| `rabbitmq` | 5672 | RabbitMQ 弱口令 |
| `cassandra` | 9042 | Cassandra 弱口令 |
| `neo4j` | 7687 | Neo4j 弱口令 |
| `rsync` | 873 | Rsync 未授权 |
| `webtitle` | 80/443 | Web 标题 + 指纹识别 |
| `webpoc` | 80/443 | Web 漏洞 POC |
## 本地插件(`-local`
```bash
fscan -local avdetect # 杀软检测
fscan -local systeminfo # 系统信息收集
fscan -local envinfo # 环境变量信息
fscan -local dcinfo # 域控信息
fscan -local fileinfo # 敏感文件搜索
```
## NDJSON 输出 Schema`-silent` 模式)
每行一个 JSON 对象,所有字段定义:
| 字段 | 类型 | 出现条件 | 说明 |
|------|------|----------|------|
| `type` | string | 必有 | `HOST` / `PORT` / `SERVICE` / `VULN` |
| `target` | string | 必有 | 原始目标 `host``host:port` |
| `status` | string | 必有 | 状态描述 |
| `host` | string | 必有 | IP 地址 |
| `port` | int | PORT/SERVICE/VULN | 端口号 |
| `service` | string | SERVICE/VULN | 服务名(ssh, smb, http 等) |
| `protocol` | string | HOST/SERVICE | 协议(ICMP, TCP, http, https |
| `banner` | string | SERVICE | 服务 Banner |
| `title` | string | SERVICE (web) | 网页标题 |
| `url` | string | SERVICE (web) | 完整 URL |
| `vulnerability` | string | VULN | 漏洞名称 |
| `username` | string | VULN (弱口令) | 用户名 |
| `password` | string | VULN (弱口令) | 密码 |
| `plugin` | string | SERVICE/VULN | 产生结果的插件名 |
| `version` | string | SERVICE | 服务版本号 |
| `os` | string | SERVICE | 操作系统信息 |
### 输出示例
```jsonl
{"type":"HOST","target":"192.168.1.5","status":"alive","host":"192.168.1.5","protocol":"ICMP"}
{"type":"PORT","target":"192.168.1.5","status":"open","host":"192.168.1.5","port":22}
{"type":"PORT","target":"192.168.1.5","status":"open","host":"192.168.1.5","port":445}
{"type":"SERVICE","target":"192.168.1.5:22","status":"identified","host":"192.168.1.5","port":22,"service":"ssh","banner":"SSH-2.0-OpenSSH_8.9p1","version":"8.9p1","plugin":"portscan"}
{"type":"SERVICE","target":"192.168.1.5:80","status":"web","host":"192.168.1.5","port":80,"service":"http","protocol":"http","url":"http://192.168.1.5:80","title":"Welcome","plugin":"webtitle"}
{"type":"VULN","target":"192.168.1.5:445","status":"MS17-010 (Windows Server 2012 R2 Standard 9600)","host":"192.168.1.5","port":445,"vulnerability":"MS17-010","service":"smb","plugin":"ms17010"}
{"type":"VULN","target":"192.168.1.5:22","status":"weak_credential: root:123456","host":"192.168.1.5","port":22,"service":"ssh","username":"root","password":"123456","plugin":"ssh"}
{"type":"VULN","target":"192.168.1.5:6379","status":"Redis unauthorized","host":"192.168.1.5","port":6379,"vulnerability":"Redis unauthorized access","service":"redis","plugin":"redis"}
```
### 结果产出顺序
1. `HOST` — 存活探测阶段
2. `PORT` — 端口扫描阶段(与 SERVICE 可能交错)
3. `SERVICE` — 服务识别阶段
4. `VULN` — 漏洞/弱口令检测阶段
同一 `host:port` 可产生多条结果(PORT + SERVICE + VULN)。
## 常用场景参数组合
### 全网段快速扫描
```bash
fscan -h 192.168.1.0/24 -silent
```
### 跳过存活检测直接扫端口(目标明确时)
```bash
fscan -h 192.168.1.0/24 -silent -np
```
### 指定端口精确扫描
```bash
fscan -h 10.0.0.0/24 -silent -p 22,80,443,445,3389,3306,6379
```
### 仅存活探测
```bash
fscan -h 172.16.0.0/16 -silent -m icmp
```
### 低速隐蔽扫描
```bash
fscan -h 192.168.1.0/24 -silent -t 30 -time 5
```
### 通过 SOCKS5 代理扫描内网
```bash
fscan -h 10.0.0.0/24 -silent -socks5 127.0.0.1:1080
```
### 仅做弱口令检测
```bash
fscan -h 192.168.1.10 -silent -m ssh -user root -pwdf /path/to/passwords.txt
```
### Web 目标扫描
```bash
fscan -u https://target.com -silent -full
```
### 多目标文件批量扫描
```bash
fscan -hf targets.txt -silent -o results.json -f json
```
### 带调试日志的排障扫描
```bash
# NDJSON 到 stdoutdebug 日志到文件,互不干扰
fscan -h 192.168.1.0/24 -silent -debug
# 事后查看:cat fscan_debug.log
```
## AI Agent 结果处理
### Python 管道消费
```python
import json, subprocess
proc = subprocess.Popen(
["fscan", "-h", "192.168.1.0/24", "-silent"],
stdout=subprocess.PIPE, text=True
)
hosts, services, vulns = [], [], []
for line in proc.stdout:
r = json.loads(line)
if r["type"] == "HOST":
hosts.append(r["host"])
elif r["type"] == "SERVICE":
services.append(r)
elif r["type"] == "VULN":
vulns.append(r)
proc.wait()
```
### jq 过滤
```bash
# 提取所有弱口令
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.username != null) | "\(.host):\(.port) \(.service) \(.username):\(.password)"'
# 提取所有漏洞
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.type=="VULN") | "\(.host):\(.port) \(.vulnerability)"'
# 提取 Web 服务
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.url != null) | "\(.url) \(.title)"'
# 统计开放端口
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.type=="PORT") | .port' | sort -n | uniq -c | sort -rn
```
## 注意事项
- `-silent` 抑制所有人类可读日志,stdout 仅输出 NDJSON
- 空字段不出现在 JSON 中(`omitempty`
- 进程退出码 `0` 正常完成,非 `0` 表示参数错误或初始化失败
- `-silent``-debug` 可同时使用,互不干扰
- SOCKS5 代理下 fscan 信任协议层连接结果,不做额外深度验证
- 扫描大网段时线程数会自动调整,资源耗尽时自适应降级
- 默认超时 3 秒,防火墙 drop 的端口会静默超时,不计入失败率
+2 -2
View File
@@ -36,7 +36,7 @@ var (
"6000,6379,6380,6443,6666,6667," +
// Web/中间件 (7001-9999)
// 注意:9100 已移除,该端口为打印机 RAW 端口,发送数据会触发打印
"7001,7002,7474,7687,8000,8005,8008,8009,8080,8081,8086,8088,8089,8090,8161,8180,8443,8500,8834,8848,8880,8888,9000,9001,9042,9080,9090,9092,9093,9160,9200,9300,9418,9443,9999," +
"7001,7002,7474,7687,8000,8005,8008,8009,8080,8081,8086,8088,8089,8090,8161,8180,8443,8500,8834,8848,8880,8883,8888,9000,9001,9042,9080,9090,9092,9093,9160,9200,9300,9418,9443,9999," +
// 管理/监控 (10000-11211)
"10000,10051,10250,10255,11211," +
// 消息队列/集群 (15672-27018)
@@ -48,7 +48,7 @@ var (
DbPorts = "1433,1521,3306,5432,5672,5984,6379,7687,8086,9042,9093,9160,9200,11211,26379,27017,27018,61616"
// ServicePorts 服务端口
ServicePorts = "21,22,23,25,53,110,111,135,139,143,161,389,445,465,502,512,513,514,587,623,636,873,993,995,1433,1521,2049,2181,2222,3306,3389,5432,5672,5671,5900,5985,5986,6379,8161,8443,9000,9092,9093,9200,10051,11211,15672,15671,27017,61616,61613"
ServicePorts = "21,22,23,25,53,110,111,135,139,143,161,389,445,465,502,512,513,514,587,623,636,873,993,995,1433,1521,1883,2049,2181,2222,3306,3389,5432,5672,5671,5900,5985,5986,6379,8161,8443,8883,9000,9092,9093,9200,10051,11211,15672,15671,27017,61616,61613"
// CommonPorts 常用端口
CommonPorts = "21,22,23,25,53,80,110,135,139,143,443,445,993,995,1723,3389,5060,5985,5986"
+103 -23
View File
@@ -4,10 +4,12 @@ import (
"encoding/hex"
"fmt"
"net"
"net/url"
"strconv"
"strings"
"github.com/shadow1ng/fscan/common/config"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/parsers"
)
@@ -28,12 +30,12 @@ func BuildConfig(fv *FlagVars, info *HostInfo) (*Config, *State, error) {
// 3. 解析凭据
if err := parseCredentials(fv, cfg); err != nil {
return nil, nil, fmt.Errorf("凭据解析失败: %w", err)
return nil, nil, fmt.Errorf("%s: %w", i18n.GetText("config_credentials_parse_failed"), err)
}
// 4. 解析目标(主机、端口、URL)
if err := parseTargets(fv, info, cfg, state); err != nil {
return nil, nil, fmt.Errorf("目标解析失败: %w", err)
return nil, nil, fmt.Errorf("%s: %w", i18n.GetText("config_targets_parse_failed"), err)
}
// 5. 应用日志级别
@@ -48,7 +50,10 @@ func BuildConfig(fv *FlagVars, info *HostInfo) (*Config, *State, error) {
func parseCredentials(fv *FlagVars, cfg *Config) error {
// 解析用户名
usernames := parseUsernames(fv)
usernames, err := parseUsernames(fv)
if err != nil {
return err
}
if len(usernames) > 0 {
for serviceName := range cfg.Credentials.Userdict {
cfg.Credentials.Userdict[serviceName] = usernames
@@ -56,7 +61,10 @@ func parseCredentials(fv *FlagVars, cfg *Config) error {
}
// 解析密码
passwords := parsePasswords(fv)
passwords, err := parsePasswords(fv)
if err != nil {
return err
}
if len(passwords) > 0 {
cfg.Credentials.Passwords = passwords
}
@@ -83,7 +91,7 @@ func parseCredentials(fv *FlagVars, cfg *Config) error {
return nil
}
func parseUsernames(fv *FlagVars) []string {
func parseUsernames(fv *FlagVars) ([]string, error) {
var usernames []string
// 命令行用户名
@@ -100,6 +108,8 @@ func parseUsernames(fv *FlagVars) []string {
if fv.UsersFile != "" {
if lines, err := parsers.ReadLinesFromFile(fv.UsersFile); err == nil {
usernames = append(usernames, lines...)
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_users_failed", fv.UsersFile, err))
}
}
@@ -113,30 +123,52 @@ func parseUsernames(fv *FlagVars) []string {
}
}
return removeDuplicate(usernames)
return removeDuplicate(usernames), nil
}
func parsePasswords(fv *FlagVars) []string {
func parsePasswords(fv *FlagVars) ([]string, error) {
var passwords []string
// 命令行密码
// 命令行密码(支持逗号分隔多个值,保留空格作为密码的一部分)
if fv.Password != "" {
passwords = append(passwords, strings.Split(fv.Password, ",")...)
for _, p := range strings.Split(fv.Password, ",") {
p = strings.TrimSpace(p)
if p != "" {
passwords = append(passwords, p)
}
}
}
// 从文件读取
if fv.PasswordsFile != "" {
if lines, err := parsers.ReadLinesFromFile(fv.PasswordsFile); err == nil {
passwords = append(passwords, lines...)
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_passwords_failed", fv.PasswordsFile, err))
}
}
// 额外密码
if fv.AddPasswords != "" {
passwords = append(passwords, strings.Split(fv.AddPasswords, ",")...)
passwords = append(passwords, splitCredentialValues(fv.AddPasswords)...)
}
return removeDuplicate(passwords)
return removeDuplicate(passwords), nil
}
func splitCredentialValues(input string) []string {
fields := strings.FieldsFunc(input, func(r rune) bool {
return r == ',' || r == ' ' || r == '\t' || r == '\n' || r == '\r'
})
values := make([]string, 0, len(fields))
for _, field := range fields {
field = strings.TrimSpace(field)
if field != "" {
values = append(values, field)
}
}
return values
}
func parseUserPassPairs(fv *FlagVars) ([]config.CredentialPair, error) {
@@ -145,6 +177,7 @@ func parseUserPassPairs(fv *FlagVars) ([]config.CredentialPair, error) {
// 如果命令行同时指定了单个用户名和单个密码(不是逗号分隔的多个)
if fv.Username != "" && fv.Password != "" &&
!strings.Contains(fv.Username, ",") && !strings.Contains(fv.Password, ",") &&
fv.AddUsers == "" && fv.AddPasswords == "" &&
fv.UsersFile == "" && fv.PasswordsFile == "" && fv.UserPassFile == "" {
pairs = append(pairs, config.CredentialPair{
Username: strings.TrimSpace(fv.Username),
@@ -169,15 +202,22 @@ func parseHashes(fv *FlagVars) ([]string, [][]byte, error) {
var hashValues []string
var hashBytes [][]byte
// 命令行哈希
// 命令行哈希(支持纯 NTLM 32字符 或 LM:NT 格式)
if fv.HashValue != "" {
hash := strings.TrimSpace(fv.HashValue)
if len(hash) == 32 {
hashValues = append(hashValues, hash)
if hashByte, err := hex.DecodeString(hash); err == nil {
hashBytes = append(hashBytes, hashByte)
}
// LM:NT 格式取 NT hash 部分
if parts := strings.SplitN(hash, ":", 2); len(parts) == 2 && len(parts[1]) == 32 {
hash = parts[1]
}
if len(hash) != 32 {
return nil, nil, fmt.Errorf("invalid hash length: %s", fv.HashValue)
}
hashByte, err := hex.DecodeString(hash)
if err != nil {
return nil, nil, err
}
hashValues = append(hashValues, hash)
hashBytes = append(hashBytes, hashByte)
}
// 从文件读取
@@ -205,13 +245,17 @@ func parseTargets(fv *FlagVars, info *HostInfo, cfg *Config, state *State) error
if port, portErr := strconv.Atoi(portStr); portErr == nil && port >= 1 && port <= 65535 {
// 有效的 host:port 格式
state.SetHostPorts([]string{info.Host})
info.Host = ""
ports = "" // 清空端口,避免双重扫描
}
}
}
// 解析 URL
urls := parseURLs(fv)
urls, err := parseURLs(fv)
if err != nil {
return err
}
if len(urls) > 0 {
state.SetURLs(urls)
if info.URL == "" && len(urls) == 1 {
@@ -227,7 +271,7 @@ func parseTargets(fv *FlagVars, info *HostInfo, cfg *Config, state *State) error
return nil
}
func parseURLs(fv *FlagVars) []string {
func parseURLs(fv *FlagVars) ([]string, error) {
var urls []string
// 命令行 URL
@@ -246,10 +290,12 @@ func parseURLs(fv *FlagVars) []string {
for _, line := range lines {
urls = append(urls, normalizeURL(line))
}
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_urls_failed", fv.URLsFile, err))
}
}
return removeDuplicate(urls)
return removeDuplicate(urls), nil
}
func normalizeURL(rawURL string) string {
@@ -257,10 +303,44 @@ func normalizeURL(rawURL string) string {
if rawURL == "" {
return rawURL
}
if !strings.HasPrefix(rawURL, "http://") && !strings.HasPrefix(rawURL, "https://") {
return "http://" + rawURL
lowerURL := strings.ToLower(rawURL)
if !strings.HasPrefix(lowerURL, "http://") && !strings.HasPrefix(lowerURL, "https://") {
return "http://" + normalizeSchemelessURLTarget(rawURL)
}
return rawURL
parsed, err := url.Parse(rawURL)
if err != nil || parsed.Host == "" {
return rawURL
}
normalizedHost := normalizeURLHost(parsed.Host)
if normalizedHost == parsed.Host {
return rawURL
}
parsed.Host = normalizedHost
normalized := parsed.String()
if schemeEnd := strings.Index(rawURL, "://"); schemeEnd >= 0 {
return rawURL[:schemeEnd] + normalized[len(parsed.Scheme):]
}
return normalized
}
func normalizeSchemelessURLTarget(rawURL string) string {
authority := rawURL
suffix := ""
if idx := strings.IndexAny(rawURL, "/?#"); idx >= 0 {
authority = rawURL[:idx]
suffix = rawURL[idx:]
}
return normalizeURLHost(authority) + suffix
}
func normalizeURLHost(host string) string {
if strings.HasPrefix(host, "[") {
return host
}
if ip := net.ParseIP(host); ip != nil && strings.Contains(host, ":") {
return "[" + host + "]"
}
return host
}
// =============================================================================
+194
View File
@@ -0,0 +1,194 @@
package common
import (
"reflect"
"testing"
fscanconfig "github.com/shadow1ng/fscan/common/config"
)
func TestParsePasswordsKeepsPrimaryPasswordLiteral(t *testing.T) {
fv := &FlagVars{
Password: "root admin,pass0",
AddPasswords: "pass1 pass2,pass3\tpass4",
}
got, err := parsePasswords(fv)
if err != nil {
t.Fatalf("parsePasswords error = %v", err)
}
// -pwd 逗号分隔,空格保留;-pwda 逗号/空格/tab 分隔
want := []string{"root admin", "pass0", "pass1", "pass2", "pass3", "pass4"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("parsePasswords() = %#v, want %#v", got, want)
}
}
func TestBuildConfigReturnsUserFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{UsersFile: "missing-users-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing users file")
}
}
func TestBuildConfigReturnsPasswordFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{PasswordsFile: "missing-passwords-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing passwords file")
}
}
func TestBuildConfigReturnsURLFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{URLsFile: "missing-urls-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing urls file")
}
}
func TestBuildConfigRejectsInvalidHashValue(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{HashValue: "not-md5"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for invalid hash value")
}
}
func TestBuildConfigDefaultsAreIndependentCopies(t *testing.T) {
cfg, _, err := BuildConfig(&FlagVars{Username: "custom-user"}, &HostInfo{})
if err != nil {
t.Fatalf("BuildConfig error = %v", err)
}
defaultSSHUsers := fscanconfig.DefaultUserDict["ssh"]
if len(defaultSSHUsers) == 1 && defaultSSHUsers[0] == "custom-user" {
t.Fatal("BuildConfig mutated DefaultUserDict")
}
cfg.Credentials.Userdict["ssh"][0] = "mutated-user"
if fscanconfig.DefaultUserDict["ssh"][0] == "mutated-user" {
t.Fatal("Config userdict shares backing storage with DefaultUserDict")
}
cfg.Credentials.Passwords[0] = "mutated-password"
if fscanconfig.DefaultPasswords[0] == "mutated-password" {
t.Fatal("Config passwords share backing storage with DefaultPasswords")
}
port := 80
cfg.PortMap[port][0] = "mutated-probe"
if fscanconfig.DefaultPortMap[port][0] == "mutated-probe" {
t.Fatal("Config port map shares backing storage with DefaultPortMap")
}
cfg.DefaultMap[0] = "mutated-default-probe"
if fscanconfig.DefaultProbeMap[0] == "mutated-default-probe" {
t.Fatal("Config default map shares backing storage with DefaultProbeMap")
}
}
func TestParseUserPassPairsKeepsAdditionalCredentialFlags(t *testing.T) {
tests := []struct {
name string
fv *FlagVars
}{
{
name: "additional passwords",
fv: &FlagVars{
Username: "root",
Password: "primary",
AddPasswords: "extra",
},
},
{
name: "additional users",
fv: &FlagVars{
Username: "root",
Password: "primary",
AddUsers: "admin",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pairs, err := parseUserPassPairs(tt.fv)
if err != nil {
t.Fatalf("parseUserPassPairs error = %v", err)
}
if len(pairs) != 0 {
t.Fatalf("parseUserPassPairs returned exact pairs %#v; additional credential flags would be ignored", pairs)
}
})
}
}
func TestNewConfigDefaultsAreIndependentCopies(t *testing.T) {
cfg := NewConfig()
cfg.Credentials.Userdict["ssh"][0] = "mutated-user"
if fscanconfig.DefaultUserDict["ssh"][0] == "mutated-user" {
t.Fatal("NewConfig userdict shares backing storage with DefaultUserDict")
}
cfg.Credentials.Passwords[0] = "mutated-password"
if fscanconfig.DefaultPasswords[0] == "mutated-password" {
t.Fatal("NewConfig passwords share backing storage with DefaultPasswords")
}
port := 80
cfg.PortMap[port][0] = "mutated-probe"
if fscanconfig.DefaultPortMap[port][0] == "mutated-probe" {
t.Fatal("NewConfig port map shares backing storage with DefaultPortMap")
}
cfg.DefaultMap[0] = "mutated-default-probe"
if fscanconfig.DefaultProbeMap[0] == "mutated-default-probe" {
t.Fatal("NewConfig default map shares backing storage with DefaultProbeMap")
}
}
func TestParseTargetsHostPortDoesNotLeaveSyntheticHost(t *testing.T) {
fv := &FlagVars{Ports: "22"}
info := &HostInfo{Host: "127.0.0.1:8080"}
cfg := BuildConfigFromFlags(fv)
state := NewState()
if err := parseTargets(fv, info, cfg, state); err != nil {
t.Fatalf("parseTargets error = %v", err)
}
if info.Host != "" {
t.Fatalf("info.Host = %q, want empty after host:port extraction", info.Host)
}
if got := state.GetHostPorts(); !reflect.DeepEqual(got, []string{"127.0.0.1:8080"}) {
t.Fatalf("hostPorts = %#v, want host:port target", got)
}
}
func TestNormalizeURLKeepsUppercaseScheme(t *testing.T) {
got := normalizeURL("HTTPS://example.com")
if got != "HTTPS://example.com" {
t.Fatalf("normalizeURL() = %q", got)
}
}
func TestNormalizeURLBracketsIPv6Literals(t *testing.T) {
tests := []struct {
name string
in string
want string
}{
{name: "bare ipv6 without scheme", in: "2001:db8::1", want: "http://[2001:db8::1]"},
{name: "bracketed ipv6 without scheme", in: "[2001:db8::1]", want: "http://[2001:db8::1]"},
{name: "bare ipv6 with scheme", in: "http://2001:db8::1", want: "http://[2001:db8::1]"},
{name: "bare ipv6 path without scheme", in: "2001:db8::1/admin", want: "http://[2001:db8::1]/admin"},
{name: "bare ipv6 query without scheme", in: "2001:db8::1?debug=1", want: "http://[2001:db8::1]?debug=1"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := normalizeURL(tt.in); got != tt.want {
t.Fatalf("normalizeURL(%q) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
+81 -27
View File
@@ -22,18 +22,25 @@ config_struct.go - 配置结构体定义
// Config 扫描器完整配置 - 初始化后只读,可安全共享
type Config struct {
// 高频访问字段 - 平铺到顶层
Timeout time.Duration // 通用超时
ThreadNum int // 主线程数
ModuleThreadNum int // 模块线程数
DisableBrute bool // 禁用暴力破解
DisablePing bool // 禁用Ping检测
Timeout time.Duration // 通用超时
TimeoutExplicit bool // 用户显式指定了 -time
ThreadNum int // 线程数
ThreadCeiling int // 线程数上限(自适应池允许的最大值)
ThreadNumExplicit bool // 用户显式指定了 -t
ModuleThreadNum int // 模块线程数
ModuleThreadNumExplicit bool // 用户显式指定了 -mt
DisableBrute bool // 禁用暴力破解
DisablePing bool // 禁用Ping检测
DisableTcpProbe bool // 禁用TCP补充探测
// 扫描模式
Mode string // 扫描模式
LocalMode bool // 本地模式
LocalPlugin string // 本地插件名
AliveOnly bool // 仅存活检测
MaxRetries int // 最大重试次数
Mode string // 扫描模式
LocalMode bool // 本地模式
LocalPlugin string // 本地插件名
AliveOnly bool // 仅存活检测
MaxRetries int // 最大重试次数
MaxRetriesExplicit bool // 用户显式指定了 -retry
DetectedNetworkEnv int // 探测到的网络环境(来自 core.NetworkEnv
// 高级功能(从AdvancedConfig合并)
Shellcode string // Shellcode
@@ -54,6 +61,9 @@ type Config struct {
LocalExploit LocalExploitConfig
Target TargetConfig // 扫描目标配置
// 全局超时
GlobalTimeout time.Duration
// SOCKS5代理端口配置
Socks5ProxyPort int // SOCKS5代理端口
}
@@ -79,14 +89,15 @@ type CredentialConfig struct {
// NetworkConfig 网络相关配置
type NetworkConfig struct {
HTTPProxy string
Socks5Proxy string
Iface string
WebTimeout time.Duration
MaxRedirects int
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
HTTPProxy string
Socks5Proxy string
Iface string
WebTimeout time.Duration
MaxRedirects int
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
ICMPRateExplicit bool
}
// OutputConfig 输出相关配置
@@ -105,11 +116,12 @@ type OutputConfig struct {
// POCConfig POC扫描相关配置
type POCConfig struct {
PocPath string // POC路径
PocName string // 指定POC名称
Full bool // 完整POC扫描
Num int // POC并发数
Disabled bool // 禁用POC扫描
PocPath string // POC路径
PocName string // 指定POC名称
Full bool // 完整POC扫描
Num int // POC并发数
NumExplicit bool // 用户显式指定了 -num
Disabled bool // 禁用POC扫描
}
// RedisConfig Redis利用相关配置
@@ -138,15 +150,57 @@ type LocalExploitConfig struct {
DownloadSavePath string // 下载保存路径
}
func cloneStringSlice(values []string) []string {
if values == nil {
return nil
}
return append([]string(nil), values...)
}
func cloneStringSliceMap(values map[string][]string) map[string][]string {
if values == nil {
return nil
}
cloned := make(map[string][]string, len(values))
for key, value := range values {
cloned[key] = cloneStringSlice(value)
}
return cloned
}
func clonePortMap(values map[int][]string) map[int][]string {
if values == nil {
return nil
}
cloned := make(map[int][]string, len(values))
for key, value := range values {
cloned[key] = cloneStringSlice(value)
}
return cloned
}
const minModuleTimeout = 3 * time.Second
// ModuleTimeout 返回插件级超时(用于弱口令测试、服务交互等多轮协议)
// 保证下限 3s,避免自适应把端口扫描超时压低后影响 SSH/SNMP 等交互型协议
func (c *Config) ModuleTimeout() time.Duration {
if c.Timeout >= minModuleTimeout {
return c.Timeout
}
return minModuleTimeout
}
// NewConfig 创建带默认值的Config(后备用,正常流程使用BuildConfigFromFlags
func NewConfig() *Config {
return &Config{
// 高频字段 - 使用默认常量
Timeout: time.Duration(DefaultTimeout) * time.Second,
ThreadNum: DefaultThreadNum,
ThreadCeiling: DefaultThreadNum,
ModuleThreadNum: 10,
DisableBrute: false,
DisablePing: false,
DisableTcpProbe: false,
// 扫描模式
Mode: DefaultScanMode,
@@ -155,13 +209,13 @@ func NewConfig() *Config {
MaxRetries: 3,
// 高级功能 - 使用默认配置
PortMap: config.DefaultPortMap,
DefaultMap: config.DefaultProbeMap,
PortMap: clonePortMap(config.DefaultPortMap),
DefaultMap: cloneStringSlice(config.DefaultProbeMap),
// 分组配置 - 使用默认字典
Credentials: CredentialConfig{
Userdict: config.DefaultUserDict,
Passwords: config.DefaultPasswords,
Userdict: cloneStringSliceMap(config.DefaultUserDict),
Passwords: cloneStringSlice(config.DefaultPasswords),
UserPassPairs: nil,
},
Network: NetworkConfig{
+24 -22
View File
@@ -9,6 +9,8 @@ import (
"runtime"
"runtime/pprof"
"runtime/trace"
"github.com/shadow1ng/fscan/common/i18n"
)
var (
@@ -19,82 +21,82 @@ var (
func Start() {
if err := os.MkdirAll(profilesPath, 0755); err != nil {
fmt.Printf("[DEBUG] 创建 profiles 目录失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_profiles_failed", err))
return
}
var err error
cpuProfile, err = os.Create(profilesPath + "/cpu.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建 CPU profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_cpu_profile_failed", err))
} else {
if err := pprof.StartCPUProfile(cpuProfile); err != nil {
fmt.Printf("[DEBUG] 启动 CPU profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_start_cpu_profile_failed", err))
cpuProfile.Close()
cpuProfile = nil
} else {
fmt.Printf("[DEBUG] CPU profiling 已启动 -> %s/cpu.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_cpu_profile_started", profilesPath))
}
}
traceFile, err = os.Create(profilesPath + "/trace.out")
if err != nil {
fmt.Printf("[DEBUG] 创建 trace 文件失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_trace_failed", err))
} else {
if err := trace.Start(traceFile); err != nil {
fmt.Printf("[DEBUG] 启动 trace 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_start_trace_failed", err))
traceFile.Close()
traceFile = nil
} else {
fmt.Printf("[DEBUG] Execution trace 已启动 -> %s/trace.out\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_trace_started", profilesPath))
}
}
fmt.Printf("[DEBUG] 性能分析已启动,程序结束时自动保存到 %s/\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_profiling_started", profilesPath))
}
func Stop() {
if cpuProfile != nil {
pprof.StopCPUProfile()
cpuProfile.Close()
fmt.Printf("[DEBUG] CPU profile 已保存\n")
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_cpu_profile_saved"))
}
if traceFile != nil {
trace.Stop()
traceFile.Close()
fmt.Printf("[DEBUG] Trace 已保存\n")
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_trace_saved"))
}
memProfile, err := os.Create(profilesPath + "/mem.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建内存 profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_mem_profile_failed", err))
} else {
runtime.GC()
if err := pprof.WriteHeapProfile(memProfile); err != nil {
fmt.Printf("[DEBUG] 写入内存 profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_write_mem_profile_failed", err))
} else {
fmt.Printf("[DEBUG] 内存 profile 已保存 -> %s/mem.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_mem_profile_saved", profilesPath))
}
memProfile.Close()
}
goroutineProfile, err := os.Create(profilesPath + "/goroutine.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建 goroutine profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_goroutine_profile_failed", err))
} else {
if err := pprof.Lookup("goroutine").WriteTo(goroutineProfile, 0); err != nil {
fmt.Printf("[DEBUG] 写入 goroutine profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_write_goroutine_profile_failed", err))
} else {
fmt.Printf("[DEBUG] Goroutine profile 已保存 -> %s/goroutine.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_goroutine_profile_saved", profilesPath))
}
goroutineProfile.Close()
}
fmt.Printf("\n[DEBUG] 所有性能分析文件已保存到 %s/\n", profilesPath)
fmt.Printf("[DEBUG] 查看方法:\n")
fmt.Printf(" CPU 火焰图: go tool pprof -http=:8081 %s/cpu.prof\n", profilesPath)
fmt.Printf(" 内存火焰图: go tool pprof -http=:8081 %s/mem.prof\n", profilesPath)
fmt.Printf(" 协程分析: go tool pprof -http=:8081 %s/goroutine.prof\n", profilesPath)
fmt.Printf(" 执行时间线: go tool trace %s/trace.out\n", profilesPath)
fmt.Printf("\n[DEBUG] %s\n", i18n.Tr("debug_profiles_saved", profilesPath))
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_view_methods"))
fmt.Printf(" %s: go tool pprof -http=:8081 %s/cpu.prof\n", i18n.GetText("debug_cpu_flamegraph"), profilesPath)
fmt.Printf(" %s: go tool pprof -http=:8081 %s/mem.prof\n", i18n.GetText("debug_mem_flamegraph"), profilesPath)
fmt.Printf(" %s: go tool pprof -http=:8081 %s/goroutine.prof\n", i18n.GetText("debug_goroutine_analysis"), profilesPath)
fmt.Printf(" %s: go tool trace %s/trace.out\n", i18n.GetText("debug_execution_timeline"), profilesPath)
}
+11
View File
@@ -0,0 +1,11 @@
//go:build !debug
// +build !debug
package debug
import "testing"
func TestStubStartStop(t *testing.T) {
Start()
Stop()
}
+28
View File
@@ -0,0 +1,28 @@
package common
import (
"net"
"sync"
)
// DNSCache 并发安全的 DNS 解析缓存
// 对纯 IP 输入零开销(直接返回),对域名避免重复系统调用
var DNSCache = &dnsCache{}
type dnsCache struct {
m sync.Map // host -> *net.IPAddr
}
// ResolveIP 解析 host 为 *net.IPAddr,结果缓存
func (c *dnsCache) ResolveIP(host string) (*net.IPAddr, error) {
if v, ok := c.m.Load(host); ok {
addr, _ := v.(*net.IPAddr)
return addr, nil
}
addr, err := net.ResolveIPAddr("ip", host)
if err != nil {
return nil, err
}
c.m.Store(host, addr)
return addr, nil
}
+23
View File
@@ -0,0 +1,23 @@
package common
import "testing"
func TestDNSCacheResolveIPAndCacheHit(t *testing.T) {
cache := &dnsCache{}
first, err := cache.ResolveIP("127.0.0.1")
if err != nil {
t.Fatalf("ResolveIP loopback error = %v", err)
}
second, err := cache.ResolveIP("127.0.0.1")
if err != nil {
t.Fatalf("ResolveIP cached loopback error = %v", err)
}
if first != second {
t.Fatal("ResolveIP should return cached address on second lookup")
}
if _, err := cache.ResolveIP("bad host with spaces"); err == nil {
t.Fatal("ResolveIP should reject an invalid host")
}
}
+99 -6
View File
@@ -15,6 +15,9 @@ import (
// ErrShowHelp 表示用户请求显示帮助(正常退出)
var ErrShowHelp = errors.New("show help requested")
// IsLocalMode 由 plugins 包注册,判断 -m 指定的是否全是本地插件
var IsLocalMode func(mode string) bool
// Banner 显示程序横幅信息
func Banner() {
// 静默模式下完全跳过Banner显示
@@ -109,7 +112,8 @@ func Flag(Info *HostInfo) error {
flag.IntVar(&fv.ModuleThreadNum, "mt", 20, i18n.GetText("flag_module_thread_num"))
flag.Int64Var(&fv.GlobalTimeout, "gt", 180, i18n.GetText("flag_global_timeout"))
flag.BoolVar(&fv.DisablePing, "np", false, i18n.GetText("flag_disable_ping"))
flag.StringVar(&fv.LocalPlugin, "local", "", "指定本地插件名称 (如: cleaner, avdetect, keylogger 等)")
flag.BoolVar(&fv.DisableTcpProbe, "ntp", false, i18n.GetText("flag_disable_tcp_probe"))
flag.StringVar(&fv.LocalPlugin, "local", "", i18n.GetText("flag_local_plugin"))
flag.BoolVar(&fv.AliveOnly, "ao", false, i18n.GetText("flag_alive_only"))
// ═════════════════════════════════════════════════
@@ -133,6 +137,7 @@ func Flag(Info *HostInfo) error {
flag.StringVar(&fv.TargetURL, "u", "", i18n.GetText("flag_target_url"))
flag.StringVar(&fv.URLsFile, "uf", "", i18n.GetText("flag_urls_file"))
flag.StringVar(&fv.Cookie, "cookie", "", i18n.GetText("flag_cookie"))
flag.StringVar(&fv.UserAgent, "ua", "", i18n.GetText("flag_user_agent"))
flag.Int64Var(&fv.WebTimeout, "wt", 5, i18n.GetText("flag_web_timeout"))
flag.IntVar(&fv.MaxRedirects, "max-redirect", 10, i18n.GetText("flag_max_redirects"))
flag.StringVar(&fv.HTTPProxy, "proxy", "", i18n.GetText("flag_http_proxy"))
@@ -181,8 +186,9 @@ func Flag(Info *HostInfo) error {
flag.BoolVar(&fv.Silent, "silent", false, i18n.GetText("flag_silent_mode"))
flag.BoolVar(&fv.NoColor, "nocolor", false, i18n.GetText("flag_no_color"))
flag.StringVar(&fv.LogLevel, "log", LogLevelBaseInfoSuccess, i18n.GetText("flag_log_level"))
flag.BoolVar(&fv.Debug, "debug", false, i18n.GetText("flag_debug"))
flag.BoolVar(&fv.DisableProgress, "nopg", false, i18n.GetText("flag_disable_progress"))
flag.BoolVar(&fv.PerfStats, "perf", false, "输出性能统计JSON")
flag.BoolVar(&fv.PerfStats, "perf", false, i18n.GetText("flag_perf_stats"))
// ═════════════════════════════════════════════════
// 其他参数
@@ -208,6 +214,24 @@ func Flag(Info *HostInfo) error {
return err
}
// 检测用户是否显式指定了 -t
flag.Visit(func(f *flag.Flag) {
switch f.Name {
case "t":
fv.ThreadNumExplicit = true
case "time":
fv.TimeoutExplicit = true
case "mt":
fv.ModuleThreadNumExplicit = true
case "retry":
fv.MaxRetriesExplicit = true
case "icmp-rate":
fv.ICMPRateExplicit = true
case "num":
fv.PocNumExplicit = true
}
})
// 设置语言
i18n.SetLanguage(fv.Language)
@@ -222,7 +246,9 @@ func Flag(Info *HostInfo) error {
// parseCommandLineArgs 解析命令行参数
func parseCommandLineArgs() error {
flag.Parse()
if err := flag.CommandLine.Parse(normalizeMultiValueFlagArgs(os.Args[1:], "-pwda")); err != nil {
return err
}
// 显示Banner
Banner()
@@ -231,6 +257,63 @@ func parseCommandLineArgs() error {
return checkParameterConflicts()
}
func normalizeMultiValueFlagArgs(args []string, names ...string) []string {
multiValueFlags := make(map[string]struct{}, len(names))
for _, name := range names {
multiValueFlags[name] = struct{}{}
}
normalized := make([]string, 0, len(args))
for i := 0; i < len(args); i++ {
arg := args[i]
name, value, ok := splitMultiValueFlag(arg, multiValueFlags)
if !ok {
normalized = append(normalized, arg)
continue
}
values := []string{}
if value != "" {
values = append(values, value)
}
j := i + 1
for ; j < len(args); j++ {
if strings.HasPrefix(args[j], "-") {
break
}
values = append(values, args[j])
}
i = j - 1
if strings.Contains(arg, "=") {
normalized = append(normalized, name+"="+strings.Join(values, ","))
} else {
normalized = append(normalized, name)
if len(values) > 0 {
normalized = append(normalized, strings.Join(values, ","))
}
}
}
return normalized
}
func splitMultiValueFlag(arg string, names map[string]struct{}) (string, string, bool) {
if _, ok := names[arg]; ok {
return arg, "", true
}
for name := range names {
prefix := name + "="
if strings.HasPrefix(arg, prefix) {
return name, strings.TrimPrefix(arg, prefix), true
}
}
return "", "", false
}
// preProcessLanguage 预处理语言参数,在定义flag之前设置语言
func preProcessLanguage() {
// 遍历命令行参数查找-lang参数
@@ -270,9 +353,14 @@ func shouldShowHelp(Info *HostInfo, fv *FlagVars) bool {
// 检查是否提供了扫描目标
hasTarget := Info.Host != "" || fv.TargetURL != "" || fv.HostsFile != "" || fv.URLsFile != ""
// 本地模式需要指定插件才算有效目标
// 本地模式需要目标主机
if fv.LocalPlugin != "" {
hasTarget = true
return false
}
// -m 指定的全是本地插件时也不需要目标
if IsLocalMode != nil && IsLocalMode(fv.ScanMode) {
return false
}
// 如果没有提供任何扫描目标,则显示帮助
@@ -284,6 +372,11 @@ func shouldShowHelp(Info *HostInfo, fv *FlagVars) bool {
func checkParameterConflicts() error {
fv := flagVars
// -debug 等价于 -log debug
if fv.Debug {
fv.LogLevel = LogLevelDebug
}
// 检查 -ao 和 -m icmp 同时指定的情况(向后兼容提示)
if fv.AliveOnly && fv.ScanMode == "icmp" {
LogInfo(i18n.GetText("param_conflict_ao_icmp_both"))
@@ -295,7 +388,7 @@ func checkParameterConflicts() error {
invalidChars := []string{",", ";", " ", "|", "&"}
for _, char := range invalidChars {
if strings.Contains(fv.LocalPlugin, char) {
return fmt.Errorf("本地插件只能指定单个插件,不支持使用 '%s' 分隔的多个插件", char)
return fmt.Errorf("%s", i18n.Tr("param_local_multi_plugin", char))
}
}
}
+39
View File
@@ -0,0 +1,39 @@
package common
import (
"reflect"
"testing"
)
func TestNormalizeMultiValueFlagArgs(t *testing.T) {
tests := []struct {
name string
args []string
want []string
}{
{
name: "space separated pwda values",
args: []string{"-h", "192.168.1.1", "-pwda", "pass1", "pass2", "pass3", "-m", "ssh"},
want: []string{"-h", "192.168.1.1", "-pwda", "pass1,pass2,pass3", "-m", "ssh"},
},
{
name: "equals form with extra values",
args: []string{"-pwda=pass1", "pass2", "-h", "192.168.1.1"},
want: []string{"-pwda=pass1,pass2", "-h", "192.168.1.1"},
},
{
name: "unrelated args unchanged",
args: []string{"-h", "192.168.1.1", "-m", "ssh"},
want: []string{"-h", "192.168.1.1", "-m", "ssh"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := normalizeMultiValueFlagArgs(tt.args, "-pwda")
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("normalizeMultiValueFlagArgs() = %#v, want %#v", got, tt.want)
}
})
}
}
+74 -43
View File
@@ -1,9 +1,11 @@
package common
import (
"os"
"time"
"github.com/shadow1ng/fscan/common/config"
"golang.org/x/term"
)
/*
@@ -25,21 +27,25 @@ type FlagVars struct {
ExcludeHostsFile string
Ports string
ExcludePorts string
AddPorts string
HostsFile string
PortsFile string
// 扫描控制
ScanMode string
ThreadNum int
ModuleThreadNum int
TimeoutSec int64 // 秒,需转换为 time.Duration
GlobalTimeout int64
DisablePing bool
LocalPlugin string
AliveOnly bool
DisableBrute bool
MaxRetries int
ScanMode string
ThreadNum int
ThreadNumExplicit bool // 用户显式指定了 -t
ModuleThreadNum int
ModuleThreadNumExplicit bool
TimeoutSec int64 // 秒,需转换为 time.Duration
TimeoutExplicit bool
GlobalTimeout int64
DisablePing bool
DisableTcpProbe bool
LocalPlugin string
AliveOnly bool
DisableBrute bool
MaxRetries int
MaxRetriesExplicit bool
// 认证凭据
Username string
@@ -72,6 +78,7 @@ type FlagVars struct {
PocFull bool
DNSLog bool
PocNum int
PocNumExplicit bool
DisablePocScan bool
// Redis利用
@@ -83,9 +90,10 @@ type FlagVars struct {
DisableRedis bool
// 发包频率
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
ICMPRateExplicit bool
// 输出控制
Outputfile string
@@ -94,6 +102,7 @@ type FlagVars struct {
Silent bool
NoColor bool
LogLevel string
Debug bool
DisableProgress bool
PerfStats bool
Language string
@@ -132,18 +141,23 @@ func GetFlagVars() *FlagVars {
func BuildConfigFromFlags(fv *FlagVars) *Config {
return &Config{
// 高频字段
Timeout: time.Duration(fv.TimeoutSec) * time.Second,
ThreadNum: fv.ThreadNum,
ModuleThreadNum: fv.ModuleThreadNum,
DisableBrute: fv.DisableBrute,
DisablePing: fv.DisablePing,
Timeout: time.Duration(fv.TimeoutSec) * time.Second,
TimeoutExplicit: fv.TimeoutExplicit,
ThreadNum: fv.ThreadNum,
ThreadNumExplicit: fv.ThreadNumExplicit,
ModuleThreadNum: fv.ModuleThreadNum,
ModuleThreadNumExplicit: fv.ModuleThreadNumExplicit,
DisableBrute: fv.DisableBrute,
DisablePing: fv.DisablePing,
DisableTcpProbe: fv.DisableTcpProbe,
// 扫描模式
Mode: fv.ScanMode,
LocalMode: fv.LocalPlugin != "",
LocalPlugin: fv.LocalPlugin,
AliveOnly: fv.AliveOnly,
MaxRetries: fv.MaxRetries,
Mode: fv.ScanMode,
LocalMode: fv.LocalPlugin != "",
LocalPlugin: fv.LocalPlugin,
AliveOnly: fv.AliveOnly,
MaxRetries: fv.MaxRetries,
MaxRetriesExplicit: fv.MaxRetriesExplicit,
// 高级功能
Shellcode: fv.Shellcode,
@@ -151,8 +165,11 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
DNSLog: fv.DNSLog,
PersistenceTargetFile: fv.PersistenceTargetFile,
WinPEFile: fv.WinPEFile,
PortMap: config.DefaultPortMap,
DefaultMap: config.DefaultProbeMap,
PortMap: clonePortMap(config.DefaultPortMap),
DefaultMap: cloneStringSlice(config.DefaultProbeMap),
// 全局超时
GlobalTimeout: time.Duration(fv.GlobalTimeout) * time.Second,
// SOCKS5代理端口
Socks5ProxyPort: fv.Socks5ProxyPort,
@@ -162,26 +179,27 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
Username: fv.Username,
Password: fv.Password,
Domain: fv.Domain,
Userdict: config.DefaultUserDict,
Passwords: config.DefaultPasswords,
Userdict: cloneStringSliceMap(config.DefaultUserDict),
Passwords: cloneStringSlice(config.DefaultPasswords),
UserPassPairs: nil, // 后续解析
SSHKeyPath: fv.SSHKeyPath,
},
Network: NetworkConfig{
HTTPProxy: fv.HTTPProxy,
Socks5Proxy: fv.Socks5Proxy,
Iface: fv.Iface,
WebTimeout: time.Duration(fv.WebTimeout) * time.Second,
MaxRedirects: fv.MaxRedirects,
PacketRateLimit: fv.PacketRateLimit,
MaxPacketCount: fv.MaxPacketCount,
ICMPRate: fv.ICMPRate,
HTTPProxy: fv.HTTPProxy,
Socks5Proxy: fv.Socks5Proxy,
Iface: fv.Iface,
WebTimeout: time.Duration(fv.WebTimeout) * time.Second,
MaxRedirects: fv.MaxRedirects,
PacketRateLimit: fv.PacketRateLimit,
MaxPacketCount: fv.MaxPacketCount,
ICMPRate: fv.ICMPRate,
ICMPRateExplicit: fv.ICMPRateExplicit,
},
Output: OutputConfig{
File: fv.Outputfile,
Format: fv.OutputFormat,
DisableSave: fv.DisableSave,
NoColor: fv.NoColor,
NoColor: fv.NoColor || !isStdoutTerminal(),
Silent: fv.Silent,
DisableProgress: fv.DisableProgress,
ShowProgress: !fv.DisableProgress,
@@ -190,11 +208,12 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
PerfStats: fv.PerfStats,
},
POC: POCConfig{
PocPath: fv.PocPath,
PocName: fv.PocName,
Full: fv.PocFull,
Num: fv.PocNum,
Disabled: fv.DisablePocScan,
PocPath: fv.PocPath,
PocName: fv.PocName,
Full: fv.PocFull,
Num: fv.PocNum,
NumExplicit: fv.PocNumExplicit,
Disabled: fv.DisablePocScan,
},
Redis: RedisConfig{
Disabled: fv.DisableRedis,
@@ -206,7 +225,7 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
},
HTTP: HTTPConfig{
Cookie: fv.Cookie,
UserAgent: fv.UserAgent,
UserAgent: defaultUserAgent(fv.UserAgent),
Accept: fv.Accept,
},
LocalExploit: LocalExploitConfig{
@@ -222,3 +241,15 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
},
}
}
func isStdoutTerminal() bool {
return term.IsTerminal(int(os.Stdout.Fd()))
}
// defaultUserAgent 用户未通过 -ua 指定时回退到默认 UA,避免发送空 User-Agent 被 WAF 识别
func defaultUserAgent(ua string) string {
if ua != "" {
return ua
}
return "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
}
+3 -2
View File
@@ -955,8 +955,9 @@ func TestBuildConfigFromFlags_BoundaryValues(t *testing.T) {
if cfg.HTTP.Cookie != "" {
t.Errorf("Cookie 应该为空")
}
if cfg.HTTP.UserAgent != "" {
t.Errorf("UserAgent 应该为空")
// 空输入回退到默认 UA,避免发送空 User-Agent
if cfg.HTTP.UserAgent == "" {
t.Errorf("UserAgent 空输入应回退到默认 UA")
}
},
},
+4 -11
View File
@@ -2,15 +2,8 @@
package common
import "flag"
// WebMode Web版本始终为true
const WebMode = true
// WebMode 表示是否启动Web管理界面
var WebMode bool
// WebPort Web服务器端口
var WebPort int
func init() {
flag.BoolVar(&WebMode, "web", false, "启动Web管理界面 (Start Web UI)")
flag.IntVar(&WebPort, "webport", 10240, "Web服务器端口 (Web server port)")
}
// WebPort 不再使用,端口由 main_web.go 的 -port 参数控制
var WebPort = 0
+14 -5
View File
@@ -2,9 +2,12 @@ package common
import (
"errors"
"fmt"
"net"
"strconv"
"strings"
"sync"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -28,7 +31,11 @@ type HostInfo struct {
// Target 返回 host:port 格式字符串
func (h *HostInfo) Target() string {
return fmt.Sprintf("%s:%d", h.Host, h.Port)
host := h.Host
if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") {
host = strings.TrimPrefix(strings.TrimSuffix(host, "]"), "[")
}
return net.JoinHostPort(host, strconv.Itoa(h.Port))
}
// =============================================================================
@@ -62,11 +69,13 @@ const (
// 版本信息,通过 ldflags 注入
var (
version = "2.1.2"
version = "2.2.0-rc.1"
commit = "unknown"
date = "unknown"
)
func GetVersion() string { return version }
// 运行时数据已迁移到Config对象中,使用GetGlobalConfig()访问
// Shell状态已迁移到State对象中,使用GetGlobalState()访问
@@ -92,9 +101,9 @@ type PacketLimitError struct {
func (e *PacketLimitError) Error() string {
if e.Sentinel == ErrMaxPacketReached {
return fmt.Sprintf("已达到最大发包数量限制: %d", e.Limit)
return i18n.Tr("packet_limit_max_reached", e.Limit)
}
return fmt.Sprintf("发包速率受限: %d包/分钟", e.Limit)
return i18n.Tr("packet_limit_rate_limited", e.Limit)
}
func (e *PacketLimitError) Unwrap() error {
+67
View File
@@ -0,0 +1,67 @@
package common
import (
"errors"
"strings"
"testing"
)
func TestHostInfoTargetUsesBracketedIPv6(t *testing.T) {
info := &HostInfo{Host: "2001:db8::1", Port: 443}
if got, want := info.Target(), "[2001:db8::1]:443"; got != want {
t.Fatalf("Target() = %q, want %q", got, want)
}
}
func TestHostInfoTargetDoesNotDoubleBracketIPv6(t *testing.T) {
info := &HostInfo{Host: "[2001:db8::1]", Port: 443}
if got, want := info.Target(), "[2001:db8::1]:443"; got != want {
t.Fatalf("Target() = %q, want %q", got, want)
}
}
func TestGlobalHelpersAndPacketLimitErrors(t *testing.T) {
if GetVersion() == "" {
t.Fatal("GetVersion returned empty string")
}
if !ContainsAny("hello fscan", "none", "scan") {
t.Fatal("ContainsAny should find a matching substring")
}
if ContainsAny("hello fscan", "none", "missing") {
t.Fatal("ContainsAny should return false when nothing matches")
}
maxErr := &PacketLimitError{Sentinel: ErrMaxPacketReached, Limit: 5, Current: 5}
if !errors.Is(maxErr, ErrMaxPacketReached) || !strings.Contains(maxErr.Error(), "5") {
t.Fatalf("max packet error = %v", maxErr)
}
rateErr := &PacketLimitError{Sentinel: ErrPacketRateLimited, Limit: 3, Current: 2}
if !errors.Is(rateErr, ErrPacketRateLimited) || !strings.Contains(rateErr.Error(), "3") {
t.Fatalf("rate limit error = %v", rateErr)
}
}
func TestCanSendPacketUsesGlobalConfigAndState(t *testing.T) {
previousConfig := GetGlobalConfig()
previousState := GetGlobalState()
t.Cleanup(func() {
SetGlobalConfig(previousConfig)
SetGlobalState(previousState)
})
cfg := NewConfig()
cfg.Network.MaxPacketCount = 1
state := NewState()
state.IncrementPacketCount()
SetGlobalConfig(cfg)
SetGlobalState(state)
ok, reason := CanSendPacket()
if ok {
t.Fatal("CanSendPacket should reject when max packet count is reached")
}
if reason == "" {
t.Fatal("CanSendPacket should return a rejection reason")
}
}
+10 -2
View File
@@ -2,6 +2,7 @@ package i18n
import (
"fmt"
"strconv"
"sync"
"github.com/nicksnyder/go-i18n/v2/i18n"
@@ -51,6 +52,13 @@ func SetLanguage(l string) {
localizer = i18n.NewLocalizer(bundle, lang, FallbackLanguage)
}
// GetLanguage returns the currently configured language.
func GetLanguage() string {
mu.RLock()
defer mu.RUnlock()
return lang
}
// GetText 获取国际化文本(无参数)
func GetText(key string) string {
mu.RLock()
@@ -73,9 +81,9 @@ func Tr(key string, args ...interface{}) string {
loc := localizer
mu.RUnlock()
data := make(map[string]interface{})
data := make(map[string]interface{}, len(args))
for i, arg := range args {
data[fmt.Sprintf("Arg%d", i+1)] = arg
data["Arg"+strconv.Itoa(i+1)] = arg
}
msg, err := loc.Localize(&i18n.LocalizeConfig{
+37
View File
@@ -0,0 +1,37 @@
package i18n
import (
"strings"
"testing"
)
func TestLanguageLifecycleAndFallbacks(t *testing.T) {
original := GetLanguage()
t.Cleanup(func() { SetLanguage(original) })
SetLanguage(LangEN)
if got := GetLanguage(); got != LangEN {
t.Fatalf("language = %q, want %q", got, LangEN)
}
if got := GetText("concurrency_plugin"); got == "" || got == "concurrency_plugin" {
t.Fatalf("english text = %q, want translated text", got)
}
if got := Tr("debug_cpu_profile_started", "/tmp/profiles"); !strings.Contains(got, "/tmp/profiles") {
t.Fatalf("formatted english text = %q, want path included", got)
}
SetLanguage(LangZH)
if got := GetLanguage(); got != LangZH {
t.Fatalf("language = %q, want %q", got, LangZH)
}
if got := GetText("concurrency_plugin"); got == "" || got == "concurrency_plugin" {
t.Fatalf("chinese text = %q, want translated text", got)
}
if got := GetText("missing_translation_key"); got != "missing_translation_key" {
t.Fatalf("missing GetText = %q, want key", got)
}
if got := Tr("missing_translation_key", "ignored"); got != "missing_translation_key" {
t.Fatalf("missing Tr = %q, want key", got)
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+17 -11
View File
@@ -2,6 +2,8 @@ package common
import (
"fmt"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -13,9 +15,10 @@ initialize.go - 统一初始化入口
// InitResult 初始化结果
type InitResult struct {
Config *Config
State *State
Info *HostInfo
Config *Config
State *State
Info *HostInfo
Session *ScanSession
}
// Initialize 统一初始化函数
@@ -27,7 +30,7 @@ func Initialize(info *HostInfo) (*InitResult, error) {
// 2. 从 FlagVars 构建 Config 和 State
cfg, state, err := BuildConfig(GetFlagVars(), info)
if err != nil {
return nil, fmt.Errorf("配置构建失败: %w", err)
return nil, fmt.Errorf("%s: %w", i18n.GetText("config_build_failed"), err)
}
// 3. 设置全局实例
@@ -36,13 +39,16 @@ func Initialize(info *HostInfo) (*InitResult, error) {
// 4. 初始化输出系统
if err := InitOutput(); err != nil {
return nil, fmt.Errorf("输出初始化失败: %w", err)
return nil, fmt.Errorf("%s: %w", i18n.GetText("output_init_failed"), err)
}
session := NewScanSession(cfg, state, GetFlagVars())
return &InitResult{
Config: cfg,
State: state,
Info: info,
Config: cfg,
State: state,
Info: info,
Session: session,
}, nil
}
@@ -61,7 +67,7 @@ func ValidateExclusiveParams(info *HostInfo) error {
if fv.TargetURL != "" {
paramCount++
if activeParam != "" {
activeParam += " 和 -u"
activeParam = i18n.Tr("param_join_and", activeParam, "-u")
} else {
activeParam = "-u"
}
@@ -69,14 +75,14 @@ func ValidateExclusiveParams(info *HostInfo) error {
if fv.LocalPlugin != "" {
paramCount++
if activeParam != "" {
activeParam += " 和 -local"
activeParam = i18n.Tr("param_join_and", activeParam, "-local")
} else {
activeParam = "-local"
}
}
if paramCount > 1 {
return fmt.Errorf("参数 %s 互斥,请只指定一个扫描目标\n -h: 网络主机扫描\n -u: Web URL扫描\n -local: 本地信息收集", activeParam)
return fmt.Errorf("%s", i18n.Tr("param_exclusive", activeParam))
}
return nil
+55
View File
@@ -0,0 +1,55 @@
package common
import (
"strings"
"testing"
)
func TestValidateExclusiveParams(t *testing.T) {
previous := GetFlagVars()
t.Cleanup(func() { flagVars = previous })
tests := []struct {
name string
info *HostInfo
flags *FlagVars
wantErr string
}{
{name: "host only", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{}},
{name: "url only", info: &HostInfo{}, flags: &FlagVars{TargetURL: "http://example.com"}},
{name: "local only", info: &HostInfo{}, flags: &FlagVars{LocalPlugin: "sshkey"}},
{name: "host and url conflict", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{TargetURL: "http://example.com"}, wantErr: "-h"},
{name: "host url local conflict", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{TargetURL: "http://example.com", LocalPlugin: "sshkey"}, wantErr: "-local"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
flagVars = tt.flags
err := ValidateExclusiveParams(tt.info)
if tt.wantErr == "" {
if err != nil {
t.Fatalf("ValidateExclusiveParams error = %v", err)
}
return
}
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("ValidateExclusiveParams error = %v, want containing %q", err, tt.wantErr)
}
})
}
}
func TestCleanupWithoutOutput(t *testing.T) {
oldResultOutput := ResultOutput
oldStdoutWriter := StdoutWriter
t.Cleanup(func() {
ResultOutput = oldResultOutput
StdoutWriter = oldStdoutWriter
})
ResultOutput = nil
StdoutWriter = nil
if err := Cleanup(); err != nil {
t.Fatalf("Cleanup error = %v", err)
}
}
+52 -2
View File
@@ -14,11 +14,16 @@ import (
)
var (
globalLogger *logging.Logger
loggerOnce sync.Once
globalLogger *logging.Logger
loggerOnce sync.Once
loggerMu sync.Mutex
silentLoggerRefs int
)
func getGlobalLogger() *logging.Logger {
loggerMu.Lock()
defer loggerMu.Unlock()
loggerOnce.Do(func() {
fv := GetFlagVars()
level := getLogLevelFromString(fv.LogLevel)
@@ -27,8 +32,12 @@ func getGlobalLogger() *logging.Logger {
EnableColor: !fv.NoColor,
SlowOutput: false,
ShowProgress: !fv.DisableProgress,
Silent: fv.Silent || silentLoggerRefs > 0,
StartTime: GetGlobalState().GetStartTime(),
}
if fv.Debug {
config.DebugLogFile = "fscan_debug.log"
}
globalLogger = logging.NewLogger(config)
globalLogger.SetCoordinatedOutput(LogWithProgress)
})
@@ -77,3 +86,44 @@ func LogVuln(result string) { getGlobalLogger().Vuln(result) }
// LogError 输出错误日志
func LogError(errMsg string) { getGlobalLogger().Error(errMsg) }
// CloseLogger 关闭日志系统,释放文件资源
func CloseLogger() {
loggerMu.Lock()
defer loggerMu.Unlock()
closeLoggerLocked()
}
// PushSilentLogger suppresses process-wide legacy log output until the returned
// restore function is called. It is reference counted so concurrent embedded
// scans can overlap safely.
func PushSilentLogger() func() {
loggerMu.Lock()
silentLoggerRefs++
resetLoggerLocked()
loggerMu.Unlock()
var once sync.Once
return func() {
once.Do(func() {
loggerMu.Lock()
if silentLoggerRefs > 0 {
silentLoggerRefs--
}
resetLoggerLocked()
loggerMu.Unlock()
})
}
}
func resetLoggerLocked() {
closeLoggerLocked()
globalLogger = nil
loggerOnce = sync.Once{}
}
func closeLoggerLocked() {
if globalLogger != nil {
globalLogger.Close()
}
}
+63
View File
@@ -0,0 +1,63 @@
package common
import "testing"
func preserveLoggerForTest(t *testing.T) {
t.Helper()
loggerMu.Lock()
oldSilentRefs := silentLoggerRefs
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
t.Cleanup(func() {
loggerMu.Lock()
closeLoggerLocked()
silentLoggerRefs = oldSilentRefs
resetLoggerLocked()
loggerMu.Unlock()
})
}
func TestLoggerFacadeSilentLifecycle(t *testing.T) {
preserveLoggerForTest(t)
previousFlags := GetFlagVars()
previousState := GetGlobalState()
t.Cleanup(func() {
flagVars = previousFlags
SetGlobalState(previousState)
})
flagVars = &FlagVars{Silent: true, LogLevel: "debug"}
SetGlobalState(NewState())
InitLogger()
LogDebug("debug")
LogInfo("info")
LogSuccess("success")
LogVuln("vuln")
LogError("error")
CloseLogger()
}
func TestPushSilentLoggerReferenceCount(t *testing.T) {
preserveLoggerForTest(t)
restoreOne := PushSilentLogger()
restoreTwo := PushSilentLogger()
if silentLoggerRefs != 2 {
t.Fatalf("silent refs = %d, want 2", silentLoggerRefs)
}
restoreOne()
restoreOne()
if silentLoggerRefs != 1 {
t.Fatalf("silent refs after first restore = %d, want 1", silentLoggerRefs)
}
restoreTwo()
if silentLoggerRefs != 0 {
t.Fatalf("silent refs after second restore = %d, want 0", silentLoggerRefs)
}
}
+43 -1
View File
@@ -2,6 +2,7 @@ package logging
import (
"fmt"
"os"
"strings"
"sync"
"time"
@@ -24,8 +25,10 @@ type LoggerConfig struct {
EnableColor bool `json:"enable_color"`
SlowOutput bool `json:"slow_output"`
ShowProgress bool `json:"show_progress"`
Silent bool `json:"silent"`
StartTime time.Time `json:"start_time"`
LevelColors map[LogLevel]interface{} `json:"-"`
DebugLogFile string `json:"debug_log_file"`
}
// DefaultLoggerConfig 默认日志器配置
@@ -47,6 +50,7 @@ type Logger struct {
startTime time.Time
coordinatedOutput func(string)
initialized bool
debugFile *os.File
}
// NewLogger 创建新的日志管理器
@@ -55,11 +59,20 @@ func NewLogger(config *LoggerConfig) *Logger {
config = DefaultLoggerConfig()
}
return &Logger{
l := &Logger{
config: config,
startTime: config.StartTime,
initialized: true,
}
if config.DebugLogFile != "" {
f, err := os.OpenFile(config.DebugLogFile, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0644)
if err == nil {
l.debugFile = f
}
}
return l
}
// Initialize 初始化日志器
@@ -111,6 +124,10 @@ func (l *Logger) log(level LogLevel, content string) {
l.mu.Lock()
defer l.mu.Unlock()
if l.config.Silent {
return
}
if !l.shouldLog(level) {
return
}
@@ -134,12 +151,37 @@ func (l *Logger) log(level LogLevel, content string) {
l.outputMessage(level, logMsg)
}
// 写入debug日志文件(纯文本,无颜色)
if l.debugFile != nil {
timestamp := time.Since(l.startTime).Truncate(time.Millisecond)
if strings.Contains(content, "\n") {
lines := strings.Split(content, "\n")
for _, line := range lines {
if line != "" {
_, _ = fmt.Fprintf(l.debugFile, "[%s] %s %s\n", timestamp, prefix, line)
}
}
} else {
_, _ = fmt.Fprintf(l.debugFile, "[%s] %s %s\n", timestamp, prefix, content)
}
}
// 根据慢速输出设置决定是否添加延迟
if l.config.SlowOutput {
time.Sleep(SlowOutputDelay)
}
}
// Close 关闭日志器,释放文件资源
func (l *Logger) Close() {
l.mu.Lock()
defer l.mu.Unlock()
if l.debugFile != nil {
_ = l.debugFile.Close()
l.debugFile = nil
}
}
// shouldLog 检查是否应该记录该级别的日志
// 层级过滤:消息级别 >= 配置级别 时显示,Error 始终显示
func (l *Logger) shouldLog(level LogLevel) bool {
+40
View File
@@ -2,6 +2,8 @@ package logging
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -160,6 +162,13 @@ func TestLogger_AllLevels(t *testing.T) {
wantMsg: "success message",
wantPfx: PrefixSuccess,
},
{
name: "Vuln级别",
logFunc: logger.Vuln,
message: "vuln message",
wantMsg: "vuln message",
wantPfx: PrefixVuln,
},
{
name: "Error级别",
logFunc: logger.Error,
@@ -650,3 +659,34 @@ func TestLogger_Initialize(t *testing.T) {
t.Logf("✓ Initialize测试通过")
}
func TestLogger_CloseClosesDebugFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "debug.log")
logger := NewLogger(&LoggerConfig{
Level: LevelAll,
EnableColor: false,
ShowProgress: false,
StartTime: time.Now(),
LevelColors: GetDefaultLevelColors(),
DebugLogFile: path,
})
if logger.debugFile == nil {
t.Fatal("debug file should be opened")
}
logger.Info("debug file line")
logger.Close()
if logger.debugFile != nil {
t.Fatal("debug file should be nil after Close")
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read debug file: %v", err)
}
if !strings.Contains(string(content), "debug file line") {
t.Fatalf("debug file content = %q", string(content))
}
logger.Close()
}
+31 -12
View File
@@ -16,6 +16,7 @@ import (
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/proxy"
)
@@ -51,16 +52,31 @@ func getGlobalDialer(timeout time.Duration) (proxy.Dialer, error) {
// parseProxyURL 解析代理URL,提取地址和认证信息
func parseProxyURL(proxyURL, fallback string) (host, username, password string) {
if !strings.Contains(proxyURL, "://") {
if host, username, password, ok := parseProxyURLCandidate("http://" + proxyURL); ok {
return host, username, password
}
}
if host, username, password, ok := parseProxyURLCandidate(proxyURL); ok {
return host, username, password
}
return fallback, "", ""
}
func parseProxyURLCandidate(proxyURL string) (host, username, password string, ok bool) {
parsedURL, err := url.Parse(proxyURL)
if err != nil {
return fallback, "", ""
return "", "", "", false
}
host = parsedURL.Host
if host == "" {
return "", "", "", false
}
if parsedURL.User != nil {
username = parsedURL.User.Username()
password, _ = parsedURL.User.Password()
}
return
return host, username, password, true
}
// createProxyConfig 根据全局设置创建代理配置
@@ -102,22 +118,20 @@ func createProxyConfig(timeout time.Duration) *proxy.ProxyConfig {
// TCP 连接
// =============================================================================
// WrapperTcpWithTimeout TCP连接包装器,带超时
// 支持通过代理管理器进行SOCKS5和HTTP代理连接,并集成发包控制
// 使用全局拨号器复用连接,避免重复创建代理握手开销
// Deprecated: WrapperTcpWithTimeout 仅供 libs/grdp 兼容使用,新代码请用 ScanSession.DialTCP
//
//nolint:revive // 保持向后兼容性,避免破坏大量现有代码
//nolint:revive
func WrapperTcpWithTimeout(network, address string, timeout time.Duration) (net.Conn, error) {
// 检查发包限制 - 在代理连接前进行控制
if canSend, reason := CanSendPacket(); !canSend {
LogError(fmt.Sprintf("TCP连接 %s 受限: %s", address, reason))
return nil, fmt.Errorf("发包受限: %s", reason)
LogError(i18n.Tr("tcp_connection_restricted", address, reason))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", reason))
}
// 获取全局拨号器(复用,避免重复创建)
dialer, err := getGlobalDialer(timeout)
if err != nil {
LogError(fmt.Sprintf("获取代理拨号器失败: %v", err))
LogError(i18n.Tr("proxy_dialer_failed", err))
GetGlobalState().IncrementTCPFailedPacketCount()
return nil, err
}
@@ -128,7 +142,7 @@ func WrapperTcpWithTimeout(network, address string, timeout time.Duration) (net.
// 统计TCP包数量 - 无论是否使用代理都要计数
if err != nil {
GetGlobalState().IncrementTCPFailedPacketCount()
LogDebug(fmt.Sprintf("连接 %s 失败: %v", address, err))
LogDebug(i18n.Tr("connection_failed", address, err))
return nil, err
}
@@ -158,12 +172,17 @@ func IsProxyReliable() bool {
return proxy.IsProxyReliable()
}
// IsSOCKS5Proxy 检查当前代理是否为SOCKS5类型
func IsSOCKS5Proxy() bool {
return proxy.IsSOCKS5Proxy()
}
// SafeHTTPDo 带发包控制的HTTP请求
func SafeHTTPDo(client *http.Client, req *http.Request) (*http.Response, error) {
// 检查发包限制
if canSend, reason := CanSendPacket(); !canSend {
LogError(fmt.Sprintf("HTTP请求 %s 受限: %s", req.URL.String(), reason))
return nil, fmt.Errorf("发包受限: %s", reason)
LogError(i18n.Tr("http_request_restricted", req.URL.String(), reason))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", reason))
}
// 执行HTTP请求
+52
View File
@@ -0,0 +1,52 @@
package common
import (
"context"
"net/http"
"testing"
"github.com/shadow1ng/fscan/common/proxy"
)
func TestNetworkFacadeProxyState(t *testing.T) {
t.Cleanup(func() { proxy.AutoConfigureProxy(proxy.DefaultProxyConfig()) })
proxy.AutoConfigureProxy(proxy.DefaultProxyConfig())
if IsProxyEnabled() || IsSOCKS5Proxy() || !IsProxyReliable() {
t.Fatal("direct global proxy state should be disabled and reliable")
}
proxy.AutoConfigureProxy(&proxy.ProxyConfig{Type: proxy.ProxyTypeSOCKS5})
if !IsProxyEnabled() || !IsSOCKS5Proxy() || !IsProxyReliable() {
t.Fatal("SOCKS5 global proxy state should be enabled and SOCKS5")
}
}
func TestSafeHTTPDoUsesGlobalPacketLimit(t *testing.T) {
previousConfig := GetGlobalConfig()
previousState := GetGlobalState()
t.Cleanup(func() {
SetGlobalConfig(previousConfig)
SetGlobalState(previousState)
})
cfg := NewConfig()
cfg.Network.MaxPacketCount = 1
state := NewState()
state.IncrementPacketCount()
SetGlobalConfig(cfg)
SetGlobalState(state)
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
t.Fatal("transport should not be called when packet limit is reached")
return nil, nil
})}
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, "http://example.com", nil)
if err != nil {
t.Fatal(err)
}
if resp, err := SafeHTTPDo(client, req); err == nil || resp != nil {
t.Fatalf("SafeHTTPDo = resp %#v err %v, want limit error", resp, err)
}
}
+30 -10
View File
@@ -1,9 +1,6 @@
package output
import (
"fmt"
"sync"
)
import "sync"
// ResultBuffer 公共的去重缓冲逻辑,供各Writer复用
type ResultBuffer struct {
@@ -25,10 +22,10 @@ type ResultBuffer struct {
// NewResultBuffer 创建新的结果缓冲
func NewResultBuffer() *ResultBuffer {
return &ResultBuffer{
seenHosts: make(map[string]struct{}),
seenPorts: make(map[string]struct{}),
seenServices: make(map[string]int),
seenVulns: make(map[string]struct{}),
seenHosts: make(map[string]struct{}, 256),
seenPorts: make(map[string]struct{}, 512),
seenServices: make(map[string]int, 128),
seenVulns: make(map[string]struct{}, 64),
}
}
@@ -59,7 +56,8 @@ func (b *ResultBuffer) Add(result *ScanResult) {
b.seenServices[key] = len(b.ServiceResults)
b.ServiceResults = append(b.ServiceResults, result)
} else {
// 保留信息更完整的记录
b.mergeDetails(b.ServiceResults[idx], result)
// 保留信息更完整的记录,同时保留另一条记录补充的字段
if b.isMoreComplete(result, b.ServiceResults[idx]) {
b.ServiceResults[idx] = result
}
@@ -72,6 +70,28 @@ func (b *ResultBuffer) Add(result *ScanResult) {
}
}
func (b *ResultBuffer) mergeDetails(oldResult, newResult *ScanResult) {
if oldResult == nil || newResult == nil {
return
}
if oldResult.Details == nil {
oldResult.Details = make(map[string]interface{})
}
if newResult.Details == nil {
newResult.Details = make(map[string]interface{})
}
for k, v := range oldResult.Details {
if _, exists := newResult.Details[k]; !exists {
newResult.Details[k] = v
}
}
for k, v := range newResult.Details {
if _, exists := oldResult.Details[k]; !exists {
oldResult.Details[k] = v
}
}
}
// generateKey 生成结果的唯一键(用于去重)
func (b *ResultBuffer) generateKey(result *ScanResult) string {
switch result.Type {
@@ -80,7 +100,7 @@ func (b *ResultBuffer) generateKey(result *ScanResult) string {
case TypePort:
if result.Details != nil {
if port, ok := result.Details["port"]; ok {
return fmt.Sprintf("%s:%v", result.Target, port)
return targetWithPort(result.Target, port)
}
}
return result.Target
+34
View File
@@ -226,6 +226,40 @@ func TestResultBuffer_ServiceUpdate(t *testing.T) {
}
}
func TestResultBuffer_ServiceUpdateMergesDetails(t *testing.T) {
buf := NewResultBuffer()
buf.Add(&ScanResult{
Type: TypeService,
Target: "192.168.1.1:80",
Status: "identified",
Details: map[string]interface{}{
"service": "http",
"banner": "HTTP/1.1 200 OK",
},
})
buf.Add(&ScanResult{
Type: TypeService,
Target: "192.168.1.1:80",
Status: "web",
Details: map[string]interface{}{
"title": "Home",
"status": 200,
"server": "nginx",
},
})
if len(buf.ServiceResults) != 1 {
t.Fatalf("期望1条服务记录,实际 %d", len(buf.ServiceResults))
}
details := buf.ServiceResults[0].Details
for _, key := range []string{"service", "banner", "title", "status", "server"} {
if _, ok := details[key]; !ok {
t.Errorf("合并后的服务记录缺少字段 %q: %#v", key, details)
}
}
}
// TestResultBuffer_ServiceNoDowngrade 测试不降级服务记录
//
// 当新记录不如旧记录完整时,不应替换
+157
View File
@@ -0,0 +1,157 @@
package output
import (
"bufio"
"encoding/json"
"fmt"
"net"
"os"
"strconv"
"strings"
"sync"
)
type StdoutNDJSONWriter struct {
mu sync.Mutex
writer *bufio.Writer
}
func NewStdoutNDJSONWriter() *StdoutNDJSONWriter {
return &StdoutNDJSONWriter{
writer: bufio.NewWriter(os.Stdout),
}
}
// ndjsonRecord NDJSON 输出的扁平化结构
type ndjsonRecord struct {
Type ResultType `json:"type"`
Target string `json:"target"`
Status string `json:"status"`
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Service string `json:"service,omitempty"`
// 通用可选字段
Protocol string `json:"protocol,omitempty"`
Banner string `json:"banner,omitempty"`
Title string `json:"title,omitempty"`
URL string `json:"url,omitempty"`
// 漏洞/弱口令
Vulnerability string `json:"vulnerability,omitempty"`
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
// 其他
Plugin string `json:"plugin,omitempty"`
Version string `json:"version,omitempty"`
OS string `json:"os,omitempty"`
}
func (w *StdoutNDJSONWriter) WriteResult(result *ScanResult) error {
w.mu.Lock()
defer w.mu.Unlock()
rec := w.flatten(result)
data, err := json.Marshal(rec)
if err != nil {
return err
}
data = append(data, '\n')
if _, err := w.writer.Write(data); err != nil {
return err
}
return w.writer.Flush()
}
func (w *StdoutNDJSONWriter) flatten(r *ScanResult) *ndjsonRecord {
rec := &ndjsonRecord{
Type: r.Type,
Target: r.Target,
Status: r.Status,
}
// 从 target 拆分 host:port
if host, port, ok := splitHostPort(r.Target); ok {
rec.Host = host
rec.Port = port
} else {
rec.Host = r.Target
}
d := r.Details
if d == nil {
return rec
}
// 从 details 提升一级字段(覆盖拆分结果)
if v, ok := d["port"]; ok {
if p, ok := toInt(v); ok {
rec.Port = p
}
}
rec.Service = strVal(d, "service")
rec.Protocol = strVal(d, "protocol")
if banner := strVal(d, "banner"); len(banner) > 200 {
rec.Banner = banner[:200] + "..."
} else {
rec.Banner = banner
}
rec.Title = strVal(d, "title")
rec.URL = strVal(d, "url")
rec.Vulnerability = strVal(d, "vulnerability")
rec.Username = strVal(d, "username")
rec.Password = strVal(d, "password")
rec.Plugin = strVal(d, "plugin")
rec.Version = strVal(d, "version")
rec.OS = strVal(d, "os")
return rec
}
func (w *StdoutNDJSONWriter) Close() error {
w.mu.Lock()
defer w.mu.Unlock()
return w.writer.Flush()
}
func strVal(d map[string]interface{}, key string) string {
v, ok := d[key]
if !ok {
return ""
}
s, ok := v.(string)
if !ok {
return fmt.Sprintf("%v", v)
}
return s
}
func toInt(v interface{}) (int, bool) {
switch n := v.(type) {
case int:
return n, true
case int64:
return int(n), true
case float64:
return int(n), true
}
return 0, false
}
func splitHostPort(target string) (string, int, bool) {
host, portText, err := net.SplitHostPort(target)
if err != nil {
if strings.Count(target, ":") != 1 {
return "", 0, false
}
parts := strings.SplitN(target, ":", 2)
host, portText = parts[0], parts[1]
}
port, err := strconv.Atoi(portText)
if err != nil {
return "", 0, false
}
if host == "" || port < 1 || port > 65535 {
return "", 0, false
}
return host, port, true
}
+122
View File
@@ -0,0 +1,122 @@
package output
import (
"bufio"
"bytes"
"encoding/json"
"testing"
)
func TestSplitHostPort(t *testing.T) {
tests := []struct {
name string
target string
wantHost string
wantPort int
wantOK bool
}{
{name: "ipv4", target: "192.168.1.1:80", wantHost: "192.168.1.1", wantPort: 80, wantOK: true},
{name: "hostname", target: "example.com:443", wantHost: "example.com", wantPort: 443, wantOK: true},
{name: "bracketed ipv6", target: "[2001:db8::1]:8443", wantHost: "2001:db8::1", wantPort: 8443, wantOK: true},
{name: "bare ipv6 without port", target: "2001:db8::1", wantOK: false},
{name: "invalid port", target: "example.com:abc", wantOK: false},
{name: "port out of range", target: "example.com:65536", wantOK: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
host, port, ok := splitHostPort(tt.target)
if ok != tt.wantOK {
t.Fatalf("splitHostPort(%q) ok = %v, want %v", tt.target, ok, tt.wantOK)
}
if !ok {
return
}
if host != tt.wantHost || port != tt.wantPort {
t.Fatalf("splitHostPort(%q) = (%q, %d), want (%q, %d)", tt.target, host, port, tt.wantHost, tt.wantPort)
}
})
}
}
func TestNewStdoutNDJSONWriter(t *testing.T) {
writer := NewStdoutNDJSONWriter()
if writer == nil || writer.writer == nil {
t.Fatalf("NewStdoutNDJSONWriter = %#v, want initialized writer", writer)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
}
func TestStdoutNDJSONWriterWriteResult(t *testing.T) {
var buf bytes.Buffer
writer := &StdoutNDJSONWriter{writer: bufio.NewWriter(&buf)}
result := &ScanResult{
Type: TypeService,
Target: "[2001:db8::1]:8443",
Status: "OPEN",
Details: map[string]interface{}{
"port": float64(9443),
"service": "https",
"protocol": "tcp",
"banner": 123,
"title": "admin",
"url": "https://[2001:db8::1]:8443",
"vulnerability": "weak credential",
"username": "admin",
"password": "secret",
"plugin": "webtitle",
"version": "1.2.3",
"os": "linux",
},
}
if err := writer.WriteResult(result); err != nil {
t.Fatalf("WriteResult error = %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
var rec ndjsonRecord
if err := json.Unmarshal(bytes.TrimSpace(buf.Bytes()), &rec); err != nil {
t.Fatalf("invalid ndjson output %q: %v", buf.String(), err)
}
if rec.Host != "2001:db8::1" || rec.Port != 9443 {
t.Fatalf("host/port = %q/%d", rec.Host, rec.Port)
}
if rec.Service != "https" || rec.Protocol != "tcp" || rec.Banner != "123" || rec.Title != "admin" {
t.Fatalf("flattened fields missing: %#v", rec)
}
if rec.URL != "https://[2001:db8::1]:8443" || rec.Vulnerability != "weak credential" {
t.Fatalf("url/vuln fields missing: %#v", rec)
}
if rec.Username != "admin" || rec.Password != "secret" || rec.Plugin != "webtitle" || rec.Version != "1.2.3" || rec.OS != "linux" {
t.Fatalf("credential/plugin fields missing: %#v", rec)
}
}
func TestStdoutNDJSONFlattenFallbacks(t *testing.T) {
writer := &StdoutNDJSONWriter{writer: bufio.NewWriter(&bytes.Buffer{})}
rec := writer.flatten(&ScanResult{
Type: TypeHost,
Target: "2001:db8::1",
Status: "ALIVE",
Details: map[string]interface{}{
"port": int64(22),
},
})
if rec.Host != "2001:db8::1" || rec.Port != 22 {
t.Fatalf("flatten fallback = %#v", rec)
}
if got, ok := toInt("22"); ok || got != 0 {
t.Fatalf("toInt string = %d/%v, want 0/false", got, ok)
}
if got := strVal(map[string]interface{}{}, "missing"); got != "" {
t.Fatalf("missing strVal = %q, want empty", got)
}
}
+124 -61
View File
@@ -5,21 +5,70 @@ import (
"encoding/csv"
"encoding/json"
"fmt"
"net"
"os"
"strings"
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
)
// escapeControlChars 转义控制字符
func escapeControlChars(s string) string {
replacer := strings.NewReplacer(
"\r\n", "\\r\\n",
"\n", "\\n",
"\r", "\\r",
"\t", "\\t",
)
return replacer.Replace(s)
s = strings.ToValidUTF8(s, "?")
var b strings.Builder
for _, r := range s {
switch r {
case '\n':
b.WriteString("\\n")
case '\r':
b.WriteString("\\r")
case '\t':
b.WriteString("\\t")
default:
if r < 0x20 || r == 0x7f {
fmt.Fprintf(&b, "\\x%02x", r)
continue
}
b.WriteRune(r)
}
}
return b.String()
}
func truncateString(s string, maxRunes int) string {
if maxRunes < 0 {
return s
}
for i := range s {
if maxRunes == 0 {
return s[:i] + "..."
}
maxRunes--
}
return s
}
func targetWithPort(target string, port interface{}) string {
if port == nil {
return target
}
if _, _, err := net.SplitHostPort(target); err == nil {
return target
}
portText := fmt.Sprint(port)
if strings.TrimSpace(portText) == "" {
return target
}
if strings.HasPrefix(target, "[") && strings.HasSuffix(target, "]") {
target = strings.TrimPrefix(strings.TrimSuffix(target, "]"), "[")
}
if strings.Count(target, ":") == 1 {
return target
}
return net.JoinHostPort(target, portText)
}
// =============================================================================
@@ -99,13 +148,13 @@ func (w *TXTWriter) Write(result *ScanResult) error {
func (w *TXTWriter) getSeparator(newType ResultType) string {
switch newType {
case TypeHost:
return "# ===== 存活主机 ====="
return i18n.GetText("output_section_hosts")
case TypePort:
return "# ===== 开放端口 ====="
return i18n.GetText("output_section_ports")
case TypeService:
return "# ===== 服务信息 ====="
return i18n.GetText("output_section_services")
case TypeVuln:
return "# ===== 漏洞信息 ====="
return i18n.GetText("output_section_vulns")
default:
return "# ===================="
}
@@ -119,7 +168,7 @@ func (w *TXTWriter) formatLine(result *ScanResult) string {
case TypePort:
port := w.getDetail(result, "port")
if port != nil {
return fmt.Sprintf("%s:%v", result.Target, port)
return targetWithPort(result.Target, port)
}
return result.Target
case TypeService:
@@ -152,12 +201,7 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
}
// 非Web服务:ip:port service banner
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
var parts []string
parts = append(parts, target)
@@ -165,10 +209,8 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
parts = append(parts, service)
}
if banner != "" {
if len(banner) > 100 {
banner = banner[:100] + "..."
}
banner = escapeControlChars(banner)
banner = truncateString(banner, 100)
parts = append(parts, banner)
}
return strings.Join(parts, " ")
@@ -176,20 +218,9 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
// formatWebServiceLine 格式化Web服务结果
func (w *TXTWriter) formatWebServiceLine(result *ScanResult) string {
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
protocol := "http"
service := w.getDetailStr(result, "service")
if service == "https" || strings.Contains(target, ":443") {
protocol = "https"
}
url := fmt.Sprintf("%s://%s", protocol, target)
url := fmt.Sprintf("%s://%s", w.webProtocol(result, target), target)
title := w.getDetailStr(result, "title")
status := w.getDetail(result, "status")
server := w.getDetailStr(result, "server")
@@ -319,13 +350,17 @@ func (w *TXTWriter) Close() error {
os.Remove(w.realtimePath)
}
var firstErr error
if err := w.bufWriter.Flush(); err != nil {
return err
firstErr = err
}
if err := w.file.Sync(); err != nil {
return err
if err := w.file.Sync(); err != nil && firstErr == nil {
firstErr = err
}
return w.file.Close()
if err := w.file.Close(); err != nil && firstErr == nil {
firstErr = err
}
return firstErr
}
// writeSection 写入一个分类的所有结果
@@ -355,27 +390,16 @@ func (w *TXTWriter) writeWebServices() {
continue
}
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
protocol := "http"
service := w.getDetailStr(result, "service")
if service == "https" || strings.Contains(target, ":443") {
protocol = "https"
}
urls = append(urls, fmt.Sprintf("%s://%s", protocol, target))
urls = append(urls, fmt.Sprintf("%s://%s", w.webProtocol(result, target), target))
}
if len(urls) == 0 {
return
}
_, _ = w.bufWriter.WriteString("# ===== Web服务 =====\n")
_, _ = w.bufWriter.WriteString(i18n.GetText("output_section_web_services") + "\n")
for _, url := range urls {
_, _ = w.bufWriter.WriteString(url + "\n")
}
@@ -397,6 +421,19 @@ func (w *TXTWriter) isWebService(result *ScanResult) bool {
return service == "http" || service == "https"
}
func (w *TXTWriter) webProtocol(result *ScanResult, target string) string {
protocol := strings.ToLower(w.getDetailStr(result, "protocol"))
if protocol == "http" || protocol == "https" {
return protocol
}
service := strings.ToLower(w.getDetailStr(result, "service"))
if service == "https" || strings.Contains(target, ":443") {
return "https"
}
return "http"
}
// GetFormat 获取格式类型
func (w *TXTWriter) GetFormat() Format {
return FormatTXT
@@ -647,7 +684,7 @@ func (w *CSVWriter) Close() error {
// 写入各分类
w.writeSection("# Hosts", []string{"Target"}, w.buffer.HostResults, w.formatHostRecord)
w.writeSection("# Ports", []string{"Target", "Port", "Status"}, w.buffer.PortResults, w.formatPortRecord)
w.writeSection("# Services", []string{"Target", "Service", "Version", "Banner"}, w.buffer.ServiceResults, w.formatServiceRecord)
w.writeSection("# Services", []string{"Target", "Service", "Version", "Title", "Status", "Server", "Fingerprints", "Banner"}, w.buffer.ServiceResults, w.formatServiceRecord)
w.writeSection("# Vulns", []string{"Target", "Type", "Details"}, w.buffer.VulnResults, w.formatVulnRecord)
w.closed = true
@@ -697,7 +734,7 @@ func (w *CSVWriter) formatPortRecord(result *ScanResult) []string {
}
func (w *CSVWriter) formatServiceRecord(result *ScanResult) []string {
service, version, banner := "", "", ""
service, version, title, status, server, fingerprints, banner := "", "", "", "", "", "", ""
if result.Details != nil {
if s, ok := result.Details["service"].(string); ok {
service = s
@@ -705,23 +742,49 @@ func (w *CSVWriter) formatServiceRecord(result *ScanResult) []string {
if s, ok := result.Details["name"].(string); ok && service == "" {
service = s
}
if s, ok := result.Details["plugin"].(string); ok && service == "" {
service = s
}
if v, ok := result.Details["version"].(string); ok {
version = v
}
if t, ok := result.Details["title"].(string); ok {
title = escapeControlChars(t)
}
if s, ok := result.Details["status"]; ok && s != nil && s != 0 {
status = fmt.Sprintf("%v", s)
}
if s, ok := result.Details["server"].(string); ok {
server = escapeControlChars(s)
}
fingerprints = formatFingerprints(result.Details["fingerprints"])
if b, ok := result.Details["banner"].(string); ok {
banner = escapeControlChars(b)
if len(banner) > 100 {
banner = banner[:100] + "..."
}
banner = truncateString(banner, 100)
}
}
target := result.Target
if !strings.Contains(target, ":") {
if p, ok := result.Details["port"]; ok {
target = fmt.Sprintf("%s:%v", target, p)
if result.Details != nil {
target = targetWithPort(target, result.Details["port"])
}
return []string{target, service, version, title, status, server, fingerprints, banner}
}
func formatFingerprints(value interface{}) string {
switch v := value.(type) {
case []string:
return strings.Join(v, ",")
case []interface{}:
parts := make([]string, 0, len(v))
for _, item := range v {
if s, ok := item.(string); ok && s != "" {
parts = append(parts, s)
}
}
return strings.Join(parts, ",")
default:
return ""
}
return []string{target, service, version, banner}
}
func (w *CSVWriter) formatVulnRecord(result *ScanResult) []string {
+195 -1
View File
@@ -9,6 +9,7 @@ import (
"sync"
"testing"
"time"
"unicode/utf8"
)
/*
@@ -57,6 +58,119 @@ func createTestResult(resultType ResultType, target, status string, details map[
}
}
func TestTargetWithPortIPv6(t *testing.T) {
tests := []struct {
name string
target string
port interface{}
want string
}{
{name: "ipv4 without port", target: "192.168.1.1", port: 80, want: "192.168.1.1:80"},
{name: "ipv4 with port", target: "192.168.1.1:80", port: 443, want: "192.168.1.1:80"},
{name: "ipv6 without port", target: "2001:db8::1", port: 443, want: "[2001:db8::1]:443"},
{name: "bracketed ipv6 without port", target: "[2001:db8::1]", port: 443, want: "[2001:db8::1]:443"},
{name: "ipv6 with port", target: "[2001:db8::1]:443", port: 80, want: "[2001:db8::1]:443"},
{name: "empty port", target: "example.com", port: "", want: "example.com"},
{name: "blank port", target: "example.com", port: " \t", want: "example.com"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := targetWithPort(tt.target, tt.port); got != tt.want {
t.Fatalf("targetWithPort(%q, %v) = %q, want %q", tt.target, tt.port, got, tt.want)
}
})
}
}
func TestScanResultFormatDetailsAndDefaultManagerConfig(t *testing.T) {
result := &ScanResult{
Details: map[string]interface{}{
"service": "ssh",
"port": 22,
"banner": "OpenSSH",
},
}
got := result.FormatDetails(";", "%s=%v")
want := "banner=OpenSSH;port=22;service=ssh"
if got != want {
t.Fatalf("FormatDetails = %q, want %q", got, want)
}
empty := (&ScanResult{}).FormatDetails(";", "%s=%v")
if empty != "" {
t.Fatalf("empty FormatDetails = %q, want empty", empty)
}
cfg := DefaultManagerConfig("out.json", FormatJSON)
if cfg.OutputPath != "out.json" || cfg.Format != FormatJSON {
t.Fatalf("DefaultManagerConfig = %#v", cfg)
}
}
func TestCSVWriterFormatRecords(t *testing.T) {
writer := &CSVWriter{}
host := writer.formatHostRecord(&ScanResult{Target: "192.168.1.1"})
if len(host) != 1 || host[0] != "192.168.1.1" {
t.Fatalf("host record = %#v", host)
}
port := writer.formatPortRecord(&ScanResult{
Target: "192.168.1.1",
Details: map[string]interface{}{"port": 22},
})
if got, want := strings.Join(port, "|"), "192.168.1.1|22|open"; got != want {
t.Fatalf("port record = %q, want %q", got, want)
}
longBanner := strings.Repeat("界", 105)
service := writer.formatServiceRecord(&ScanResult{
Target: "2001:db8::1",
Details: map[string]interface{}{
"port": 443,
"name": "https",
"version": "1.2.3",
"title": "hello\nworld",
"status": 200,
"server": "nginx\r\nunit",
"fingerprints": []interface{}{"fp1", "", "fp2", 3},
"banner": longBanner,
},
})
if service[0] != "[2001:db8::1]:443" || service[1] != "https" || service[2] != "1.2.3" {
t.Fatalf("service identity fields = %#v", service)
}
if service[3] != "hello\\nworld" || service[4] != "200" || service[5] != "nginx\\r\\nunit" {
t.Fatalf("service text fields = %#v", service)
}
if service[6] != "fp1,fp2" {
t.Fatalf("fingerprints = %q, want fp1,fp2", service[6])
}
if !utf8.ValidString(service[7]) || len([]rune(service[7])) != 103 || !strings.HasSuffix(service[7], "...") {
t.Fatalf("truncated banner = len %d value %q", len(service[7]), service[7])
}
vuln := writer.formatVulnRecord(&ScanResult{
Target: "http://example.com",
Status: "vulnerable",
Details: map[string]interface{}{"type": "poc"},
})
if got, want := strings.Join(vuln, "|"), "http://example.com|poc|vulnerable"; got != want {
t.Fatalf("vuln record = %q, want %q", got, want)
}
if got := formatFingerprints([]string{"a", "b"}); got != "a,b" {
t.Fatalf("string fingerprints = %q", got)
}
if got := formatFingerprints(123); got != "" {
t.Fatalf("unsupported fingerprints = %q, want empty", got)
}
if writer.GetFormat() != FormatCSV {
t.Fatalf("csv GetFormat = %q", writer.GetFormat())
}
}
// =============================================================================
// TXTWriter - 基础功能测试
// =============================================================================
@@ -1139,7 +1253,7 @@ func TestCSVWriter_ErrorHandling(t *testing.T) {
// TestCSVWriter_DetailsFormatting 测试CSV的Details字段格式化
//
// CSVWriter 对不同类型有不同的格式:
// - Service类型:Target, Service, Version, Banner
// - Service类型:Target, Service, Version, Title, Status, Server, Fingerprints, Banner
func TestCSVWriter_DetailsFormatting(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test.csv")
@@ -1188,6 +1302,86 @@ func TestCSVWriter_DetailsFormatting(t *testing.T) {
t.Logf("✓ CSV Details格式化测试通过")
}
func TestCSVWriter_WebServiceFields(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test.csv")
writer, _ := NewCSVWriter(filePath)
defer func() { _ = writer.Close() }()
_ = writer.WriteHeader()
result := createTestResult(
TypeService,
"192.168.1.1:80",
"web",
map[string]interface{}{
"plugin": "webtitle",
"is_web": true,
"port": 80,
"title": "Home",
"status": 200,
"server": "nginx",
"fingerprints": []string{"nginx", "php"},
"banner": "HTTP/1.1 200 OK\x00\nServer: nginx",
},
)
_ = writer.Write(result)
writer.Close()
content := readFileContent(t, filePath)
for _, want := range []string{
"Target,Service,Version,Title,Status,Server,Fingerprints,Banner",
"webtitle",
"Home",
"200",
"nginx",
"nginx,php",
"\\x00\\nServer: nginx",
} {
if !strings.Contains(content, want) {
t.Errorf("CSV文件缺少 %q,内容:\n%s", want, content)
}
}
}
func TestTXTWriter_WebServiceProtocolFromDetails(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test_web_protocol.txt")
writer, err := NewTXTWriter(filePath)
if err != nil {
t.Fatalf("创建TXTWriter失败: %v", err)
}
result := createTestResult(
TypeService,
"192.168.1.1:8443",
"web",
map[string]interface{}{
"plugin": "webtitle",
"is_web": true,
"port": 8443,
"protocol": "https",
"title": "Home",
"status": 200,
},
)
if err := writer.Write(result); err != nil {
t.Fatalf("Write()失败: %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close()失败: %v", err)
}
content := readFileContent(t, filePath)
if !strings.Contains(content, "https://192.168.1.1:8443") {
t.Fatalf("TXT输出缺少HTTPS URL,内容:\n%s", content)
}
if strings.Contains(content, "http://192.168.1.1:8443") {
t.Fatalf("TXT输出不应把HTTPS目标降级为HTTP,内容:\n%s", content)
}
}
// TestJSONWriter_FlushAndFormat 测试JSON的Flush和GetFormat
func TestJSONWriter_FlushAndFormat(t *testing.T) {
dir := createTestDir(t)
+17 -1
View File
@@ -15,10 +15,18 @@ import (
// ResultOutput 全局输出管理器
var ResultOutput *output.Manager
// StdoutWriter silent模式下的NDJSON stdout写入器
var StdoutWriter *output.StdoutNDJSONWriter
// InitOutput 初始化输出系统
func InitOutput() error {
fv := GetFlagVars()
// silent模式:初始化NDJSON stdout写入器(独立于文件输出)
if fv.Silent {
StdoutWriter = output.NewStdoutNDJSONWriter()
}
// 用户通过-no flag禁用保存时,跳过文件初始化避免不必要的资源开销
if fv.DisableSave {
return nil
@@ -59,6 +67,9 @@ func InitOutput() error {
// CloseOutput 关闭输出系统
func CloseOutput() error {
if StdoutWriter != nil {
_ = StdoutWriter.Close()
}
if ResultOutput == nil {
return nil
}
@@ -80,8 +91,13 @@ func SaveResult(result *output.ScanResult) error {
"details": result.Details,
})
// silent模式:NDJSON实时输出到stdout
if StdoutWriter != nil {
_ = StdoutWriter.WriteResult(result)
}
// 用户禁用保存或输出未初始化时,跳过文件保存
if GetFlagVars().DisableSave || ResultOutput == nil {
if GetGlobalConfig().Output.DisableSave || ResultOutput == nil {
return nil
}
return ResultOutput.SaveResult(result)
+153
View File
@@ -0,0 +1,153 @@
package common
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/shadow1ng/fscan/common/output"
)
func readTestFile(t *testing.T, path string) string {
t.Helper()
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return string(content)
}
func preserveOutputAPIGlobals(t *testing.T) {
t.Helper()
globalMu.RLock()
oldConfig := globalConfig
oldState := globalState
globalMu.RUnlock()
oldFlagVars := flagVars
oldResultOutput := ResultOutput
oldStdoutWriter := StdoutWriter
t.Cleanup(func() {
if ResultOutput != nil && ResultOutput != oldResultOutput {
_ = ResultOutput.Close()
}
if StdoutWriter != nil && StdoutWriter != oldStdoutWriter {
_ = StdoutWriter.Close()
}
ClearResultCallback()
globalMu.Lock()
globalConfig = oldConfig
globalState = oldState
globalMu.Unlock()
flagVars = oldFlagVars
ResultOutput = oldResultOutput
StdoutWriter = oldStdoutWriter
})
ClearResultCallback()
flagVars = &FlagVars{}
ResultOutput = nil
StdoutWriter = nil
SetGlobalConfig(NewConfig())
SetGlobalState(NewState())
}
func TestInitOutputValidationAndDefaultExtension(t *testing.T) {
preserveOutputAPIGlobals(t)
flagVars = &FlagVars{DisableSave: true}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput disable save error = %v", err)
}
if ResultOutput != nil {
t.Fatalf("ResultOutput = %#v, want nil when save is disabled", ResultOutput)
}
flagVars = &FlagVars{OutputFormat: "txt"}
if err := InitOutput(); err == nil || !strings.Contains(err.Error(), "output file not specified") {
t.Fatalf("missing output error = %v", err)
}
flagVars = &FlagVars{Outputfile: "out.bad", OutputFormat: "xml"}
if err := InitOutput(); err == nil || !strings.Contains(err.Error(), "invalid output format") {
t.Fatalf("invalid format error = %v", err)
}
dir := t.TempDir()
t.Chdir(dir)
flagVars = &FlagVars{Outputfile: "result.txt", OutputFormat: "json"}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput json error = %v", err)
}
if ResultOutput == nil {
t.Fatal("ResultOutput should be initialized")
}
if err := SaveResult(&output.ScanResult{
Time: time.Date(2026, 6, 13, 1, 2, 3, 0, time.UTC),
Type: output.TypeHost,
Target: "127.0.0.1",
Status: "ALIVE",
}); err != nil {
t.Fatalf("SaveResult json error = %v", err)
}
if err := CloseOutput(); err != nil {
t.Fatalf("CloseOutput error = %v", err)
}
if content := readTestFile(t, filepath.Join(dir, "result.json")); !strings.Contains(content, "127.0.0.1") {
t.Fatalf("result.json content = %q, want saved target", content)
}
}
func TestSaveResultFacadeCallbackAndDisabledSave(t *testing.T) {
preserveOutputAPIGlobals(t)
cfg := NewConfig()
cfg.Output.DisableSave = true
SetGlobalConfig(cfg)
flagVars = &FlagVars{DisableSave: true}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput disable save error = %v", err)
}
called := false
SetResultCallback(func(payload interface{}) {
called = true
data, ok := payload.(map[string]interface{})
if !ok {
t.Fatalf("callback payload type = %T", payload)
}
if data["type"] != string(output.TypeVuln) || data["target"] != "http://example.com" {
t.Fatalf("callback payload = %#v", data)
}
})
if err := SaveResult(nil); err != nil {
t.Fatalf("SaveResult nil error = %v", err)
}
if called {
t.Fatal("nil result should not notify callback")
}
if err := SaveResult(&output.ScanResult{
Type: output.TypeVuln,
Target: "http://example.com",
Status: "vulnerable",
Details: map[string]interface{}{"type": "poc"},
}); err != nil {
t.Fatalf("SaveResult disabled save error = %v", err)
}
if !called {
t.Fatal("callback was not notified")
}
if err := CloseOutput(); err != nil {
t.Fatalf("CloseOutput disabled save error = %v", err)
}
}
+3
View File
@@ -45,6 +45,9 @@ func applyLogLevel() {
StartTime: GetGlobalState().GetStartTime(),
LevelColors: logging.GetDefaultLevelColors(),
}
if fv.Debug {
config.DebugLogFile = "fscan_debug.log"
}
newLogger := logging.NewLogger(config)
newLogger.SetCoordinatedOutput(LogWithProgress)
-9
View File
@@ -21,15 +21,6 @@ const (
MaxPort = 65535
)
// =============================================================================
// IP/主机解析常量
// =============================================================================
const (
// SimpleMaxHosts 最大主机数量限制
SimpleMaxHosts = 10000
)
// =============================================================================
// 哈希验证常量
// =============================================================================
+490
View File
@@ -0,0 +1,490 @@
package parsers
import (
"bufio"
"context"
"fmt"
"net"
"os"
"strconv"
"strings"
"github.com/shadow1ng/fscan/common/i18n"
)
const DefaultHostBatchSize = 4096
type HostIterator struct {
sources []hostSource
current hostSource
exclude *hostMatcher
}
func NewHostIterator(host string, filename string, nohosts ...string) (*HostIterator, error) {
var sources []hostSource
if filename != "" {
fileSrc, err := newFileHostSource(filename)
if err != nil {
return nil, err
}
sources = append(sources, fileSrc)
}
hostSources, err := newHostSources(host)
if err != nil {
closeHostSources(sources)
return nil, err
}
sources = append(sources, hostSources...)
matcher := newHostMatcher()
for _, exclude := range nohosts {
if strings.TrimSpace(exclude) == "" {
continue
}
if err := matcher.add(exclude); err != nil {
closeHostSources(sources)
return nil, err
}
}
return &HostIterator{
sources: sources,
exclude: matcher,
}, nil
}
func (it *HostIterator) Close() error {
if it == nil {
return nil
}
var firstErr error
if it.current != nil {
firstErr = it.current.Close()
it.current = nil
}
for _, src := range it.sources {
if err := src.Close(); err != nil && firstErr == nil {
firstErr = err
}
}
it.sources = nil
return firstErr
}
func (it *HostIterator) Next() (string, bool, error) {
for {
if it.current == nil {
if len(it.sources) == 0 {
return "", false, nil
}
it.current = it.sources[0]
it.sources = it.sources[1:]
}
host, ok, err := it.current.Next()
if err != nil {
return "", false, err
}
if !ok {
if err := it.current.Close(); err != nil {
return "", false, err
}
it.current = nil
continue
}
if it.exclude != nil && it.exclude.match(host) {
continue
}
return host, true, nil
}
}
func (it *HostIterator) NextBatch(ctx context.Context, size int) ([]string, error) {
if size <= 0 {
size = DefaultHostBatchSize
}
batch := make([]string, 0, size)
seen := make(map[string]struct{}, size)
for len(batch) < size {
select {
case <-ctx.Done():
return batch, ctx.Err()
default:
}
host, ok, err := it.Next()
if err != nil {
return batch, err
}
if !ok {
return batch, nil
}
if _, exists := seen[host]; exists {
continue
}
seen[host] = struct{}{}
batch = append(batch, host)
}
return batch, nil
}
type hostSource interface {
Next() (string, bool, error)
Close() error
}
type singleHostSource struct {
host string
done bool
}
func (s *singleHostSource) Next() (string, bool, error) {
if s.done {
return "", false, nil
}
s.done = true
return s.host, true, nil
}
func (s *singleHostSource) Close() error { return nil }
type cidrHostSource struct {
current uint32
end uint32
done bool
}
func (s *cidrHostSource) Next() (string, bool, error) {
if s.done || s.current > s.end {
return "", false, nil
}
host := uint32ToIP(s.current)
if s.current == s.end {
s.done = true
} else {
s.current++
}
return host, true, nil
}
func (s *cidrHostSource) Close() error { return nil }
type fileHostSource struct {
file *os.File
scanner *bufio.Scanner
current hostSource
}
func newFileHostSource(filename string) (*fileHostSource, error) {
file, err := os.Open(filename)
if err != nil {
return nil, err
}
src := &fileHostSource{
file: file,
scanner: bufio.NewScanner(file),
}
src.scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
return src, nil
}
func (s *fileHostSource) Next() (string, bool, error) {
for {
if s.current != nil {
host, ok, err := s.current.Next()
if err != nil {
return "", false, err
}
if ok {
return host, true, nil
}
_ = s.current.Close()
s.current = nil
}
if !s.scanner.Scan() {
if err := s.scanner.Err(); err != nil {
return "", false, err
}
return "", false, nil
}
line := strings.TrimSpace(s.scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
sources, err := newHostSources(line)
if err != nil || len(sources) == 0 {
continue
}
if len(sources) == 1 {
s.current = sources[0]
continue
}
s.current = &multiHostSource{sources: sources}
}
}
func (s *fileHostSource) Close() error {
if s.current != nil {
_ = s.current.Close()
s.current = nil
}
if s.file == nil {
return nil
}
err := s.file.Close()
s.file = nil
return err
}
type multiHostSource struct {
sources []hostSource
current hostSource
}
func (s *multiHostSource) Next() (string, bool, error) {
for {
if s.current == nil {
if len(s.sources) == 0 {
return "", false, nil
}
s.current = s.sources[0]
s.sources = s.sources[1:]
}
host, ok, err := s.current.Next()
if err != nil {
return "", false, err
}
if ok {
return host, true, nil
}
_ = s.current.Close()
s.current = nil
}
}
func (s *multiHostSource) Close() error {
if s.current != nil {
_ = s.current.Close()
s.current = nil
}
closeHostSources(s.sources)
s.sources = nil
return nil
}
func newHostSources(host string) ([]hostSource, error) {
var sources []hostSource
for _, h := range strings.Split(host, ",") {
h = strings.TrimSpace(h)
if h == "" {
continue
}
src, err := newHostSource(h)
if err != nil {
closeHostSources(sources)
return nil, err
}
sources = append(sources, src)
}
return sources, nil
}
func newHostSource(host string) (hostSource, error) {
switch {
case host == "192":
return newCIDRHostSource("192.168.0.0/16")
case host == "172":
return newCIDRHostSource("172.16.0.0/12")
case host == "10":
return newCIDRHostSource("10.0.0.0/8")
case strings.Contains(host, "/"):
src, err := newCIDRHostSource(host)
if err != nil {
return nil, fmt.Errorf(i18n.Tr("parser_cidr_failed", host)+": %w", err)
}
return src, nil
case strings.Contains(host, "-") && !strings.Contains(host, ":") && looksLikeIPRange(host):
src, err := newRangeHostSource(host)
if err != nil {
return nil, fmt.Errorf(i18n.Tr("parser_ip_range_failed", host)+": %w", err)
}
return src, nil
default:
return &singleHostSource{host: host}, nil
}
}
func newCIDRHostSource(cidr string) (hostSource, error) {
_, ipNet, err := net.ParseCIDR(cidr)
if err != nil {
return nil, err
}
start, ok := ipToUint32(ipNet.IP)
if !ok {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
ones, bits := ipNet.Mask.Size()
if bits != 32 {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
size := uint64(1) << uint(32-ones)
end := start + uint32(size-1)
if size > 2 {
start++
end--
}
return &cidrHostSource{current: start, end: end}, nil
}
func newRangeHostSource(rangeStr string) (hostSource, error) {
parts := strings.Split(rangeStr, "-")
if len(parts) != 2 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_fmt", rangeStr))
}
startIPStr := strings.TrimSpace(parts[0])
endIPStr := strings.TrimSpace(parts[1])
startIP := net.ParseIP(startIPStr)
if startIP == nil {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_start_ip", startIPStr))
}
if len(endIPStr) < 4 || !strings.Contains(endIPStr, ".") {
endNum, err := strconv.Atoi(endIPStr)
if err != nil || endNum > 255 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_end_val", endIPStr))
}
parts := strings.Split(startIPStr, ".")
if len(parts) != 4 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_fmt", startIPStr))
}
parts[3] = strconv.Itoa(endNum)
endIPStr = strings.Join(parts, ".")
}
start, ok := ipToUint32(startIP)
if !ok {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
end, ok := ipToUint32(net.ParseIP(endIPStr))
if !ok {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_end_ip", endIPStr))
}
if start > end {
return nil, fmt.Errorf("%s", i18n.GetText("parser_start_gt_end"))
}
return &cidrHostSource{current: start, end: end}, nil
}
func closeHostSources(sources []hostSource) {
for _, src := range sources {
_ = src.Close()
}
}
type hostMatcher struct {
exact map[string]struct{}
ranges []ipRange
}
type ipRange struct {
start uint32
end uint32
}
func newHostMatcher() *hostMatcher {
return &hostMatcher{exact: make(map[string]struct{})}
}
func (m *hostMatcher) add(input string) error {
for _, h := range strings.Split(input, ",") {
h = strings.TrimSpace(h)
if h == "" {
continue
}
switch {
case h == "192":
if err := m.addCIDR("192.168.0.0/16"); err != nil {
return err
}
case h == "172":
if err := m.addCIDR("172.16.0.0/12"); err != nil {
return err
}
case h == "10":
if err := m.addCIDR("10.0.0.0/8"); err != nil {
return err
}
case strings.Contains(h, "/"):
if err := m.addCIDR(h); err != nil {
return err
}
case strings.Contains(h, "-") && !strings.Contains(h, ":") && looksLikeIPRange(h):
if err := m.addRange(h); err != nil {
return err
}
default:
m.exact[h] = struct{}{}
}
}
return nil
}
func (m *hostMatcher) addCIDR(cidr string) error {
src, err := newCIDRHostSource(cidr)
if err != nil {
return err
}
rangeSrc, ok := src.(*cidrHostSource)
if !ok {
return fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
m.ranges = append(m.ranges, ipRange{start: rangeSrc.current, end: rangeSrc.end})
return nil
}
func (m *hostMatcher) addRange(rangeStr string) error {
src, err := newRangeHostSource(rangeStr)
if err != nil {
return err
}
rangeSrc, ok := src.(*cidrHostSource)
if !ok {
return fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
m.ranges = append(m.ranges, ipRange{start: rangeSrc.current, end: rangeSrc.end})
return nil
}
func (m *hostMatcher) match(host string) bool {
if _, ok := m.exact[host]; ok {
return true
}
ip, ok := ipToUint32(net.ParseIP(host))
if !ok {
return false
}
for _, r := range m.ranges {
if ip >= r.start && ip <= r.end {
return true
}
}
return false
}
func ipToUint32(ip net.IP) (uint32, bool) {
ip4 := ip.To4()
if ip4 == nil {
return 0, false
}
return uint32(ip4[0])<<24 | uint32(ip4[1])<<16 | uint32(ip4[2])<<8 | uint32(ip4[3]), true
}
func uint32ToIP(v uint32) string {
return fmt.Sprintf("%d.%d.%d.%d", byte(v>>24), byte(v>>16), byte(v>>8), byte(v))
}
+168
View File
@@ -0,0 +1,168 @@
package parsers
import (
"context"
"errors"
"os"
"reflect"
"strings"
"testing"
)
func TestHostIteratorCIDRBatch(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/30", "", "")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.2"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorDoesNotExpandWholeRangeAtOnce(t *testing.T) {
iter, err := NewHostIterator("10", "", "")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 3)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"10.0.0.1", "10.0.0.2", "10.0.0.3"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorExcludeCIDR(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/29", "", "192.168.1.2-192.168.1.4")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.5", "192.168.1.6"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorAcceptsMultipleExcludeSources(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/29", "", "192.168.1.2", "192.168.1.5")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.3", "192.168.1.4", "192.168.1.6"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorReadsLongHostFileLine(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
longPrefix := strings.Repeat("a", 70*1024)
host := longPrefix + ".example.com"
if err := os.WriteFile(path, []byte(host+"\n"), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
iter, err := NewHostIterator("", path)
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 1)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
if !reflect.DeepEqual(batch, []string{host}) {
t.Fatalf("batch = %#v, want long host", batch)
}
}
func TestMultiHostSourceAndMatcherCIDR(t *testing.T) {
src := &multiHostSource{sources: []hostSource{
&singleHostSource{host: "192.168.1.1"},
&singleHostSource{host: "192.168.1.2"},
}}
host, ok, err := src.Next()
if err != nil || !ok || host != "192.168.1.1" {
t.Fatalf("first Next = %q/%v/%v", host, ok, err)
}
host, ok, err = src.Next()
if err != nil || !ok || host != "192.168.1.2" {
t.Fatalf("second Next = %q/%v/%v", host, ok, err)
}
host, ok, err = src.Next()
if err != nil || ok || host != "" {
t.Fatalf("exhausted Next = %q/%v/%v", host, ok, err)
}
if err := src.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
matcher := newHostMatcher()
if err := matcher.add("192.168.1.0/30,example.com"); err != nil {
t.Fatalf("matcher add error = %v", err)
}
if !matcher.match("192.168.1.1") || !matcher.match("192.168.1.2") || !matcher.match("example.com") {
t.Fatal("matcher should match CIDR hosts and exact host")
}
if matcher.match("192.168.1.3") || matcher.match("nope.example") {
t.Fatal("matcher matched hosts outside its rules")
}
if err := matcher.add("2001:db8::/126"); err == nil {
t.Fatal("IPv6 CIDR should be rejected by IPv4-only matcher")
}
}
func TestCloseHostSourcesIgnoresCloseErrors(t *testing.T) {
first := &closeTrackingSource{err: errors.New("close failed")}
second := &closeTrackingSource{}
closeHostSources([]hostSource{first, second})
if !first.closed || !second.closed {
t.Fatalf("sources closed = %v/%v, want both true", first.closed, second.closed)
}
}
type closeTrackingSource struct {
closed bool
err error
}
func (s *closeTrackingSource) Next() (string, bool, error) {
return "", false, nil
}
func (s *closeTrackingSource) Close() error {
s.closed = true
return s.err
}
+57 -10
View File
@@ -386,6 +386,15 @@ func TestParsePort_PortGroups(t *testing.T) {
}
}
func TestParsePortGroupsRequireWholeToken(t *testing.T) {
if got := ParsePort("web8080"); len(got) != 0 {
t.Fatalf("ParsePort(web8080) = %v, want empty invalid token", got)
}
if got := ParsePort("web,8080"); len(got) == 0 || got[len(got)-1] != 28018 {
t.Fatalf("ParsePort(web,8080) = %v, want expanded web group", got)
}
}
// TestParsePort_WhitespaceHandling 测试空格处理
func TestParsePort_WhitespaceHandling(t *testing.T) {
tests := []struct {
@@ -590,6 +599,18 @@ func TestParseIP_IPRange(t *testing.T) {
}
}
func TestParseIP_IPRangeNoLimit(t *testing.T) {
result, err := parseIPRangeString("192.168.1.1-5")
if err != nil {
t.Fatalf("parseIPRangeString error = %v", err)
}
expected := []string{"192.168.1.1", "192.168.1.2", "192.168.1.3", "192.168.1.4", "192.168.1.5"}
if !reflect.DeepEqual(result, expected) {
t.Errorf("parseIPRangeString = %v, want %v", result, expected)
}
}
// TestParseIP_FromFile 测试从文件读取
//
// 验证:文件中的IP列表被正确读取
@@ -684,6 +705,18 @@ func TestParseIP_Exclude(t *testing.T) {
}
}
func TestParseIPMultipleExcludeSources(t *testing.T) {
result, err := ParseIP("192.168.1.1-192.168.1.4", "", "192.168.1.2", "192.168.1.4")
if err != nil {
t.Fatalf("ParseIP error = %v", err)
}
expected := []string{"192.168.1.1", "192.168.1.3"}
if !reflect.DeepEqual(result, expected) {
t.Fatalf("ParseIP with multiple excludes = %v, want %v", result, expected)
}
}
// TestParseIP_Deduplicate 测试去重
func TestParseIP_Deduplicate(t *testing.T) {
result, err := ParseIP("192.168.1.1,192.168.1.1,192.168.1.2,192.168.1.2", "", "")
@@ -788,7 +821,9 @@ func TestParsePortRange(t *testing.T) {
// TestExcludeHosts 测试排除主机
func TestExcludeHosts(t *testing.T) {
hosts := []string{"host1", "host2", "host3", "host4"}
exclude := []string{"host2", "host4"}
exclude := newHostMatcher()
exclude.exact["host2"] = struct{}{}
exclude.exact["host4"] = struct{}{}
result := excludeFromList(hosts, exclude)
expected := []string{"host1", "host3"}
@@ -803,7 +838,7 @@ func TestExcludeHosts(t *testing.T) {
// TestExcludeHosts_EmptyExclude 测试空排除列表
func TestExcludeHosts_EmptyExclude(t *testing.T) {
hosts := []string{"host1", "host2"}
result := excludeFromList(hosts, []string{})
result := excludeFromList(hosts, nil)
if !reflect.DeepEqual(result, hosts) {
t.Errorf("excludeFromList(空排除列表) 应该返回原列表")
@@ -878,12 +913,6 @@ func TestParseIP_InternalNetworkShortcuts(t *testing.T) {
100, // 172.16.0.0/12 应该很多
"172.",
},
{
"10简写",
"10",
100, // 10.0.0.0/8 应该很多
"10.",
},
}
for _, tt := range tests {
@@ -969,6 +998,24 @@ func TestParseIP_FullIPRange(t *testing.T) {
}
}
func TestParseIP_FullIPRangeComplete(t *testing.T) {
result, err := parseIPRangeString("192.168.1.1-192.168.1.5")
if err != nil {
t.Fatalf("parseIPRangeString error = %v", err)
}
expected := []string{
"192.168.1.1",
"192.168.1.2",
"192.168.1.3",
"192.168.1.4",
"192.168.1.5",
}
if !reflect.DeepEqual(result, expected) {
t.Errorf("parseIPRangeString no limit = %v, want %v", result, expected)
}
}
// TestParseIP_InvalidCIDR 测试无效CIDR
func TestParseIP_InvalidCIDR(t *testing.T) {
tests := []struct {
@@ -977,7 +1024,7 @@ func TestParseIP_InvalidCIDR(t *testing.T) {
expectErr bool
}{
{"无效掩码/33", "192.168.1.0/33", true},
{"效掩码/0", "192.168.1.0/0", false}, // /0 技术上是有效的
{"效掩码/32", "192.168.1.1/32", false},
{"格式错误", "192.168.1.0/abc", true},
{"缺少掩码", "192.168.1.0/", true},
}
@@ -1173,7 +1220,7 @@ test:
{"admin", "password123"},
{"root", "toor"},
{"user", "pass:with:colons"}, // 密码可以包含冒号
{"test", ""}, // 空密码
{"test", ""}, // 空密码
}
if len(result) != len(tests) {
+49 -49
View File
@@ -11,6 +11,7 @@ import (
"strings"
"github.com/shadow1ng/fscan/common/config"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -37,7 +38,7 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
if filename != "" {
fileHosts, err := ReadLinesFromFile(filename)
if err != nil {
return nil, fmt.Errorf("读取主机文件失败: %w", err)
return nil, fmt.Errorf(i18n.GetText("parser_read_hosts_failed")+": %w", err)
}
for _, h := range fileHosts {
parsed, err := parseHostString(h)
@@ -52,18 +53,27 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
if host != "" {
hostList, err := parseHostString(host)
if err != nil {
return nil, fmt.Errorf("解析主机失败: %w", err)
return nil, fmt.Errorf(i18n.GetText("parser_parse_host_failed")+": %w", err)
}
hosts = append(hosts, hostList...)
}
// 处理排除主机
if len(nohosts) > 0 && nohosts[0] != "" {
excludeList, err := parseHostString(nohosts[0])
if err != nil {
return nil, fmt.Errorf("解析排除主机失败: %w", err)
if len(nohosts) > 0 {
matcher := newHostMatcher()
hasExclude := false
for _, exclude := range nohosts {
if strings.TrimSpace(exclude) == "" {
continue
}
hasExclude = true
if err := matcher.add(exclude); err != nil {
return nil, fmt.Errorf(i18n.GetText("parser_parse_exclude_failed")+": %w", err)
}
}
if hasExclude {
hosts = excludeFromList(hosts, matcher)
}
hosts = excludeFromList(hosts, excludeList)
}
// 去重和排序
@@ -71,7 +81,7 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
sort.Strings(hosts)
if len(hosts) == 0 {
return nil, fmt.Errorf("没有找到有效的主机")
return nil, fmt.Errorf("%s", i18n.GetText("parser_no_valid_hosts"))
}
return hosts, nil
@@ -89,33 +99,33 @@ func parseHostString(host string) ([]string, error) {
switch {
case h == "192":
cidrHosts, err := parseIPCIDR("192.168.0.0/16", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("192.168.0.0/16")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case h == "172":
cidrHosts, err := parseIPCIDR("172.16.0.0/12", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("172.16.0.0/12")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case h == "10":
cidrHosts, err := parseIPCIDR("10.0.0.0/8", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("10.0.0.0/8")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case strings.Contains(h, "/"):
cidrHosts, err := parseIPCIDR(h, SimpleMaxHosts)
cidrHosts, err := parseIPCIDR(h)
if err != nil {
return nil, fmt.Errorf("CIDR解析失败 %s: %w", h, err)
return nil, fmt.Errorf(i18n.Tr("parser_cidr_failed", h)+": %w", err)
}
hosts = append(hosts, cidrHosts...)
case strings.Contains(h, "-") && !strings.Contains(h, ":") && looksLikeIPRange(h):
rangeHosts, err := parseIPRangeString(h, SimpleMaxHosts)
rangeHosts, err := parseIPRangeString(h)
if err != nil {
return nil, fmt.Errorf("IP范围解析失败 %s: %w", h, err)
return nil, fmt.Errorf(i18n.Tr("parser_ip_range_failed", h)+": %w", err)
}
hosts = append(hosts, rangeHosts...)
default:
@@ -190,11 +200,14 @@ func parsePortRange(rangeStr string) []int {
// expandPortGroups 展开端口组
func expandPortGroups(ports string) string {
portGroups := config.GetPortGroups()
result := ports
for group, portList := range portGroups {
result = strings.ReplaceAll(result, group, portList)
parts := strings.Split(ports, ",")
for i, part := range parts {
token := strings.TrimSpace(part)
if portList, ok := portGroups[token]; ok {
parts[i] = portList
}
}
return result
return strings.Join(parts, ",")
}
// =============================================================================
@@ -211,6 +224,7 @@ func ReadLinesFromFile(filename string) ([]string, error) {
var lines []string
scanner := bufio.NewScanner(file)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line != "" && !strings.HasPrefix(line, "#") {
@@ -288,7 +302,7 @@ func ParseHashFile(filename string) ([]string, [][]byte, error) {
// =============================================================================
// parseIPCIDR 解析CIDR网段
func parseIPCIDR(cidr string, maxTargets int) ([]string, error) {
func parseIPCIDR(cidr string) ([]string, error) {
_, ipNet, err := net.ParseCIDR(cidr)
if err != nil {
return nil, err
@@ -298,17 +312,11 @@ func parseIPCIDR(cidr string, maxTargets int) ([]string, error) {
ip := make(net.IP, len(ipNet.IP))
copy(ip, ipNet.IP)
count := 0
for ipNet.Contains(ip) {
ips = append(ips, ip.String())
count++
if count >= maxTargets {
break
}
incrementIP(ip)
}
// 移除网络地址和广播地址
if len(ips) > 2 {
ips = ips[1 : len(ips)-1]
}
@@ -330,10 +338,10 @@ func looksLikeIPRange(s string) bool {
}
// parseIPRangeString 解析IP范围字符串
func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
func parseIPRangeString(rangeStr string) ([]string, error) {
parts := strings.Split(rangeStr, "-")
if len(parts) != 2 {
return nil, fmt.Errorf("无效的IP范围格式: %s", rangeStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_fmt", rangeStr))
}
startIPStr := strings.TrimSpace(parts[0])
@@ -341,7 +349,7 @@ func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
startIP := net.ParseIP(startIPStr)
if startIP == nil {
return nil, fmt.Errorf("无效的起始IP地址: %s", startIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_start_ip", startIPStr))
}
// 处理简写格式 (如: 192.168.1.1-100)
@@ -352,28 +360,28 @@ func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
// 处理完整格式 (如: 192.168.1.1-192.168.1.100)
endIP := net.ParseIP(endIPStr)
if endIP == nil {
return nil, fmt.Errorf("无效的结束IP地址: %s", endIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_end_ip", endIPStr))
}
return parseIPFullRange(startIP, endIP, maxTargets)
return parseIPFullRange(startIP, endIP)
}
// parseIPShortRange 解析短格式IP范围
func parseIPShortRange(startIPStr, endSuffix string) ([]string, error) {
endNum, err := strconv.Atoi(endSuffix)
if err != nil || endNum > 255 {
return nil, fmt.Errorf("无效的IP范围结束值: %s", endSuffix)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_end_val", endSuffix))
}
ipParts := strings.Split(startIPStr, ".")
if len(ipParts) != 4 {
return nil, fmt.Errorf("无效的IP地址格式: %s", startIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_fmt", startIPStr))
}
prefixIP := strings.Join(ipParts[0:3], ".")
startNum, err := strconv.Atoi(ipParts[3])
if err != nil || startNum > endNum {
return nil, fmt.Errorf("无效的IP范围: %s-%s", startIPStr, endSuffix)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_val", startIPStr, endSuffix))
}
var allIP []string
@@ -385,30 +393,27 @@ func parseIPShortRange(startIPStr, endSuffix string) ([]string, error) {
}
// parseIPFullRange 解析完整格式的IP范围
func parseIPFullRange(startIP, endIP net.IP, maxTargets int) ([]string, error) {
func parseIPFullRange(startIP, endIP net.IP) ([]string, error) {
start4 := startIP.To4()
end4 := endIP.To4()
if start4 == nil || end4 == nil {
return nil, fmt.Errorf("仅支持IPv4地址范围")
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
startInt := (int(start4[0]) << 24) | (int(start4[1]) << 16) | (int(start4[2]) << 8) | int(start4[3])
endInt := (int(end4[0]) << 24) | (int(end4[1]) << 16) | (int(end4[2]) << 8) | int(end4[3])
if startInt > endInt {
return nil, fmt.Errorf("起始IP大于结束IP")
return nil, fmt.Errorf("%s", i18n.GetText("parser_start_gt_end"))
}
var ips []string
current := make(net.IP, len(start4))
copy(current, start4)
count := 0
for {
ips = append(ips, current.String())
count++
if current.Equal(end4) || count >= maxTargets {
if current.Equal(end4) {
break
}
incrementIP(current)
@@ -428,19 +433,14 @@ func incrementIP(ip net.IP) {
}
// excludeFromList 从列表中排除指定项
func excludeFromList(hosts, excludeList []string) []string {
if len(excludeList) == 0 {
func excludeFromList(hosts []string, matcher *hostMatcher) []string {
if matcher == nil {
return hosts
}
excludeMap := make(map[string]struct{}, len(excludeList))
for _, e := range excludeList {
excludeMap[e] = struct{}{}
}
result := make([]string, 0, len(hosts))
for _, h := range hosts {
if _, found := excludeMap[h]; !found {
if !matcher.match(h) {
result = append(result, h)
}
}
+78 -56
View File
@@ -28,11 +28,11 @@ ProgressManager.go - 固定底部进度条管理器
type ProgressManager struct {
mu sync.RWMutex
enabled bool
total int64
current int64
total atomic.Int64
current atomic.Int64
description string
startTime time.Time
isActive bool
isActive atomic.Bool
terminalHeight int
reservedLines int // 为进度条保留的行数
lastContentLine int // 最后一行内容的位置
@@ -48,6 +48,10 @@ type ProgressManager struct {
// 进度条更新控制(减少 Windows 终端的重复输出)
lastRenderedPercent int
// 引用,避免读全局
state *State
noColor bool
}
// =============================================================================
@@ -102,20 +106,22 @@ func GetProgressManager() *ProgressManager {
// InitProgress 初始化进度条
func (pm *ProgressManager) InitProgress(total int64, description string) {
fv := GetFlagVars()
if fv.DisableProgress || fv.Silent {
cfg := GetGlobalConfig()
if cfg.Output.DisableProgress || cfg.Output.Silent || cfg.Output.NoColor {
pm.enabled = false
return
}
pm.state = GetGlobalState()
pm.noColor = cfg.Output.NoColor
pm.mu.Lock()
defer pm.mu.Unlock()
pm.total = total
pm.current = 0
pm.total.Store(total)
pm.current.Store(0)
pm.description = description
pm.startTime = time.Now()
pm.isActive = true
pm.isActive.Store(true)
pm.enabled = true
pm.lastActivity = time.Now()
pm.spinnerIndex = 0
@@ -133,20 +139,28 @@ func (pm *ProgressManager) InitProgress(total int64, description string) {
// UpdateProgress 更新进度
func (pm *ProgressManager) UpdateProgress(increment int64) {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
// 原子累加,避免高并发下的锁竞争
newCurrent := pm.current.Add(increment)
if newCurrent > pm.total.Load() {
pm.current.Store(pm.total.Load())
}
// 节流渲染:距上次渲染不足 50ms 则跳过
now := time.Now()
pm.mu.RLock()
lastAct := pm.lastActivity
pm.mu.RUnlock()
if now.Sub(lastAct) < 50*time.Millisecond {
return
}
pm.mu.Lock()
defer pm.mu.Unlock()
pm.current += increment
if pm.current > pm.total {
pm.current = pm.total
}
// 更新活跃时间
pm.lastActivity = time.Now()
pm.lastActivity = now
pm.mu.Unlock()
pm.renderProgress()
}
@@ -157,14 +171,14 @@ func (pm *ProgressManager) UpdateProgress(increment int64) {
// FinishProgress 完成进度条
func (pm *ProgressManager) FinishProgress() {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
pm.mu.Lock()
defer pm.mu.Unlock()
pm.current = pm.total
pm.current.Store(pm.total.Load())
pm.renderProgress()
// 停止活跃指示器
@@ -175,7 +189,7 @@ func (pm *ProgressManager) FinishProgress() {
// 清理进度条区域,恢复正常输出
pm.clearProgressArea()
pm.isActive = false
pm.isActive.Store(false)
}
// setupProgressSpace 设置进度条空间
@@ -205,20 +219,21 @@ func (pm *ProgressManager) generateProgressBar() string {
// 获取发包统计
packetInfo := pm.getPacketInfo()
if pm.total == 0 {
if pm.total.Load() == 0 {
spinner := pm.getActivityIndicator()
base := fmt.Sprintf("%s %s 等待中...", pm.description, spinner)
base := fmt.Sprintf("%s %s %s", pm.description, spinner, i18n.GetText("progress_waiting"))
if packetInfo != "" {
return base + " " + packetInfo
}
return base
}
percentage := float64(pm.current) / float64(pm.total) * 100
percentage := float64(pm.current.Load()) / float64(pm.total.Load()) * 100
elapsed := time.Since(pm.startTime)
current := pm.current.Load()
// 计算速度
speed := float64(pm.current) / elapsed.Seconds()
speed := float64(current) / elapsed.Seconds()
speedStr := ""
if speed > 0 {
speedStr = fmt.Sprintf(" %.0f/s", speed)
@@ -226,8 +241,8 @@ func (pm *ProgressManager) generateProgressBar() string {
// 计算预估剩余时间
var eta string
if pm.current > 0 && pm.current < pm.total {
totalTime := elapsed * time.Duration(pm.total) / time.Duration(pm.current)
if current > 0 && current < pm.total.Load() {
totalTime := elapsed * time.Duration(pm.total.Load()) / time.Duration(current)
remaining := totalTime - elapsed
if remaining > 0 {
eta = fmt.Sprintf(" ETA:%s", formatDuration(remaining))
@@ -239,7 +254,7 @@ func (pm *ProgressManager) generateProgressBar() string {
// 计算固定部分的宽度
fixedPart := fmt.Sprintf("%s %s %5.1f%% [] (%d/%d)%s%s %s",
pm.description, spinner, percentage, pm.current, pm.total, speedStr, eta, packetInfo)
pm.description, spinner, percentage, current, pm.total.Load(), speedStr, eta, packetInfo)
fixedWidth := displayWidth(fixedPart)
// 计算进度条槽位可用宽度(预留2字符余量)
@@ -266,7 +281,7 @@ func (pm *ProgressManager) generateProgressBar() string {
// 构建最终进度条
result := fmt.Sprintf("%s %s %5.1f%% %s (%d/%d)%s%s",
pm.description, spinner, percentage, bar, pm.current, pm.total, speedStr, eta)
pm.description, spinner, percentage, bar, current, pm.total.Load(), speedStr, eta)
if packetInfo != "" {
result += " " + packetInfo
@@ -277,13 +292,16 @@ func (pm *ProgressManager) generateProgressBar() string {
// getPacketInfo 获取发包统计信息(简化版)
func (pm *ProgressManager) getPacketInfo() string {
packetCount := GetGlobalState().GetPacketCount()
if pm.state == nil {
return ""
}
packetCount := pm.state.GetPacketCount()
if packetCount == 0 {
return ""
}
tcpSuccess := GetGlobalState().GetTCPSuccessPacketCount()
tcpFailed := GetGlobalState().GetTCPFailedPacketCount()
tcpSuccess := pm.state.GetTCPSuccessPacketCount()
tcpFailed := pm.state.GetTCPFailedPacketCount()
// 简化格式:TCP:成功/失败
if tcpSuccess > 0 || tcpFailed > 0 {
@@ -301,13 +319,16 @@ func (pm *ProgressManager) showCompletionInfo() {
fmt.Print("\n")
completionMsg := i18n.GetText("progress_scan_completed")
if GetFlagVars().NoColor {
fmt.Printf("[完成] %s %d/%d (耗时: %s)\n",
completionMsg, pm.total, pm.total, formatDuration(elapsed))
doneMsg := i18n.GetText("progress_done")
durationMsg := i18n.GetText("progress_duration")
total := pm.total.Load()
if pm.noColor {
fmt.Printf("[%s] %s: %d/%d (%s: %s)\n",
doneMsg, completionMsg, total, total, durationMsg, formatDuration(elapsed))
} else {
fmt.Printf("%s[完成] %s %d/%d%s %s(耗时: %s)%s\n",
AnsiGreen, completionMsg, pm.total, pm.total, AnsiReset,
AnsiGray, formatDuration(elapsed), AnsiReset)
fmt.Printf("%s[%s] %s: %d/%d%s %s(%s: %s)%s\n",
AnsiGreen, doneMsg, completionMsg, total, total, AnsiReset,
AnsiGray, durationMsg, formatDuration(elapsed), AnsiReset)
}
}
@@ -321,7 +342,7 @@ func (pm *ProgressManager) clearProgressArea() {
func (pm *ProgressManager) IsActive() bool {
pm.mu.RLock()
defer pm.mu.RUnlock()
return pm.isActive && pm.enabled
return pm.isActive.Load() && pm.enabled
}
// getTerminalHeight 获取终端高度
@@ -458,10 +479,10 @@ func (pm *ProgressManager) GetPercent() float64 {
pm.mu.RLock()
defer pm.mu.RUnlock()
if !pm.isActive || pm.total == 0 {
if !pm.isActive.Load() || pm.total.Load() == 0 {
return 0
}
return float64(pm.current) / float64(pm.total) * 100
return float64(pm.current.Load()) / float64(pm.total.Load()) * 100
}
// =============================================================================
@@ -470,6 +491,10 @@ func (pm *ProgressManager) GetPercent() float64 {
// LogWithProgress 在进度条活跃时协调日志输出
func LogWithProgress(message string) {
if cfg := GetGlobalConfig(); cfg != nil && cfg.Output.Silent {
return
}
pm := GetProgressManager()
if !pm.IsActive() {
// 如果进度条不活跃,直接输出
@@ -492,14 +517,14 @@ func LogWithProgress(message string) {
// renderProgressUnsafe 不加锁的进度条渲染(内部使用)
func (pm *ProgressManager) renderProgressUnsafe() {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
// 计算当前百分比(避免除零)
currentPercent := 0
if pm.total > 0 {
currentPercent = int((pm.current * 100) / pm.total)
if pm.total.Load() > 0 {
currentPercent = int((pm.current.Load() * 100) / pm.total.Load())
}
// 只在百分比变化时更新,减少不必要的渲染
@@ -532,7 +557,7 @@ func (pm *ProgressManager) renderProgressUnsafe() {
fmt.Print(clearStr)
// 输出进度条(带颜色,如果启用)
if GetFlagVars().NoColor {
if pm.noColor {
fmt.Print(progressBar)
} else {
fmt.Printf("%s%s%s", AnsiCyan, progressBar, AnsiReset)
@@ -561,7 +586,7 @@ func (pm *ProgressManager) startActivityIndicator() {
select {
case <-pm.activityTicker.C:
// 只有在活跃状态下才更新指示器
if pm.isActive && pm.enabled {
if pm.isActive.Load() && pm.enabled {
pm.mu.Lock()
pm.spinnerIndex = (pm.spinnerIndex + 1) % len(spinnerChars)
pm.mu.Unlock()
@@ -618,8 +643,8 @@ ConcurrencyMonitor - 并发监控器
// ConcurrencyMonitor 并发监控器
type ConcurrencyMonitor struct {
// 主扫描器层级
activePluginTasks int64 // 当前活跃的插件任务数
totalPluginTasks int64 // 总插件任务数
activePluginTasks atomic.Int64 // 当前活跃的插件任务数
totalPluginTasks atomic.Int64 // 总插件任务数
// 插件内连接层级已移除 - 原代码为死代码,无任何调用者
}
@@ -634,10 +659,7 @@ var (
// GetConcurrencyMonitor 获取全局并发监控器
func GetConcurrencyMonitor() *ConcurrencyMonitor {
concurrencyMutex.Do(func() {
globalConcurrencyMonitor = &ConcurrencyMonitor{
activePluginTasks: 0,
totalPluginTasks: 0,
}
globalConcurrencyMonitor = &ConcurrencyMonitor{}
})
return globalConcurrencyMonitor
}
@@ -648,18 +670,18 @@ func GetConcurrencyMonitor() *ConcurrencyMonitor {
// StartPluginTask 开始插件任务
func (m *ConcurrencyMonitor) StartPluginTask() {
atomic.AddInt64(&m.activePluginTasks, 1)
atomic.AddInt64(&m.totalPluginTasks, 1)
m.activePluginTasks.Add(1)
m.totalPluginTasks.Add(1)
}
// FinishPluginTask 完成插件任务
func (m *ConcurrencyMonitor) FinishPluginTask() {
atomic.AddInt64(&m.activePluginTasks, -1)
m.activePluginTasks.Add(-1)
}
// GetPluginTaskStats 获取插件任务统计
func (m *ConcurrencyMonitor) GetPluginTaskStats() (active int64, total int64) {
return atomic.LoadInt64(&m.activePluginTasks), atomic.LoadInt64(&m.totalPluginTasks)
return m.activePluginTasks.Load(), m.totalPluginTasks.Load()
}
// =============================================================================
+102
View File
@@ -0,0 +1,102 @@
package common
import (
"strings"
"testing"
"time"
)
func TestProgressTextHelpers(t *testing.T) {
tests := []struct {
name string
in string
want int
}{
{name: "ascii", in: "abc", want: 3},
{name: "cjk", in: "中文", want: 4},
{name: "mixed", in: "a中", want: 3},
{name: "symbol", in: "★", want: 2},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := displayWidth(tt.in); got != tt.want {
t.Fatalf("displayWidth(%q) = %d, want %d", tt.in, got, tt.want)
}
})
}
truncateTests := []struct {
name string
in string
width int
want string
}{
{name: "exact mixed width", in: "abc中文", width: 5, want: "abc中"},
{name: "wide char does not fit", in: "中文", width: 1, want: ""},
{name: "zero width", in: "abc", width: 0, want: ""},
{name: "negative width", in: "abc", width: -1, want: ""},
}
for _, tt := range truncateTests {
t.Run(tt.name, func(t *testing.T) {
if got := truncateToWidth(tt.in, tt.width); got != tt.want {
t.Fatalf("truncateToWidth(%q, %d) = %q, want %q", tt.in, tt.width, got, tt.want)
}
})
}
if got := stripAnsiCodes("\033[31mred\033[0m plain"); got != "red plain" {
t.Fatalf("stripAnsiCodes removed ANSI = %q, want %q", got, "red plain")
}
if got := stripAnsiCodes("plain"); got != "plain" {
t.Fatalf("stripAnsiCodes plain = %q, want plain", got)
}
}
func TestFormatDuration(t *testing.T) {
tests := []struct {
name string
in time.Duration
want string
}{
{name: "seconds", in: 1500 * time.Millisecond, want: "1.5s"},
{name: "minutes", in: 90 * time.Second, want: "1.5m"},
{name: "hours", in: 150 * time.Minute, want: "2.5h"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := formatDuration(tt.in); got != tt.want {
t.Fatalf("formatDuration(%s) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
func TestConcurrencyMonitorTaskStats(t *testing.T) {
monitor := &ConcurrencyMonitor{}
if status := monitor.GetConcurrencyStatus(); status != "" {
t.Fatalf("initial status = %q, want empty", status)
}
monitor.StartPluginTask()
monitor.StartPluginTask()
active, total := monitor.GetPluginTaskStats()
if active != 2 || total != 2 {
t.Fatalf("stats after start = active %d total %d, want 2/2", active, total)
}
if status := monitor.GetConcurrencyStatus(); !strings.HasSuffix(status, ":2") {
t.Fatalf("status after start = %q, want suffix :2", status)
}
monitor.FinishPluginTask()
active, total = monitor.GetPluginTaskStats()
if active != 1 || total != 2 {
t.Fatalf("stats after one finish = active %d total %d, want 1/2", active, total)
}
monitor.FinishPluginTask()
if status := monitor.GetConcurrencyStatus(); status != "" {
t.Fatalf("status after all finish = %q, want empty", status)
}
}
+18 -16
View File
@@ -2,6 +2,8 @@ package proxy
import (
"time"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -151,41 +153,41 @@ const (
// 错误消息常量
// =============================================================================
const (
var (
// ErrMsgUnsupportedProxyType Manager错误消息 - 不支持的代理类型
ErrMsgUnsupportedProxyType = "不支持的代理类型"
ErrMsgUnsupportedProxyType = i18n.GetText("proxy_unsupported_type")
// ErrMsgEmptyConfig 配置不能为空
ErrMsgEmptyConfig = "配置不能为空"
ErrMsgEmptyConfig = i18n.GetText("proxy_empty_config")
// ErrMsgSOCKS5ParseFailed SOCKS5错误消息 - 地址解析失败
ErrMsgSOCKS5ParseFailed = "SOCKS5代理地址解析失败"
ErrMsgSOCKS5ParseFailed = i18n.GetText("proxy_socks5_parse_failed")
// ErrMsgSOCKS5CreateFailed 拨号器创建失败
ErrMsgSOCKS5CreateFailed = "SOCKS5拨号器创建失败"
ErrMsgSOCKS5CreateFailed = i18n.GetText("proxy_socks5_create_failed")
// ErrMsgSOCKS5ConnTimeout 连接超时
ErrMsgSOCKS5ConnTimeout = "SOCKS5连接超时"
ErrMsgSOCKS5ConnTimeout = i18n.GetText("proxy_socks5_conn_timeout")
// ErrMsgSOCKS5ConnFailed 连接失败
ErrMsgSOCKS5ConnFailed = "SOCKS5连接失败"
ErrMsgSOCKS5ConnFailed = i18n.GetText("proxy_socks5_conn_failed")
// ErrMsgDirectConnFailed 直连错误消息 - 直连失败
ErrMsgDirectConnFailed = "直连失败"
ErrMsgDirectConnFailed = i18n.GetText("proxy_direct_conn_failed")
// ErrMsgHTTPConnFailed HTTP代理错误消息 - 连接失败
ErrMsgHTTPConnFailed = "连接HTTP代理服务器失败"
ErrMsgHTTPConnFailed = i18n.GetText("proxy_http_conn_failed")
// ErrMsgHTTPSetWriteTimeout 设置写超时失败
ErrMsgHTTPSetWriteTimeout = "设置写超时失败"
ErrMsgHTTPSetWriteTimeout = i18n.GetText("proxy_http_set_write_timeout")
// ErrMsgHTTPSendConnectFail 发送CONNECT请求失败
ErrMsgHTTPSendConnectFail = "发送CONNECT请求失败"
ErrMsgHTTPSendConnectFail = i18n.GetText("proxy_http_send_connect_failed")
// ErrMsgHTTPSetReadTimeout 设置读超时失败
ErrMsgHTTPSetReadTimeout = "设置读超时失败"
ErrMsgHTTPSetReadTimeout = i18n.GetText("proxy_http_set_read_timeout")
// ErrMsgHTTPReadRespFailed 读取响应失败
ErrMsgHTTPReadRespFailed = "读取HTTP响应失败"
ErrMsgHTTPReadRespFailed = i18n.GetText("proxy_http_read_response_failed")
// ErrMsgHTTPProxyAuthFailed 代理认证失败
ErrMsgHTTPProxyAuthFailed = "HTTP代理连接失败,状态码: %d"
ErrMsgHTTPProxyAuthFailed = i18n.GetText("proxy_http_status_failed")
// ErrMsgTLSTCPConnFailed TLS错误消息 - TCP连接失败
ErrMsgTLSTCPConnFailed = "建立TCP连接失败"
ErrMsgTLSTCPConnFailed = i18n.GetText("proxy_tls_tcp_conn_failed")
// ErrMsgTLSHandshakeFailed TLS握手失败
ErrMsgTLSHandshakeFailed = "TLS握手失败"
ErrMsgTLSHandshakeFailed = i18n.GetText("proxy_tls_handshake_failed")
)
// =============================================================================
+9
View File
@@ -19,6 +19,9 @@ var (
// proxyProbed 标记代理是否已经探测过(避免重复探测)
proxyProbed atomic.Bool
// currentProxyType 当前代理类型
currentProxyType atomic.Int32
)
// SetProxyEnabled 设置代理启用状态
@@ -61,6 +64,11 @@ func IsProxyProbed() bool {
return proxyProbed.Load()
}
// IsSOCKS5Proxy 检查当前代理是否为SOCKS5类型
func IsSOCKS5Proxy() bool {
return proxyEnabled.Load() && ProxyType(currentProxyType.Load()) == ProxyTypeSOCKS5
}
// AutoConfigureProxy 自动配置代理相关行为
// 根据代理类型和状态自动调整扫描策略
func AutoConfigureProxy(config *ProxyConfig) {
@@ -74,6 +82,7 @@ func AutoConfigureProxy(config *ProxyConfig) {
// 启用代理标记
SetProxyEnabled(true)
currentProxyType.Store(int32(config.Type))
// SOCKS5代理默认假设非标准(后续由探测函数验证)
if config.Type == ProxyTypeSOCKS5 {
+13 -9
View File
@@ -7,7 +7,7 @@ import (
"fmt"
"net"
"net/http"
"sync/atomic"
"strings"
"time"
)
@@ -24,27 +24,27 @@ func (h *httpDialer) Dial(network, address string) (net.Conn, error) {
func (h *httpDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&h.stats.TotalConnections, 1)
h.stats.addTotal(1)
// 连接到HTTP代理服务器
proxyConn, err := h.baseDial.DialContext(ctx, NetworkTCP, h.config.Address)
if err != nil {
atomic.AddInt64(&h.stats.FailedConnections, 1)
h.stats.LastError = err.Error()
h.stats.addFailed(1)
h.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgHTTPConnFailed, ErrCodeHTTPConnFailed, err)
}
// 发送CONNECT请求
if err := h.sendConnectRequest(proxyConn, address); err != nil {
_ = proxyConn.Close() // 错误处理路径,Close错误可忽略
atomic.AddInt64(&h.stats.FailedConnections, 1)
h.stats.LastError = err.Error()
h.stats.addFailed(1)
h.stats.setLastError(err.Error())
return nil, err
}
duration := time.Since(start)
h.stats.LastConnectTime = start
atomic.AddInt64(&h.stats.ActiveConnections, 1)
h.stats.setLastConnectTime(start)
h.stats.addActive(1)
h.updateAverageConnectTime(duration)
return &trackedConn{
@@ -55,6 +55,9 @@ func (h *httpDialer) DialContext(ctx context.Context, network, address string) (
// sendConnectRequest 发送HTTP CONNECT请求
func (h *httpDialer) sendConnectRequest(conn net.Conn, address string) error {
if strings.ContainsAny(address, "\r\n") {
return NewProxyError(ErrTypeProtocol, "invalid CONNECT target", ErrCodeHTTPReadRespFailed, nil)
}
// 构建CONNECT请求
req := fmt.Sprintf(HTTPConnectRequestFormat, address, address)
@@ -108,7 +111,8 @@ func (h *httpDialer) sendConnectRequest(conn net.Conn, address string) error {
// updateAverageConnectTime 更新平均连接时间
func (h *httpDialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
h.stats.mu.Lock()
defer h.stats.mu.Unlock()
if h.stats.AverageConnectTime == 0 {
h.stats.AverageConnectTime = duration
} else {
+23
View File
@@ -0,0 +1,23 @@
package proxy
import (
"net"
"testing"
"time"
)
func TestHTTPDialerRejectsConnectTargetWithLineBreak(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
dialer := &httpDialer{
config: &ProxyConfig{Timeout: time.Second},
stats: &ProxyStats{},
}
err := dialer.sendConnectRequest(client, "example.com:80\r\nX-Injected: yes")
if err == nil {
t.Fatal("sendConnectRequest() error = nil, want invalid target error")
}
}
+26 -37
View File
@@ -4,9 +4,7 @@ import (
"context"
"fmt"
"net"
"net/url"
"sync"
"sync/atomic"
"time"
"golang.org/x/net/proxy"
@@ -128,9 +126,7 @@ func (m *manager) Stats() *ProxyStats {
m.mu.RLock()
defer m.mu.RUnlock()
// 返回副本以避免并发问题
statsCopy := *m.stats
return &statsCopy
return m.stats.snapshot()
}
// createDirectDialer 创建直连拨号器
@@ -146,6 +142,9 @@ func (m *manager) createDirectDialer() Dialer {
func (m *manager) createSOCKS5Dialer() (Dialer, error) {
// 检查缓存
cacheKey := fmt.Sprintf(CacheKeySOCKS5, m.config.Address)
if m.config.Username != "" || m.config.Password != "" {
cacheKey = fmt.Sprintf("%s_%s_%s", cacheKey, m.config.Username, m.config.Password)
}
m.cacheMu.RLock()
if time.Now().Before(m.cacheExpiry) {
if cached, exists := m.dialerCache[cacheKey]; exists {
@@ -155,18 +154,6 @@ func (m *manager) createSOCKS5Dialer() (Dialer, error) {
}
m.cacheMu.RUnlock()
// 解析代理地址
proxyURL := fmt.Sprintf(SOCKS5URLFormat, m.config.Address)
if m.config.Username != "" {
proxyURL = fmt.Sprintf(SOCKS5URLAuthFormat,
m.config.Username, m.config.Password, m.config.Address)
}
u, err := url.Parse(proxyURL)
if err != nil {
return nil, NewProxyError(ErrTypeConfig, ErrMsgSOCKS5ParseFailed, ErrCodeSOCKS5ParseFailed, err)
}
// 创建基础拨号器
baseDial := &net.Dialer{
Timeout: m.config.Timeout,
@@ -175,16 +162,14 @@ func (m *manager) createSOCKS5Dialer() (Dialer, error) {
// 创建SOCKS5拨号器
var auth *proxy.Auth
if u.User != nil {
if m.config.Username != "" || m.config.Password != "" {
auth = &proxy.Auth{
User: u.User.Username(),
}
if password, hasPassword := u.User.Password(); hasPassword {
auth.Password = password
User: m.config.Username,
Password: m.config.Password,
}
}
socksDialer, err := proxy.SOCKS5(NetworkTCP, u.Host, auth, baseDial)
socksDialer, err := proxy.SOCKS5(NetworkTCP, m.config.Address, auth, baseDial)
if err != nil {
return nil, NewProxyError(ErrTypeConnection, ErrMsgSOCKS5CreateFailed, ErrCodeSOCKS5CreateFailed, err)
}
@@ -248,7 +233,7 @@ func (d *directDialer) Dial(network, address string) (net.Conn, error) {
func (d *directDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&d.stats.TotalConnections, 1)
d.stats.addTotal(1)
dialer := &net.Dialer{
Timeout: d.timeout,
@@ -264,15 +249,16 @@ func (d *directDialer) DialContext(ctx context.Context, network, address string)
conn, err := dialer.DialContext(ctx, network, address)
duration := time.Since(start)
d.stats.LastConnectTime = start
d.stats.setLastConnectTime(start)
if err != nil {
atomic.AddInt64(&d.stats.FailedConnections, 1)
d.stats.LastError = err.Error()
d.stats.addFailed(1)
d.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgDirectConnFailed, ErrCodeDirectConnFailed, err)
}
atomic.AddInt64(&d.stats.ActiveConnections, 1)
d.stats.addActive(1)
d.updateAverageConnectTime(duration)
return &trackedConn{
@@ -294,7 +280,7 @@ func (s *socks5Dialer) Dial(network, address string) (net.Conn, error) {
func (s *socks5Dialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&s.stats.TotalConnections, 1)
s.stats.addTotal(1)
// 创建一个带超时的上下文
dialCtx, cancel := context.WithTimeout(ctx, s.config.Timeout)
@@ -322,20 +308,21 @@ func (s *socks5Dialer) DialContext(ctx context.Context, network, address string)
select {
case <-dialCtx.Done():
atomic.AddInt64(&s.stats.FailedConnections, 1)
s.stats.LastError = dialCtx.Err().Error()
s.stats.addFailed(1)
s.stats.setLastError(dialCtx.Err().Error())
return nil, NewProxyError(ErrTypeTimeout, ErrMsgSOCKS5ConnTimeout, ErrCodeSOCKS5ConnTimeout, dialCtx.Err())
case result := <-connChan:
duration := time.Since(start)
s.stats.LastConnectTime = start
s.stats.setLastConnectTime(start)
if result.err != nil {
atomic.AddInt64(&s.stats.FailedConnections, 1)
s.stats.LastError = result.err.Error()
s.stats.addFailed(1)
s.stats.setLastError(result.err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgSOCKS5ConnFailed, ErrCodeSOCKS5ConnFailed, result.err)
}
atomic.AddInt64(&s.stats.ActiveConnections, 1)
s.stats.addActive(1)
s.updateAverageConnectTime(duration)
return &trackedConn{
@@ -347,7 +334,8 @@ func (s *socks5Dialer) DialContext(ctx context.Context, network, address string)
// updateAverageConnectTime 更新平均连接时间
func (d *directDialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
d.stats.mu.Lock()
defer d.stats.mu.Unlock()
if d.stats.AverageConnectTime == 0 {
d.stats.AverageConnectTime = duration
} else {
@@ -356,7 +344,8 @@ func (d *directDialer) updateAverageConnectTime(duration time.Duration) {
}
func (s *socks5Dialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
s.stats.mu.Lock()
defer s.stats.mu.Unlock()
if s.stats.AverageConnectTime == 0 {
s.stats.AverageConnectTime = duration
} else {
+156
View File
@@ -1,6 +1,9 @@
package proxy
import (
"fmt"
"io"
"net"
"testing"
"time"
)
@@ -242,6 +245,73 @@ func TestGetDialer_HTTPS(t *testing.T) {
t.Logf("✓ GetDialer 返回HTTPS代理拨号器")
}
func TestGetDialer_SOCKS5AuthSpecialChars(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen failed: %v", err)
}
defer ln.Close()
type credential struct {
user string
pass string
}
authCh := make(chan credential, 1)
errCh := make(chan error, 1)
go func() {
conn, err := ln.Accept()
if err != nil {
errCh <- err
return
}
defer conn.Close()
user, pass, err := handleTestSOCKS5Auth(conn)
if err != nil {
errCh <- err
return
}
authCh <- credential{user: user, pass: pass}
}()
origProbed := IsProxyProbed()
SetProxyProbed(true)
defer SetProxyProbed(origProbed)
config := &ProxyConfig{
Type: ProxyTypeSOCKS5,
Address: ln.Addr().String(),
Username: "user",
Password: "p@ss:word#1",
Timeout: time.Second,
}
manager := NewProxyManager(config)
dialer, err := manager.GetDialer()
if err != nil {
t.Fatalf("GetDialer failed: %v", err)
}
conn, err := dialer.Dial("tcp", "127.0.0.1:80")
if err != nil {
t.Fatalf("SOCKS5 dial failed: %v", err)
}
_ = conn.Close()
select {
case got := <-authCh:
if got.user != config.Username || got.pass != config.Password {
t.Fatalf("auth = %q/%q, want %q/%q", got.user, got.pass, config.Username, config.Password)
}
case err := <-errCh:
t.Fatalf("SOCKS5 test server failed: %v", err)
case <-time.After(time.Second):
t.Fatal("timed out waiting for SOCKS5 auth")
}
t.Logf("✓ SOCKS5认证支持特殊字符密码")
}
// =============================================================================
// GetTLSDialer - TLS拨号器获取测试
// =============================================================================
@@ -559,3 +629,89 @@ func TestDirectDialer_LocalAddr_Loopback(t *testing.T) {
t.Logf("✓ LocalAddr 绑定正常工作(连接失败是预期的): %v", err)
}
}
func handleTestSOCKS5Auth(conn net.Conn) (string, string, error) {
header := make([]byte, 2)
if _, err := io.ReadFull(conn, header); err != nil {
return "", "", err
}
if header[0] != 0x05 {
return "", "", fmt.Errorf("unexpected socks version: %d", header[0])
}
methods := make([]byte, int(header[1]))
if _, err := io.ReadFull(conn, methods); err != nil {
return "", "", err
}
hasAuth := false
for _, method := range methods {
if method == 0x02 {
hasAuth = true
break
}
}
if !hasAuth {
return "", "", fmt.Errorf("client did not offer username/password auth")
}
if _, err := conn.Write([]byte{0x05, 0x02}); err != nil {
return "", "", err
}
authHeader := make([]byte, 2)
if _, err := io.ReadFull(conn, authHeader); err != nil {
return "", "", err
}
if authHeader[0] != 0x01 {
return "", "", fmt.Errorf("unexpected auth version: %d", authHeader[0])
}
userBytes := make([]byte, int(authHeader[1]))
if _, err := io.ReadFull(conn, userBytes); err != nil {
return "", "", err
}
passLen := make([]byte, 1)
if _, err := io.ReadFull(conn, passLen); err != nil {
return "", "", err
}
passBytes := make([]byte, int(passLen[0]))
if _, err := io.ReadFull(conn, passBytes); err != nil {
return "", "", err
}
if _, err := conn.Write([]byte{0x01, 0x00}); err != nil {
return "", "", err
}
reqHeader := make([]byte, 4)
if _, err := io.ReadFull(conn, reqHeader); err != nil {
return "", "", err
}
if reqHeader[0] != 0x05 || reqHeader[1] != 0x01 {
return "", "", fmt.Errorf("unexpected request header: %v", reqHeader)
}
if err := discardSOCKS5Address(conn, reqHeader[3]); err != nil {
return "", "", err
}
if _, err := conn.Write([]byte{0x05, 0x00, 0x00, 0x01, 0, 0, 0, 0, 0, 0}); err != nil {
return "", "", err
}
return string(userBytes), string(passBytes), nil
}
func discardSOCKS5Address(conn net.Conn, atyp byte) error {
switch atyp {
case 0x01:
_, err := io.CopyN(io.Discard, conn, 6)
return err
case 0x03:
length := make([]byte, 1)
if _, err := io.ReadFull(conn, length); err != nil {
return err
}
_, err := io.CopyN(io.Discard, conn, int64(length[0])+2)
return err
case 0x04:
_, err := io.CopyN(io.Discard, conn, 18)
return err
default:
return fmt.Errorf("unsupported atyp: %d", atyp)
}
}
+10 -9
View File
@@ -49,8 +49,8 @@ func (t *tlsDialerWrapper) DialTLSContext(ctx context.Context, network, address
// 进行TLS握手
if err := tlsConn.Handshake(); err != nil {
_ = tcpConn.Close() // TLS握手失败,Close错误可忽略
atomic.AddInt64(&t.stats.FailedConnections, 1)
t.stats.LastError = err.Error()
t.stats.addFailed(1)
t.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgTLSHandshakeFailed, ErrCodeTLSHandshakeFailed, err)
}
@@ -71,7 +71,8 @@ func (t *tlsDialerWrapper) DialTLSContext(ctx context.Context, network, address
// updateAverageConnectTime 更新平均连接时间
func (t *tlsDialerWrapper) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
t.stats.mu.Lock()
defer t.stats.mu.Unlock()
if t.stats.AverageConnectTime == 0 {
t.stats.AverageConnectTime = duration
} else {
@@ -81,16 +82,16 @@ func (t *tlsDialerWrapper) updateAverageConnectTime(duration time.Duration) {
// trackedConn 带统计的连接
type trackedConn struct {
bytesSent atomic.Int64
bytesRecv atomic.Int64
net.Conn
stats *ProxyStats
bytesSent int64
bytesRecv int64
stats *ProxyStats
}
func (tc *trackedConn) Read(b []byte) (n int, err error) {
n, err = tc.Conn.Read(b)
if n > 0 {
atomic.AddInt64(&tc.bytesRecv, int64(n))
tc.bytesRecv.Add(int64(n))
}
return n, err
}
@@ -98,13 +99,13 @@ func (tc *trackedConn) Read(b []byte) (n int, err error) {
func (tc *trackedConn) Write(b []byte) (n int, err error) {
n, err = tc.Conn.Write(b)
if n > 0 {
atomic.AddInt64(&tc.bytesSent, int64(n))
tc.bytesSent.Add(int64(n))
}
return n, err
}
func (tc *trackedConn) Close() error {
atomic.AddInt64(&tc.stats.ActiveConnections, -1)
tc.stats.addActive(-1)
return tc.Conn.Close()
}
+48
View File
@@ -4,6 +4,7 @@ import (
"context"
"crypto/tls"
"net"
"sync"
"time"
)
@@ -98,6 +99,7 @@ type ProxyStats struct {
TotalConnections int64 `json:"total_connections"`
ActiveConnections int64 `json:"active_connections"`
FailedConnections int64 `json:"failed_connections"`
mu sync.Mutex `json:"-"`
AverageConnectTime time.Duration `json:"average_connect_time"`
LastConnectTime time.Time `json:"last_connect_time"`
LastError string `json:"last_error,omitempty"`
@@ -105,6 +107,52 @@ type ProxyStats struct {
ProxyAddress string `json:"proxy_address"`
}
func (s *ProxyStats) addTotal(delta int64) {
s.mu.Lock()
s.TotalConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) addActive(delta int64) {
s.mu.Lock()
s.ActiveConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) addFailed(delta int64) {
s.mu.Lock()
s.FailedConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) setLastConnectTime(t time.Time) {
s.mu.Lock()
s.LastConnectTime = t
s.mu.Unlock()
}
func (s *ProxyStats) setLastError(err string) {
s.mu.Lock()
s.LastError = err
s.mu.Unlock()
}
func (s *ProxyStats) snapshot() *ProxyStats {
s.mu.Lock()
defer s.mu.Unlock()
return &ProxyStats{
TotalConnections: s.TotalConnections,
ActiveConnections: s.ActiveConnections,
FailedConnections: s.FailedConnections,
AverageConnectTime: s.AverageConnectTime,
LastConnectTime: s.LastConnectTime,
LastError: s.LastError,
ProxyType: s.ProxyType,
ProxyAddress: s.ProxyAddress,
}
}
// ProxyError 代理错误类型
//
//nolint:revive // 保持与现有代码的向后兼容性
+62
View File
@@ -0,0 +1,62 @@
package common
import "testing"
func TestResultCallbackLifecycle(t *testing.T) {
ClearResultCallback()
t.Cleanup(ClearResultCallback)
called := false
SetResultCallback(func(result interface{}) {
called = true
if result != "payload" {
t.Fatalf("callback payload = %#v", result)
}
})
NotifyResult("payload")
if !called {
t.Fatal("callback was not called")
}
called = false
ClearResultCallback()
NotifyResult("payload")
if called {
t.Fatal("callback should not be called after ClearResultCallback")
}
}
func TestStateRuntimeTargetsAndShellFlags(t *testing.T) {
state := NewState()
urls := []string{"http://example.com", "https://example.org"}
state.SetURLs(urls)
if got := state.GetURLs(); len(got) != 2 || got[0] != urls[0] || got[1] != urls[1] {
t.Fatalf("urls = %#v", got)
}
hostPorts := []string{"127.0.0.1:80", "[::1]:443"}
state.SetHostPorts(hostPorts)
if got := state.GetHostPorts(); len(got) != 2 || got[0] != hostPorts[0] || got[1] != hostPorts[1] {
t.Fatalf("hostPorts = %#v", got)
}
state.ClearHostPorts()
if got := state.GetHostPorts(); got != nil {
t.Fatalf("hostPorts after clear = %#v, want nil", got)
}
state.SetForwardShellActive(true)
state.SetReverseShellActive(true)
state.SetSocks5ProxyActive(true)
if !state.IsForwardShellActive() || !state.IsReverseShellActive() || !state.IsSocks5ProxyActive() {
t.Fatal("shell/proxy flags should be active")
}
state.SetForwardShellActive(false)
state.SetReverseShellActive(false)
state.SetSocks5ProxyActive(false)
if state.IsForwardShellActive() || state.IsReverseShellActive() || state.IsSocks5ProxyActive() {
t.Fatal("shell/proxy flags should be inactive")
}
}
+231
View File
@@ -0,0 +1,231 @@
package common
import (
"context"
"fmt"
"net"
"net/http"
"strings"
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/output"
"github.com/shadow1ng/fscan/common/proxy"
)
// ResultSink receives structured scan results for one scan session.
type ResultSink func(result *output.ScanResult) error
// ScanSession 封装单次扫描的全部上下文
// 一次扫描一个 session,并发扫描各自独立
type ScanSession struct {
Config *Config // 不可变,创建后只读
State *State // 可变,原子操作,每会话独立
Params *FlagVars // 原始参数,只读
ResultSink ResultSink // 可选,覆盖全局输出
PauseGate func(ctx context.Context) error
// 每会话 dialer(按 timeout 懒初始化,取决于代理配置)
dialerMu sync.Mutex
dialers map[time.Duration]proxy.Dialer
dialerErrs map[time.Duration]error
}
// NewScanSession 从已构建的 Config、State 和 FlagVars 创建会话
func NewScanSession(config *Config, state *State, params *FlagVars) *ScanSession {
return &ScanSession{
Config: config,
State: state,
Params: params,
}
}
// SaveResult saves a scan result through the session sink if present, otherwise
// falls back to the process-wide output pipeline used by the CLI.
func (s *ScanSession) SaveResult(result *output.ScanResult) error {
if s != nil && s.ResultSink != nil {
return s.ResultSink(result)
}
return SaveResult(result)
}
func (s *ScanSession) loggingEnabled() bool {
return s == nil || s.Config == nil || !s.Config.Output.Silent
}
// LogDebug writes through the session's logging policy.
func (s *ScanSession) LogDebug(msg string) {
if s.loggingEnabled() {
LogDebug(msg)
}
}
// LogInfo writes through the session's logging policy.
func (s *ScanSession) LogInfo(msg string) {
if s.loggingEnabled() {
LogInfo(msg)
}
}
// LogSuccess writes through the session's logging policy.
func (s *ScanSession) LogSuccess(result string) {
if s.loggingEnabled() {
LogSuccess(result)
}
}
// LogVuln writes through the session's logging policy.
func (s *ScanSession) LogVuln(result string) {
if s.loggingEnabled() {
LogVuln(result)
}
}
// LogError writes through the session's logging policy.
func (s *ScanSession) LogError(errMsg string) {
if s.loggingEnabled() {
LogError(errMsg)
}
}
// DialTCP 创建 TCP 连接,内含限速检查、代理、计数
func (s *ScanSession) DialTCP(ctx context.Context, network, address string, timeout time.Duration) (net.Conn, error) {
// 检查发包限制
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
s.LogError(i18n.Tr("tcp_connection_restricted", address, err.Error()))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
// 获取 dialer
dialer, err := s.getDialer(timeout)
if err != nil {
s.LogError(i18n.Tr("proxy_dialer_failed", err))
s.State.IncrementTCPFailedPacketCount()
return nil, err
}
conn, err := dialer.DialContext(ctx, network, address)
if err != nil {
s.State.IncrementTCPFailedPacketCount()
s.LogDebug(i18n.Tr("connection_failed", address, err))
return nil, err
}
// SO_LINGER=0: 连接关闭时立即发送 RST,避免 TIME_WAIT 堆积
if tc, ok := conn.(*net.TCPConn); ok {
_ = tc.SetLinger(0)
}
s.State.IncrementTCPSuccessPacketCount()
return conn, nil
}
// DialUDP creates a connected UDP socket with rate limiting and packet counting.
// UDP cannot be proxied; if a proxy is configured the connection is made directly.
func (s *ScanSession) DialUDP(ctx context.Context, address string, timeout time.Duration) (net.Conn, error) {
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
conn, err := net.DialTimeout("udp", address, timeout)
if err != nil {
s.State.IncrementUDPPacketCount()
return nil, err
}
_ = conn.SetDeadline(time.Now().Add(timeout))
s.State.IncrementUDPPacketCount()
return conn, nil
}
// HTTPDo executes an HTTP request with the session's packet limits and counters.
func (s *ScanSession) HTTPDo(client *http.Client, req *http.Request) (*http.Response, error) {
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
s.LogError(i18n.Tr("http_request_restricted", req.URL.String(), err.Error()))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
resp, err := client.Do(req)
if err != nil {
s.State.IncrementTCPFailedPacketCount()
return nil, err
}
s.State.IncrementTCPSuccessPacketCount()
return resp, nil
}
// ProxyEnabled reports whether this scan session uses a network proxy.
func (s *ScanSession) ProxyEnabled() bool {
if s == nil || s.Config == nil {
return false
}
return s.Config.Network.Socks5Proxy != "" || s.Config.Network.HTTPProxy != ""
}
// IsSOCKS5Proxy reports whether this scan session uses SOCKS5.
func (s *ScanSession) IsSOCKS5Proxy() bool {
return s != nil && s.Config != nil && s.Config.Network.Socks5Proxy != ""
}
// ProxyReliable reports whether the session proxy should be treated as reliable.
func (s *ScanSession) ProxyReliable() bool {
if !s.ProxyEnabled() || !s.IsSOCKS5Proxy() {
return true
}
return proxy.IsProxyReliable()
}
func (s *ScanSession) getDialer(timeout time.Duration) (proxy.Dialer, error) {
if timeout <= 0 {
timeout = s.Config.Timeout
}
s.dialerMu.Lock()
defer s.dialerMu.Unlock()
if s.dialers == nil {
s.dialers = make(map[time.Duration]proxy.Dialer)
s.dialerErrs = make(map[time.Duration]error)
}
if dialer, ok := s.dialers[timeout]; ok {
return dialer, s.dialerErrs[timeout]
}
cfg := s.createProxyConfig(timeout)
manager := proxy.NewProxyManager(cfg)
dialer, err := manager.GetDialer()
s.dialers[timeout] = dialer
s.dialerErrs[timeout] = err
return dialer, err
}
func (s *ScanSession) createProxyConfig(timeout time.Duration) *proxy.ProxyConfig {
cfg := proxy.DefaultProxyConfig()
cfg.Timeout = timeout
cfg.LocalAddr = s.Config.Network.Iface
// 优先 SOCKS5
if s.Config.Network.Socks5Proxy != "" {
cfg.Type = proxy.ProxyTypeSOCKS5
socks5URL := s.Config.Network.Socks5Proxy
if !strings.HasPrefix(socks5URL, "socks5://") {
socks5URL = "socks5://" + socks5URL
}
cfg.Address, cfg.Username, cfg.Password = parseProxyURL(socks5URL, s.Config.Network.Socks5Proxy)
return cfg
}
// 其次 HTTP
if s.Config.Network.HTTPProxy != "" {
if strings.HasPrefix(s.Config.Network.HTTPProxy, "https://") {
cfg.Type = proxy.ProxyTypeHTTPS
} else {
cfg.Type = proxy.ProxyTypeHTTP
}
cfg.Address, cfg.Username, cfg.Password = parseProxyURL(s.Config.Network.HTTPProxy, s.Config.Network.HTTPProxy)
return cfg
}
cfg.Type = proxy.ProxyTypeNone
return cfg
}
+168
View File
@@ -0,0 +1,168 @@
package common
import (
"io"
"net/http"
"strings"
"testing"
"time"
)
func TestScanSessionLogMethodsHonorSilentConfig(t *testing.T) {
loggerMu.Lock()
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
t.Cleanup(func() {
loggerMu.Lock()
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
})
cfg := NewConfig()
cfg.Output.Silent = true
session := NewScanSession(cfg, NewState(), &FlagVars{})
session.LogDebug("debug")
session.LogInfo("info")
session.LogSuccess("success")
session.LogVuln("vuln")
session.LogError("error")
loggerMu.Lock()
defer loggerMu.Unlock()
if globalLogger != nil {
t.Fatal("silent session log methods initialized global logger")
}
}
func TestScanSessionDialerCacheIsTimeoutAware(t *testing.T) {
cfg := NewConfig()
cfg.Timeout = 5 * time.Second
session := NewScanSession(cfg, NewState(), &FlagVars{})
shortTimeout := 100 * time.Millisecond
longTimeout := 2 * time.Second
shortDialer, err := session.getDialer(shortTimeout)
if err != nil {
t.Fatal(err)
}
shortDialerAgain, err := session.getDialer(shortTimeout)
if err != nil {
t.Fatal(err)
}
longDialer, err := session.getDialer(longTimeout)
if err != nil {
t.Fatal(err)
}
if shortDialer != shortDialerAgain {
t.Fatal("same timeout should reuse the session dialer")
}
if shortDialer == longDialer {
t.Fatal("different timeouts should not share one session dialer")
}
if got := session.createProxyConfig(shortTimeout).Timeout; got != shortTimeout {
t.Fatalf("proxy timeout = %v, want %v", got, shortTimeout)
}
}
func TestScanSessionHTTPDoUsesSessionState(t *testing.T) {
previousState := GetGlobalState()
globalState := NewState()
SetGlobalState(globalState)
t.Cleanup(func() { SetGlobalState(previousState) })
sessionState := NewState()
session := NewScanSession(NewConfig(), sessionState, &FlagVars{})
client := &http.Client{
Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusNoContent,
Body: io.NopCloser(strings.NewReader("")),
Header: make(http.Header),
}, nil
}),
}
req, err := http.NewRequest(http.MethodHead, "http://example.com", nil)
if err != nil {
t.Fatal(err)
}
resp, err := session.HTTPDo(client, req)
if err != nil {
t.Fatal(err)
}
_ = resp.Body.Close()
if got := sessionState.GetTCPSuccessPacketCount(); got != 1 {
t.Fatalf("session TCP success count = %d, want 1", got)
}
if got := globalState.GetTCPSuccessPacketCount(); got != 0 {
t.Fatalf("global TCP success count = %d, want 0", got)
}
}
func TestScanSessionProxyStateComesFromConfig(t *testing.T) {
direct := NewScanSession(NewConfig(), NewState(), &FlagVars{})
if direct.ProxyEnabled() {
t.Fatal("direct session should not report proxy enabled")
}
if direct.IsSOCKS5Proxy() {
t.Fatal("direct session should not report SOCKS5")
}
if !direct.ProxyReliable() {
t.Fatal("direct session should be reliable")
}
httpCfg := NewConfig()
httpCfg.Network.HTTPProxy = "http://127.0.0.1:8080"
httpSession := NewScanSession(httpCfg, NewState(), &FlagVars{})
if !httpSession.ProxyEnabled() {
t.Fatal("HTTP proxy session should report proxy enabled")
}
if httpSession.IsSOCKS5Proxy() {
t.Fatal("HTTP proxy session should not report SOCKS5")
}
if !httpSession.ProxyReliable() {
t.Fatal("HTTP proxy session should be reliable")
}
socksCfg := NewConfig()
socksCfg.Network.Socks5Proxy = "127.0.0.1:1080"
socksSession := NewScanSession(socksCfg, NewState(), &FlagVars{})
if !socksSession.ProxyEnabled() {
t.Fatal("SOCKS5 proxy session should report proxy enabled")
}
if !socksSession.IsSOCKS5Proxy() {
t.Fatal("SOCKS5 proxy session should report SOCKS5")
}
}
func TestParseProxyURLFallsBackWhenHostIsEmpty(t *testing.T) {
host, username, password := parseProxyURL("127.0.0.1:8080", "127.0.0.1:8080")
if host != "127.0.0.1:8080" {
t.Fatalf("host = %q, want fallback address", host)
}
if username != "" || password != "" {
t.Fatalf("unexpected credentials: %q/%q", username, password)
}
}
func TestParseProxyURLExtractsAuthWithoutScheme(t *testing.T) {
host, username, password := parseProxyURL("user:[email protected]:8080", "user:[email protected]:8080")
if host != "127.0.0.1:8080" {
t.Fatalf("host = %q, want proxy address", host)
}
if username != "user" || password != "pass" {
t.Fatalf("credentials = %q/%q, want user/pass", username, password)
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
+67 -49
View File
@@ -23,17 +23,17 @@ state.go - 运行时状态管理
// State 扫描器运行时状态 - 线程安全
type State struct {
// 计数器 - 原子操作
packetCount int64
tcpPacketCount int64
tcpSuccessPacketCount int64
tcpFailedPacketCount int64
udpPacketCount int64
httpPacketCount int64
resourceExhaustedCount int64
packetCount atomic.Int64
tcpPacketCount atomic.Int64
tcpSuccessPacketCount atomic.Int64
tcpFailedPacketCount atomic.Int64
udpPacketCount atomic.Int64
httpPacketCount atomic.Int64
resourceExhaustedCount atomic.Int64
// 任务计数
end int64
num int64
end atomic.Int64
num atomic.Int64
// 时间
startTime time.Time
@@ -56,6 +56,10 @@ type State struct {
forwardShellActive int32 // 使用int32以便原子操作
reverseShellActive int32
socks5ProxyActive int32
// 服务识别缓存(per-session,避免跨扫描污染)
// key: "host:port", value: interface{}core.ServiceInfo 指针)
serviceCache sync.Map
}
// NewState 创建新的状态对象
@@ -71,38 +75,38 @@ func NewState() *State {
// IncrementPacketCount 增加总包计数
func (s *State) IncrementPacketCount() int64 {
return atomic.AddInt64(&s.packetCount, 1)
return s.packetCount.Add(1)
}
// IncrementTCPSuccessPacketCount 增加TCP成功连接包计数
func (s *State) IncrementTCPSuccessPacketCount() int64 {
atomic.AddInt64(&s.tcpSuccessPacketCount, 1)
atomic.AddInt64(&s.tcpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.tcpSuccessPacketCount.Add(1)
s.tcpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementTCPFailedPacketCount 增加TCP失败连接包计数
func (s *State) IncrementTCPFailedPacketCount() int64 {
atomic.AddInt64(&s.tcpFailedPacketCount, 1)
atomic.AddInt64(&s.tcpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.tcpFailedPacketCount.Add(1)
s.tcpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementUDPPacketCount 增加UDP包计数
func (s *State) IncrementUDPPacketCount() int64 {
atomic.AddInt64(&s.udpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.udpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementHTTPPacketCount 增加HTTP包计数
func (s *State) IncrementHTTPPacketCount() int64 {
atomic.AddInt64(&s.httpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.httpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementResourceExhaustedCount 增加资源耗尽错误计数
func (s *State) IncrementResourceExhaustedCount() {
atomic.AddInt64(&s.resourceExhaustedCount, 1)
s.resourceExhaustedCount.Add(1)
}
// =============================================================================
@@ -111,48 +115,48 @@ func (s *State) IncrementResourceExhaustedCount() {
// GetPacketCount 获取总包计数
func (s *State) GetPacketCount() int64 {
return atomic.LoadInt64(&s.packetCount)
return s.packetCount.Load()
}
// GetTCPPacketCount 获取TCP包计数
func (s *State) GetTCPPacketCount() int64 {
return atomic.LoadInt64(&s.tcpPacketCount)
return s.tcpPacketCount.Load()
}
// GetTCPSuccessPacketCount 获取TCP成功连接包计数
func (s *State) GetTCPSuccessPacketCount() int64 {
return atomic.LoadInt64(&s.tcpSuccessPacketCount)
return s.tcpSuccessPacketCount.Load()
}
// GetTCPFailedPacketCount 获取TCP失败连接包计数
func (s *State) GetTCPFailedPacketCount() int64 {
return atomic.LoadInt64(&s.tcpFailedPacketCount)
return s.tcpFailedPacketCount.Load()
}
// GetUDPPacketCount 获取UDP包计数
func (s *State) GetUDPPacketCount() int64 {
return atomic.LoadInt64(&s.udpPacketCount)
return s.udpPacketCount.Load()
}
// GetHTTPPacketCount 获取HTTP包计数
func (s *State) GetHTTPPacketCount() int64 {
return atomic.LoadInt64(&s.httpPacketCount)
return s.httpPacketCount.Load()
}
// GetResourceExhaustedCount 获取资源耗尽错误计数
func (s *State) GetResourceExhaustedCount() int64 {
return atomic.LoadInt64(&s.resourceExhaustedCount)
return s.resourceExhaustedCount.Load()
}
// ResetPacketCounters 重置所有包计数器
func (s *State) ResetPacketCounters() {
atomic.StoreInt64(&s.packetCount, 0)
atomic.StoreInt64(&s.tcpPacketCount, 0)
atomic.StoreInt64(&s.tcpSuccessPacketCount, 0)
atomic.StoreInt64(&s.tcpFailedPacketCount, 0)
atomic.StoreInt64(&s.udpPacketCount, 0)
atomic.StoreInt64(&s.httpPacketCount, 0)
atomic.StoreInt64(&s.resourceExhaustedCount, 0)
s.packetCount.Store(0)
s.tcpPacketCount.Store(0)
s.tcpSuccessPacketCount.Store(0)
s.tcpFailedPacketCount.Store(0)
s.udpPacketCount.Store(0)
s.httpPacketCount.Store(0)
s.resourceExhaustedCount.Store(0)
}
// =============================================================================
@@ -161,32 +165,32 @@ func (s *State) ResetPacketCounters() {
// GetEnd 获取结束计数
func (s *State) GetEnd() int64 {
return atomic.LoadInt64(&s.end)
return s.end.Load()
}
// GetNum 获取数量计数
func (s *State) GetNum() int64 {
return atomic.LoadInt64(&s.num)
return s.num.Load()
}
// IncrementEnd 增加结束计数
func (s *State) IncrementEnd() int64 {
return atomic.AddInt64(&s.end, 1)
return s.end.Add(1)
}
// IncrementNum 增加数量计数
func (s *State) IncrementNum() int64 {
return atomic.AddInt64(&s.num, 1)
return s.num.Add(1)
}
// SetEnd 设置结束计数
func (s *State) SetEnd(val int64) {
atomic.StoreInt64(&s.end, val)
s.end.Store(val)
}
// SetNum 设置数量计数
func (s *State) SetNum(val int64) {
atomic.StoreInt64(&s.num, val)
s.num.Store(val)
}
// =============================================================================
@@ -271,10 +275,10 @@ type PerfStatsData struct {
func (s *State) GetPerfStats() PerfStatsData {
duration := time.Since(s.startTime)
durationMs := duration.Milliseconds()
totalPackets := atomic.LoadInt64(&s.packetCount)
tcpSuccess := atomic.LoadInt64(&s.tcpSuccessPacketCount)
tcpFailed := atomic.LoadInt64(&s.tcpFailedPacketCount)
tcpTotal := atomic.LoadInt64(&s.tcpPacketCount)
totalPackets := s.packetCount.Load()
tcpSuccess := s.tcpSuccessPacketCount.Load()
tcpFailed := s.tcpFailedPacketCount.Load()
tcpTotal := s.tcpPacketCount.Load()
var pps float64
if durationMs > 0 {
@@ -291,13 +295,13 @@ func (s *State) GetPerfStats() PerfStatsData {
TCPPackets: tcpTotal,
TCPSuccess: tcpSuccess,
TCPFailed: tcpFailed,
UDPPackets: atomic.LoadInt64(&s.udpPacketCount),
HTTPPackets: atomic.LoadInt64(&s.httpPacketCount),
ResourceExhausted: atomic.LoadInt64(&s.resourceExhaustedCount),
UDPPackets: s.udpPacketCount.Load(),
HTTPPackets: s.httpPacketCount.Load(),
ResourceExhausted: s.resourceExhaustedCount.Load(),
ScanDurationMs: durationMs,
PacketsPerSecond: pps,
SuccessRate: successRate,
TargetsScanned: atomic.LoadInt64(&s.num),
TargetsScanned: s.num.Load(),
}
}
@@ -455,3 +459,17 @@ func (s *State) CheckAndIncrementPacketRate(rateLimit int64) (bool, error) {
return true, nil
}
// =============================================================================
// 服务识别缓存 - per-session,消除跨扫描污染
// =============================================================================
// CacheService 缓存服务信息
func (s *State) CacheService(key string, info interface{}) {
s.serviceCache.Store(key, info)
}
// GetCachedService 获取缓存的服务信息
func (s *State) GetCachedService(key string) (interface{}, bool) {
return s.serviceCache.Load(key)
}
+250 -85
View File
@@ -1,147 +1,312 @@
package core
import (
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/panjf2000/ants/v2"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// AdaptivePool 自适应线程池
// 封装 ants.PoolWithFunc,支持根据资源耗尽率动态调整线程数
// HealthSignal 健康评估结果
type HealthSignal int
const (
HealthUnknown HealthSignal = iota // 样本不足,无法判断
HealthGood // 一切正常,可以提速
HealthOK // 正常,维持现状
HealthStressed // 有压力信号,轻微降速
HealthCongested // 明确拥塞,大幅降速
)
// AdaptivePool 自适应线程池(AIMD + 慢启动)
//
// 三阶段工作模式:
// 1. 慢启动:从 target/4 起步,每个检查周期翻倍,直到达到 target 或检测到拥塞
// 2. 稳态 AIMD:健康时加性增(+5% target),拥塞时乘性减(×0.5)
// 3. 恢复上限受 ceiling 约束,不会无限增长
//
// 健康评估基于两个信号:
// - 资源耗尽率(fd/端口不足)
// - RTT 趋势(fast EMA / slow EMA
type AdaptivePool struct {
pool *ants.PoolWithFunc
state *common.State
pool *ants.PoolWithFunc
metrics *ScanMetrics
initialSize int
minSize int
maxSize int
currentSize int32 // 原子操作
// 网络环境(影响健康评估阈值)
networkEnv NetworkEnv
// 监控参数
checkInterval time.Duration
lastCheck time.Time
lastExhaustedCount int64
lastPacketCount int64
// 并发控制
target int32 // 探测推荐的目标值
ceiling int32 // 绝对上限(用户指定或探测推荐)
currentSize int32
// 阈值
exhaustedThreshold float64 // 资源耗尽率阈值(触发降级)
recoveryThreshold float64 // 恢复阈值(允许升级
// 慢启动
inSlowStart bool
ssThreshold int32 // 慢启动阈值(拥塞后降为当前值
mu sync.Mutex
// 检查定时
checkInterval time.Duration
lastCheck atomic.Int64 // UnixNano
// 增量计算
mu sync.Mutex
prevSnapshot MetricsSnapshot
}
// NewAdaptivePool 创建自适应线程池
func NewAdaptivePool(size int, fn func(interface{}), state *common.State) (*AdaptivePool, error) {
// 移除 WithPreAlloc(true),在大规模扫描时预分配可能导致内存问题
pool, err := ants.NewPoolWithFunc(size, fn)
// target: 目标并发数(来自 NetworkProfile.RecommendConcurrency
// ceiling: 最大并发上限
// metrics: 共享的扫描度量(scanSinglePort 写入,pool 读取)
func NewAdaptivePool(target, ceiling int, fn func(interface{}), metrics *ScanMetrics, env ...NetworkEnv) (*AdaptivePool, error) {
// 慢启动初始值:target 的 25%,但不低于 10
initial := target / 4
if initial < 10 {
initial = 10
}
if initial > target {
initial = target
}
pool, err := ants.NewPoolWithFunc(initial, fn)
if err != nil {
return nil, err
}
minSize := size / 4
if minSize < 10 {
minSize = 10
netEnv := EnvWAN
if len(env) > 0 {
netEnv = env[0]
}
return &AdaptivePool{
pool: pool,
state: state,
initialSize: size,
minSize: minSize,
maxSize: size,
currentSize: int32(size),
checkInterval: time.Second,
exhaustedThreshold: 0.10, // 10% 资源耗尽率触发降级
recoveryThreshold: 0.02, // 2% 以下允许恢复
pool: pool,
metrics: metrics,
networkEnv: netEnv,
target: int32(target),
ceiling: int32(ceiling),
currentSize: int32(initial),
inSlowStart: true,
ssThreshold: int32(target),
checkInterval: 500 * time.Millisecond,
}, nil
}
// Invoke 提交任务,并在适当时机检查是否需要调整线程数
// Invoke 提交任务
func (ap *AdaptivePool) Invoke(task interface{}) error {
ap.maybeAdjust()
return ap.pool.Invoke(task)
}
// maybeAdjust 检查并可能调整线程池大小
// maybeAdjust 周期性检查并调整并发数
func (ap *AdaptivePool) maybeAdjust() {
now := time.Now()
ap.mu.Lock()
if now.Sub(ap.lastCheck) < ap.checkInterval {
ap.mu.Unlock()
last := ap.lastCheck.Load()
now := time.Now().UnixNano()
if now-last < int64(ap.checkInterval) {
return
}
ap.lastCheck = now
// 获取当前计数
currentExhausted := ap.state.GetResourceExhaustedCount()
currentPackets := ap.state.GetPacketCount()
// 计算增量(本周期内的耗尽率)
deltaExhausted := currentExhausted - ap.lastExhaustedCount
deltaPackets := currentPackets - ap.lastPacketCount
ap.lastExhaustedCount = currentExhausted
ap.lastPacketCount = currentPackets
ap.mu.Unlock()
// 需要足够的样本才能判断
if deltaPackets < 100 {
if !ap.lastCheck.CompareAndSwap(last, now) {
return
}
rate := float64(deltaExhausted) / float64(deltaPackets)
currentSize := int(atomic.LoadInt32(&ap.currentSize))
ap.adjust()
}
if rate > ap.exhaustedThreshold && currentSize > ap.minSize {
// 降级:减少 20% 线程
newSize := int(float64(currentSize) * 0.8)
if newSize < ap.minSize {
newSize = ap.minSize
}
func (ap *AdaptivePool) adjust() {
health := ap.assessHealth()
if health == HealthUnknown {
return
}
// RTT 漂移微调:fast EMA 远高于 slow EMA 说明延迟持续恶化
// 压低 target 让 AIMD 的天花板跟着降,而不是只靠乘性减
ap.maybeReduceTarget()
current := int(atomic.LoadInt32(&ap.currentSize))
target := int(atomic.LoadInt32(&ap.target))
ceiling := int(atomic.LoadInt32(&ap.ceiling))
var newSize int
if ap.inSlowStart {
newSize = ap.adjustSlowStart(health, current, target)
} else {
newSize = ap.adjustAIMD(health, current, target)
}
// 下限:ceiling 的 5%,但不低于 10
minSize := ceiling / 20
if minSize < 10 {
minSize = 10
}
if newSize < minSize {
newSize = minSize
}
if newSize > ceiling {
newSize = ceiling
}
if newSize != current {
ap.tune(newSize)
common.LogInfo(fmt.Sprintf("[AdaptivePool] 资源耗尽率 %.1f%%, 线程数 %d -> %d", rate*100, currentSize, newSize))
} else if rate < ap.recoveryThreshold && currentSize < ap.maxSize {
// 恢复:增加 10% 线程(保守恢复)
newSize := int(float64(currentSize) * 1.1)
if newSize > ap.maxSize {
newSize = ap.maxSize
// 显著变化时记录日志
delta := newSize - current
if delta < 0 {
delta = -delta
}
if newSize > currentSize {
ap.tune(newSize)
if delta > current/5 {
if newSize < current {
common.LogInfo(i18n.Tr("adaptive_pool_decrease", current, newSize))
} else {
common.LogDebug(i18n.Tr("adaptive_pool_increase", current, newSize))
}
}
}
}
// tune 调整线程池大小
func (ap *AdaptivePool) adjustSlowStart(health HealthSignal, current, target int) int {
switch health {
case HealthCongested, HealthStressed:
// 退出慢启动,设置阈值
ap.ssThreshold = int32(current)
ap.inSlowStart = false
common.LogDebug(i18n.Tr("adaptive_pool_slowstart_exit", current))
return int(float64(current) * 0.5)
default:
// 翻倍
newSize := current * 2
if newSize >= target {
newSize = target
ap.inSlowStart = false
}
return newSize
}
}
func (ap *AdaptivePool) adjustAIMD(health HealthSignal, current, target int) int {
switch health {
case HealthCongested:
// 乘性减:×0.5
newSize := int(float64(current) * 0.5)
ap.ssThreshold = int32(newSize)
return newSize
case HealthStressed:
// 温和降低:×0.85
return int(float64(current) * 0.85)
case HealthGood:
// 加性增:+5% of target,至少 +1
inc := target / 20
if inc < 1 {
inc = 1
}
return current + inc
default:
return current
}
}
// assessHealth 综合健康评估
func (ap *AdaptivePool) assessHealth() HealthSignal {
snap := ap.metrics.Snapshot()
ap.mu.Lock()
prev := ap.prevSnapshot
ap.prevSnapshot = snap
ap.mu.Unlock()
// 计算本周期增量
deltaTotal := snap.Total() - prev.Total()
deltaExhausted := snap.Exhausted - prev.Exhausted
// 样本不足
if deltaTotal < 30 {
return HealthUnknown
}
exhaustRate := float64(deltaExhausted) / float64(deltaTotal)
rttRatio := ap.metrics.RTTRatio()
// 阈值根据网络环境调整:内网收紧,公网放宽
var congestExhaust, stressExhaust, congestRTT, stressRTT, goodRTT float64
switch ap.networkEnv {
case EnvLAN:
congestExhaust, stressExhaust = 0.08, 0.03
congestRTT, stressRTT, goodRTT = 1.8, 1.4, 1.15
case EnvWAN:
congestExhaust, stressExhaust = 0.15, 0.05
congestRTT, stressRTT, goodRTT = 2.5, 1.8, 1.3
default: // Internet / Slow
congestExhaust, stressExhaust = 0.25, 0.10
congestRTT, stressRTT, goodRTT = 3.5, 2.5, 1.5
}
switch {
case exhaustRate > congestExhaust:
return HealthCongested
case rttRatio > congestRTT:
return HealthCongested
case exhaustRate > stressExhaust:
return HealthStressed
case rttRatio > stressRTT:
return HealthStressed
case exhaustRate < 0.01 && rttRatio < goodRTT:
return HealthGood
default:
return HealthOK
}
}
// maybeReduceTarget 当 RTT 持续恶化时压低 target
// 不低于 ceiling 的 20%,避免过度收缩
func (ap *AdaptivePool) maybeReduceTarget() {
rttRatio := ap.metrics.RTTRatio()
if rttRatio <= 3.0 {
return
}
target := atomic.LoadInt32(&ap.target)
ceiling := atomic.LoadInt32(&ap.ceiling)
minTarget := ceiling / 5
if minTarget < 10 {
minTarget = 10
}
// 压低 10%
newTarget := int32(float64(target) * 0.9)
if newTarget < minTarget {
newTarget = minTarget
}
if newTarget < target {
atomic.StoreInt32(&ap.target, newTarget)
}
}
func (ap *AdaptivePool) tune(newSize int) {
ap.pool.Tune(newSize)
atomic.StoreInt32(&ap.currentSize, int32(newSize))
}
// Running 返回当前运行中的 goroutine 数量
func (ap *AdaptivePool) Running() int {
return ap.pool.Running()
}
func (ap *AdaptivePool) Running() int { return ap.pool.Running() }
// Cap 返回当前池容量
func (ap *AdaptivePool) Cap() int {
return int(atomic.LoadInt32(&ap.currentSize))
}
func (ap *AdaptivePool) Cap() int { return int(atomic.LoadInt32(&ap.currentSize)) }
// Release 释放线程池
func (ap *AdaptivePool) Release() {
ap.pool.Release()
}
func (ap *AdaptivePool) Release() { ap.pool.Release() }
// Wait 等待所有任务完成
// Wait 等待所有任务完成(最多等待 10 分钟)
func (ap *AdaptivePool) Wait() {
// ants 没有原生 Wait,通过 Running() == 0 轮询
deadline := time.After(10 * time.Minute)
for ap.pool.Running() > 0 {
time.Sleep(10 * time.Millisecond)
select {
case <-deadline:
common.LogError(i18n.Tr("adaptive_pool_wait_timeout"))
return
default:
time.Sleep(10 * time.Millisecond)
}
}
}
+60 -143
View File
@@ -1,160 +1,101 @@
package core
/*
adaptive_pool_test.go - AdaptivePool 高价值测试
测试重点
1. 并发安全 - 多goroutine同时调整不崩溃
2. 降级逻辑 - 资源耗尽率高时正确减少线程
3. 恢复逻辑 - 资源耗尽率低时正确增加线程
4. 边界条件 - 不超过minSize/maxSize
不测试
- 简单的getter方法太简单不值得
- ants库本身的正确性库作者负责
*/
import (
"testing"
"time"
"github.com/shadow1ng/fscan/common"
)
// =============================================================================
// 场景1:降级逻辑测试(高价值)
// =============================================================================
// TestAdaptivePool_DowngradeOnHighExhaustion 验证资源耗尽率高时降低线程数
// 这是个核心业务逻辑:耗尽率 > 10% 时应该减少线程
func TestAdaptivePool_DowngradeOnHighExhaustion(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(100, func(interface{}) {}, state)
// newTestPool 测试辅助:创建测试用的自适应线程池
func newTestPool(t *testing.T, size int, fn func(interface{})) (*AdaptivePool, *ScanMetrics) {
t.Helper()
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(size, size, fn, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
return pool, metrics
}
// TestAdaptivePool_DowngradeOnHighExhaustion 验证资源耗尽率高时降低线程数
func TestAdaptivePool_DowngradeOnHighExhaustion(t *testing.T) {
pool, metrics := newTestPool(t, 100, func(interface{}) {})
defer pool.Release()
// 慢启动先跑到 target
pool.inSlowStart = false
pool.tune(100)
initialCap := pool.Cap()
// 模拟高资源耗尽率:20% 的包都失败了
// 需要至少100个样本才会触发调整
// 模拟高资源耗尽率:20%
for i := 0; i < 200; i++ {
state.IncrementPacketCount()
if i < 40 { // 前40个失败(20%
state.IncrementResourceExhaustedCount()
if i < 40 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
// 触发调整:提交足够多的任务让maybeAdjust被调用
// 触发调整
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
time.Sleep(time.Millisecond * 10) // 等待异步调整
time.Sleep(time.Millisecond * 30)
}
// 等待调整完成
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:线程数应该减少
if finalCap >= initialCap {
t.Errorf("应该降级: 初始 %d, 最终 %d", initialCap, finalCap)
}
// 验证:不应该降到minSize以下
minSize := initialCap / 4
if minSize < 10 {
minSize = 10
}
if finalCap < minSize {
t.Errorf("降到minSize以下: %d < %d", finalCap, minSize)
if finalCap < 10 {
t.Errorf("降到 minSize 以下: %d", finalCap)
}
t.Logf("降级成功: %d -> %d (min=%d)", initialCap, finalCap, minSize)
t.Logf("降级成功: %d -> %d", initialCap, finalCap)
}
// =============================================================================
// 场景3:恢复逻辑测试(高价值)
// =============================================================================
// TestAdaptivePool_NoRecoveryOnLowExhaustion 验证低耗尽率时不升级
// 防止线程数盲目增长
func TestAdaptivePool_NoRecoveryOnLowExhaustion(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(50, func(interface{}) {}, state)
// TestAdaptivePool_SlowStart 验证慢启动行为
func TestAdaptivePool_SlowStart(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
// 先降到minSize
for i := 0; i < 500; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount() // 100% 耗尽
// 初始应该是 target/4 = 25
initialCap := pool.Cap()
if initialCap > 30 {
t.Errorf("慢启动初始值应该 <= 30, got %d", initialCap)
}
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
reducedCap := pool.Cap()
// 现在模拟低耗尽率:只有1%失败
for i := 0; i < 500; i++ {
state.IncrementPacketCount()
if i%100 == 0 { // 只有5个失败(1%
state.IncrementResourceExhaustedCount()
}
if !pool.inSlowStart {
t.Error("应该处于慢启动状态")
}
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:即使耗尽率低,也不应该立即恢复(保守策略)
// 或者即使恢复,也很有限
if finalCap > reducedCap+5 {
t.Logf("恢复行为: %d -> %d", reducedCap, finalCap)
}
t.Logf("慢启动初始: cap=%d, inSlowStart=%v", initialCap, pool.inSlowStart)
}
// =============================================================================
// 场景4:边界条件测试(中价值)
// =============================================================================
// TestAdaptivePool_MinSizeBoundary 验证不会降到minSize以下
// TestAdaptivePool_MinSizeBoundary 验证不会降到 minSize 以下
func TestAdaptivePool_MinSizeBoundary(t *testing.T) {
state := common.NewState()
// 创建小线程池,minSize会是10
pool, err := NewAdaptivePool(40, func(interface{}) {}, state)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
pool, metrics := newTestPool(t, 40, func(interface{}) {})
defer pool.Release()
// 模拟极端的资源耗尽:100%失败
for i := 0; i < 1000; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount()
pool.inSlowStart = false
pool.tune(40)
// 极端耗尽
for i := 0; i < 500; i++ {
metrics.RecordExhausted()
}
// 触发多次调整
for i := 0; i < 50; i++ {
_ = pool.Invoke(nil)
time.Sleep(time.Millisecond)
time.Sleep(time.Millisecond * 15)
}
finalCap := pool.Cap()
// 验证:不应该低于10
if finalCap < 10 {
t.Errorf("线程数 < 10: %d", finalCap)
}
@@ -162,76 +103,52 @@ func TestAdaptivePool_MinSizeBoundary(t *testing.T) {
t.Logf("最小边界测试通过: cap=%d", finalCap)
}
// =============================================================================
// 场景5:样本不足测试(低价值但重要)
// =============================================================================
// TestAdaptivePool_NotEnoughSamples 验证样本不足时不调整
// 防止基于小样本做错误决策
func TestAdaptivePool_NotEnoughSamples(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(100, func(interface{}) {}, state)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
pool, metrics := newTestPool(t, 100, func(interface{}) {})
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
initialCap := pool.Cap()
// 只增加少量样本(<100),不足以触发调整
for i := 0; i < 50; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount() // 即使100%失败也不调整
// 只 20 个样本,不足 30 的阈值
for i := 0; i < 20; i++ {
metrics.RecordExhausted()
}
// 提交任务
for i := 0; i < 10; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:样本不足时不应该调整
if finalCap != initialCap {
t.Errorf("样本不足时不应该调整: %d -> %d", initialCap, finalCap)
}
}
// =============================================================================
// 辅助函数
// =============================================================================
// TestAdaptivePool_Wait 验证Wait方法正确等待所有任务完成
// TestAdaptivePool_Wait 验证 Wait 方法
func TestAdaptivePool_Wait(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(10, func(interface{}) {
pool, _ := newTestPool(t, 10, func(interface{}) {
time.Sleep(time.Millisecond * 50)
}, state)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
})
defer pool.Release()
// 提交任务
pool.inSlowStart = false
pool.tune(10)
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
// Wait应该在所有任务完成后返回
start := time.Now()
pool.Wait()
duration := time.Since(start)
// 20个任务,每个50ms,10个线程,应该约100ms完成
if duration < 80*time.Millisecond {
t.Logf("Wait提前返回?可能测试有问题: %v", duration)
}
if duration > 200*time.Millisecond {
t.Errorf("Wait耗时过长: %v", duration)
if duration > 300*time.Millisecond {
t.Errorf("Wait 耗时过长: %v", duration)
}
t.Logf("Wait测试通过: %v", duration)
t.Logf("Wait 测试通过: %v", duration)
}
+110
View File
@@ -0,0 +1,110 @@
package core
import (
"math"
"sync"
"time"
)
// AdaptiveTimeout 基于 RTT 采样的自适应超时计算器
// 算法:timeout = mean(RTT) + 4 * stddev(RTT)clamp 到 [min, max]
// 冷启动阶段(样本不足)返回用户配置的固定超时
type AdaptiveTimeout struct {
mu sync.Mutex
samples []float64 // 环形缓冲区,单位 ms
pos int // 写入位置
count int // 已采集总数
size int // 缓冲区容量
minTO time.Duration
maxTO time.Duration
warmup int // 冷启动所需最小样本数
cachedTO time.Duration
dirty bool
}
// NewAdaptiveTimeout 创建自适应超时计算器
// maxTimeout: 用户配置的超时上限(即原始固定超时)
func NewAdaptiveTimeout(maxTimeout time.Duration) *AdaptiveTimeout {
return &AdaptiveTimeout{
samples: make([]float64, 64),
size: 64,
minTO: 100 * time.Millisecond,
maxTO: maxTimeout,
warmup: 10,
}
}
// Record 记录一次成功连接的 RTT
func (a *AdaptiveTimeout) Record(rtt time.Duration) {
a.mu.Lock()
a.samples[a.pos%a.size] = float64(rtt.Milliseconds())
a.pos++
a.count++
a.dirty = true
a.mu.Unlock()
}
// Timeout 获取当前推荐超时值
// 样本不足时返回 maxTO(冷启动)
// 锁外执行均值/标准差计算,减少锁持有时间
func (a *AdaptiveTimeout) Timeout() time.Duration {
a.mu.Lock()
if a.count < a.warmup {
a.mu.Unlock()
return a.maxTO
}
if !a.dirty {
cached := a.cachedTO
a.mu.Unlock()
return cached
}
n := a.size
if a.count < a.size {
n = a.count
}
// 拷贝样本到本地,释放锁后再计算
localSamples := make([]float64, n)
start := a.pos % a.size
if a.count < a.size {
copy(localSamples, a.samples[:n])
} else {
copy(localSamples[:a.size-start], a.samples[start:])
copy(localSamples[a.size-start:], a.samples[:start])
}
a.mu.Unlock()
// 锁外计算
var sum float64
for _, s := range localSamples {
sum += s
}
mean := sum / float64(n)
var variance float64
for _, s := range localSamples {
d := s - mean
variance += d * d
}
stddev := math.Sqrt(variance / float64(n))
ms := mean + 4*stddev
to := time.Duration(ms) * time.Millisecond
if to < a.minTO {
to = a.minTO
}
if to > a.maxTO {
to = a.maxTO
}
// 短暂加锁更新缓存
a.mu.Lock()
a.cachedTO = to
a.dirty = false
a.mu.Unlock()
return to
}
+37 -33
View File
@@ -1,6 +1,7 @@
package core
import (
"context"
"fmt"
"sync"
"time"
@@ -37,7 +38,7 @@ type AliveStats struct {
// NewAliveScanStrategy 创建新的存活探测扫描策略
func NewAliveScanStrategy() *AliveScanStrategy {
return &AliveScanStrategy{
BaseScanStrategy: NewBaseScanStrategy("存活探测", FilterNone),
BaseScanStrategy: NewBaseScanStrategy(i18n.GetText("scan_strategy_alive_name"), FilterNone),
startTime: time.Now(),
}
}
@@ -53,65 +54,68 @@ func (s *AliveScanStrategy) Description() string {
}
// Execute 执行存活探测扫描策略
func (s *AliveScanStrategy) Execute(config *common.Config, state *common.State, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
func (s *AliveScanStrategy) Execute(ctx context.Context, session *common.ScanSession, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
// 验证扫描目标(需要同时检查 -h 和 -hf 参数)
fv := common.GetFlagVars()
if info.Host == "" && fv.HostsFile == "" {
common.LogError(i18n.GetText("parse_error_target_empty"))
if info.Host == "" && session.Params.HostsFile == "" {
session.LogError(i18n.GetText("parse_error_target_empty"))
return
}
// 执行存活探测
s.performAliveScan(info, config, state)
// 输出统计信息
s.outputStats()
s.performAliveScan(ctx, info, session)
}
// performAliveScan 执行存活探测
func (s *AliveScanStrategy) performAliveScan(info common.HostInfo, config *common.Config, state *common.State) {
// 解析目标主机
fv := common.GetFlagVars()
hosts, err := parsers.ParseIP(info.Host, fv.HostsFile, fv.ExcludeHosts)
func (s *AliveScanStrategy) performAliveScan(ctx context.Context, info common.HostInfo, session *common.ScanSession) {
excludes, err := loadHostExcludes(session.Params)
if err != nil {
common.LogError(i18n.Tr("parse_target_failed", err))
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
if len(hosts) == 0 {
common.LogError(i18n.GetText("parse_error_no_hosts"))
iter, err := parsers.NewHostIterator(info.Host, session.Params.HostsFile, excludes...)
if err != nil {
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
defer func() {
_ = iter.Close()
}()
// 初始化统计信息
s.stats.TotalHosts = len(hosts)
s.stats.TotalHosts = 0
s.stats.AliveHosts = 0
s.stats.DeadHosts = 0
for {
hosts, err := iter.NextBatch(ctx, targetHostBatchSize(session.Config))
if err != nil {
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
if len(hosts) == 0 {
break
}
// 执行存活检测
aliveList := CheckLive(hosts, false, config, state) // 使用ICMP探测
s.stats.TotalHosts += len(hosts)
aliveList := CheckLive(ctx, hosts, false, session)
s.stats.AliveHosts += len(aliveList)
for _, host := range aliveList {
session.LogSuccess(fmt.Sprintf("alive %s", host))
}
}
if s.stats.TotalHosts == 0 {
session.LogError(i18n.GetText("parse_error_no_hosts"))
return
}
// 更新统计信息
s.stats.AliveHosts = len(aliveList)
s.stats.DeadHosts = s.stats.TotalHosts - s.stats.AliveHosts
s.stats.ScanDuration = time.Since(s.startTime)
s.stats.AliveHostList = aliveList // 存储存活主机列表
if s.stats.TotalHosts > 0 {
s.stats.SuccessRate = float64(s.stats.AliveHosts) / float64(s.stats.TotalHosts) * 100
}
}
// outputStats 输出统计信息(精简版)
func (s *AliveScanStrategy) outputStats() {
// 只输出存活主机列表,不输出冗余统计
for _, host := range s.stats.AliveHostList {
common.LogSuccess(fmt.Sprintf("alive %s", host))
}
}
// PrepareTargets 存活探测不需要准备扫描目标
func (s *AliveScanStrategy) PrepareTargets(info common.HostInfo) []common.HostInfo {
// 存活探测不需要返回目标列表,因为它不进行后续扫描
+69 -47
View File
@@ -29,6 +29,7 @@ const (
type BaseScanStrategy struct {
strategyName string
filterType PluginFilterType
state *common.State
}
// NewBaseScanStrategy 创建基础扫描策略
@@ -39,6 +40,11 @@ func NewBaseScanStrategy(name string, filterType PluginFilterType) *BaseScanStra
}
}
// SetState 注入 session state(用于 per-session 服务缓存)
func (b *BaseScanStrategy) SetState(state *common.State) {
b.state = state
}
// GetPlugins 获取插件列表
func (b *BaseScanStrategy) GetPlugins(config *common.Config) ([]string, bool) {
scanMode := config.Mode
@@ -81,6 +87,16 @@ func (b *BaseScanStrategy) IsPluginApplicableByName(pluginName string, targetHos
return false
}
// 显式指定插件时,尊重调用方选择,不再强制使用插件默认端口过滤。
if isCustomMode {
return b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
// -full 模式下,web 插件对所有开放端口生效(跳过 IsMarkedWebService 检查)
if config.POC.Full && b.isWebPlugin(pluginName) {
return b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
// 检查端口匹配和过滤器类型
return b.isPluginApplicableToPortWithHost(pluginName, targetHost, targetPort) && b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
@@ -101,14 +117,19 @@ func (b *BaseScanStrategy) isLocalPlugin(pluginName string) bool {
return plugins.HasType(pluginName, plugins.PluginTypeLocal)
}
func (b *BaseScanStrategy) isUDPPlugin(pluginName string) bool {
return plugins.IsUDP(pluginName)
}
func (b *BaseScanStrategy) isLocalPluginExplicitlySpecified(pluginName string, config *common.Config) bool {
return config.LocalPlugin == pluginName
}
// isPluginApplicableToPortWithHost 检查插件是否适用于指定端口
// 匹配策略:端口匹配 → 服务名称匹配(解决非标准端口问题)
func (b *BaseScanStrategy) isPluginApplicableToPortWithHost(pluginName string, targetHost string, targetPort int) bool {
if b.isWebPlugin(pluginName) {
return IsMarkedWebService(targetHost, targetPort)
return IsMarkedWebServiceWithState(b.state, targetHost, targetPort)
}
pluginPorts := b.getPluginPorts(pluginName)
@@ -127,15 +148,33 @@ func (b *BaseScanStrategy) isPluginApplicableToPortWithHost(pluginName string, t
}
}
// 端口不匹配时,按指纹识别结果匹配
// 例:8881 端口上识别到 ssh 服务 → ssh 插件应该执行
if targetHost != "" && targetPort > 0 {
if info, ok := GetCachedServiceInfoWithState(b.state, targetHost, targetPort); ok && info != nil {
if strings.EqualFold(info.Name, pluginName) {
return true
}
}
}
return false
}
func (b *BaseScanStrategy) isPluginApplicableToPort(pluginName string, targetPort int) bool {
if b.isWebPlugin(pluginName) {
return true
}
return b.isPluginApplicableToPortWithHost(pluginName, "", targetPort)
}
// isPluginPassesFilterType 检查插件是否通过过滤器类型检查
func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomMode bool, config *common.Config) bool {
// UDP 插件有独立分发路径,不参与 TCP 端口匹配流水线
if b.isUDPPlugin(pluginName) {
return false
}
// 自定义模式下强制运行所有明确指定的插件
if isCustomMode {
return true
@@ -150,8 +189,8 @@ func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomM
}
return false
case FilterService:
// 服务扫描策略:排除本地插件
return !b.isLocalPlugin(pluginName)
// 服务扫描策略:排除本地插件和UDP插件(UDP有独立分发路径)
return !b.isLocalPlugin(pluginName) && !b.isUDPPlugin(pluginName)
case FilterWeb:
// Web扫描策略:只允许Web插件
return b.isWebPlugin(pluginName)
@@ -164,26 +203,9 @@ func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomM
}
}
// LogPluginInfo 输出插件信息
func (b *BaseScanStrategy) LogPluginInfo(config *common.Config) {
allPlugins, isCustomMode := b.GetPlugins(config)
var prefix string
switch b.filterType {
case FilterLocal:
prefix = i18n.GetText("concurrency_local_plugin")
case FilterService:
prefix = i18n.GetText("concurrency_service_plugin")
case FilterWeb:
prefix = i18n.GetText("concurrency_web_plugin")
default:
prefix = i18n.GetText("concurrency_plugin")
}
// 插件信息不再输出,减少干扰
_ = allPlugins
_ = isCustomMode
_ = prefix
// LogPluginInfo 默认不输出插件信息(service 默认端口模式有意保持安静,减少干扰)。
// 子类 LocalScanStrategy / ServiceScanStrategy 按需重写。
func (b *BaseScanStrategy) LogPluginInfo(config *common.Config, session *common.ScanSession) {
}
// formatPluginList 格式化插件列表(超过5个时精简显示)
@@ -191,7 +213,7 @@ func formatPluginList(plugins []string) string {
if len(plugins) <= 5 {
return strings.Join(plugins, ", ")
}
return fmt.Sprintf("%s ... 等%d个", strings.Join(plugins[:5], ", "), len(plugins))
return i18n.Tr("plugin_list_summary", strings.Join(plugins[:5], ", "), len(plugins))
}
// ValidateConfiguration 验证扫描配置
@@ -200,14 +222,14 @@ func (b *BaseScanStrategy) ValidateConfiguration() error {
}
// LogScanStart 输出扫描开始信息(已精简,仅在非服务扫描模式下显示)
func (b *BaseScanStrategy) LogScanStart() {
func (b *BaseScanStrategy) LogScanStart(session *common.ScanSession) {
// 服务扫描模式下不显示(插件信息已足够说明)
// 仅在本地/Web等特殊模式下显示
switch b.filterType {
case FilterLocal:
common.LogInfo(i18n.GetText("start_local_scan"))
session.LogInfo(i18n.GetText("start_local_scan"))
case FilterWeb:
common.LogInfo(i18n.GetText("start_web_scan"))
session.LogInfo(i18n.GetText("start_web_scan"))
}
}
@@ -225,9 +247,9 @@ func (b *BaseScanStrategy) getPluginsByFilterType() []string {
}
}
case FilterService:
// 服务扫描策略:排除本地插件和纯Web插件,保留服务插件
// 服务扫描策略:排除本地插件和UDP插件,保留TCP服务插件
for _, pluginName := range allPlugins {
if !b.isLocalPlugin(pluginName) {
if !b.isLocalPlugin(pluginName) && !b.isUDPPlugin(pluginName) {
filteredPlugins = append(filteredPlugins, pluginName)
}
}
@@ -238,32 +260,32 @@ func (b *BaseScanStrategy) getPluginsByFilterType() []string {
filteredPlugins = append(filteredPlugins, pluginName)
}
}
// 确保 webtitle 在 webpoc 之前执行,避免指纹识别竞态
sort.Slice(filteredPlugins, func(i, j int) bool {
// webtitle 必须在 webpoc 之前
if filteredPlugins[i] == "webtitle" {
return true
}
if filteredPlugins[j] == "webtitle" {
return false
}
if filteredPlugins[i] == "webpoc" {
return false
}
if filteredPlugins[j] == "webpoc" {
return true
}
// 其他插件保持字母顺序
return filteredPlugins[i] < filteredPlugins[j]
})
default:
// 无过滤器:返回所有插件
filteredPlugins = allPlugins
}
orderWebPlugins(filteredPlugins)
return filteredPlugins
}
func orderWebPlugins(pluginNames []string) {
sort.SliceStable(pluginNames, func(i, j int) bool {
return webPluginOrder(pluginNames[i]) < webPluginOrder(pluginNames[j])
})
}
func webPluginOrder(pluginName string) int {
switch pluginName {
case "webtitle":
return 0
case "webpoc":
return 2
default:
return 1
}
}
// parsePluginList 解析插件列表字符串
func parsePluginList(pluginStr string) []string {
if pluginStr == "" {
+110
View File
@@ -2,6 +2,9 @@ package core
import (
"testing"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/plugins"
)
// =============================================================================
@@ -261,6 +264,113 @@ func slicesEqual(a, b []string) bool {
return true
}
func TestOrderWebPlugins(t *testing.T) {
plugins := []string{"ssh", "webpoc", "redis", "webtitle", "mysql"}
orderWebPlugins(plugins)
expected := []string{"webtitle", "ssh", "redis", "mysql", "webpoc"}
if !slicesEqual(plugins, expected) {
t.Fatalf("orderWebPlugins = %#v, want %#v", plugins, expected)
}
}
func TestBaseScanStrategyPluginSelectionAndApplicability(t *testing.T) {
registerTestPlugins(t)
plugins.RegisterWithOptions("core_test_local", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeLocal}, false)
plugins.RegisterWithOptions("core_test_udp", func() plugins.Plugin { return nil }, []int{161}, []string{plugins.PluginTypeUDP}, true)
clearServiceCache()
cfg := common.NewConfig()
cfg.Mode = "ssh, missing_plugin, webtitle"
strategy := NewBaseScanStrategy("service", FilterService)
got, custom := strategy.GetPlugins(cfg)
if !custom {
t.Fatal("explicit mode should be marked as custom")
}
if !slicesEqual(got, []string{"ssh", "webtitle"}) {
t.Fatalf("custom plugins = %#v, want ssh/webtitle", got)
}
cfg.Mode = "all"
servicePlugins, custom := strategy.GetPlugins(cfg)
if custom {
t.Fatal("all mode should not be custom")
}
if !containsString(servicePlugins, "ssh") || containsString(servicePlugins, "core_test_local") || containsString(servicePlugins, "core_test_udp") {
t.Fatalf("service filtered plugins = %#v", servicePlugins)
}
if !strategy.pluginExists("ssh") || strategy.pluginExists("missing_plugin") {
t.Fatal("pluginExists returned wrong result")
}
if !strategy.isPluginApplicableToPort("ssh", 22) || strategy.isPluginApplicableToPort("ssh", 23) {
t.Fatal("port applicability for ssh is wrong")
}
CacheServiceInfo("10.0.0.9", 22222, &ServiceInfo{Name: "ssh"})
if !strategy.isPluginApplicableToPortWithHost("ssh", "10.0.0.9", 22222) {
t.Fatal("service cache should allow ssh on a non-standard port")
}
if !strategy.IsPluginApplicableByName("ssh", "10.0.0.9", 1, true, cfg) {
t.Fatal("custom mode should respect explicitly selected plugin")
}
if strategy.IsPluginApplicableByName("missing_plugin", "10.0.0.9", 22, true, cfg) {
t.Fatal("missing plugin should never be applicable")
}
}
func TestBaseScanStrategyFilterTypes(t *testing.T) {
plugins.RegisterWithOptions("core_test_local_filter", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeLocal}, false)
plugins.RegisterWithOptions("core_test_web_filter", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeWeb}, true)
plugins.RegisterWithOptions("core_test_udp_filter", func() plugins.Plugin { return nil }, []int{53}, []string{plugins.PluginTypeUDP}, true)
cfg := common.NewConfig()
localStrategy := NewBaseScanStrategy("local", FilterLocal)
if localStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) {
t.Fatal("local plugin should require explicit -local selection")
}
cfg.LocalPlugin = "core_test_local_filter"
if !localStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) {
t.Fatal("explicit local plugin should pass local filter")
}
serviceStrategy := NewBaseScanStrategy("service", FilterService)
if !serviceStrategy.isPluginPassesFilterType("ssh", false, cfg) {
t.Fatal("service plugin should pass service filter")
}
if serviceStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) ||
serviceStrategy.isPluginPassesFilterType("core_test_udp_filter", false, cfg) {
t.Fatal("service filter should reject local and UDP plugins")
}
webStrategy := NewBaseScanStrategy("web", FilterWeb)
if !webStrategy.isPluginPassesFilterType("core_test_web_filter", false, cfg) ||
webStrategy.isPluginPassesFilterType("ssh", false, cfg) {
t.Fatal("web filter should only allow web plugins")
}
if webPluginOrder("webtitle") != 0 || webPluginOrder("webpoc") != 2 || webPluginOrder("other") != 1 {
t.Fatal("web plugin order changed")
}
}
func TestFormatPluginList(t *testing.T) {
if got := formatPluginList([]string{"a", "b", "c"}); got != "a, b, c" {
t.Fatalf("short plugin list = %q", got)
}
if got := formatPluginList([]string{"a", "b", "c", "d", "e", "f"}); got == "" || got == "a, b, c, d, e, f" {
t.Fatalf("long plugin list should be summarized, got %q", got)
}
}
func containsString(values []string, want string) bool {
for _, value := range values {
if value == want {
return true
}
}
return false
}
// TestNewBaseScanStrategy 测试构造函数
func TestNewBaseScanStrategy(t *testing.T) {
tests := []struct {
-66
View File
@@ -1,66 +0,0 @@
package core
import (
"hash/fnv"
)
// BloomFilter 布隆过滤器,用于ICMP包去重
type BloomFilter struct {
bits []bool
size uint32
k uint32 // hash函数数量
}
// NewBloomFilter 创建布隆过滤器
// size: 预期元素数量
// falsePositiveRate: 期望的误判率(通常0.01即1%)
func NewBloomFilter(size int, falsePositiveRate float64) *BloomFilter {
// 计算最优bit数组大小: m = -n*ln(p) / (ln(2)^2)
// 简化计算:m ≈ n * 10 for p=0.01
m := uint32(size * 10)
if m < 1024 {
m = 1024 // 最小1KB
}
// 计算最优hash函数数量: k = (m/n) * ln(2)
// 简化:k ≈ 7 for p=0.01
k := uint32(7)
return &BloomFilter{
bits: make([]bool, m),
size: m,
k: k,
}
}
// Add 添加元素到过滤器
func (bf *BloomFilter) Add(data string) {
for i := uint32(0); i < bf.k; i++ {
pos := bf.hash(data, i)
bf.bits[pos] = true
}
}
// Contains 检查元素是否可能存在
// 返回true:可能存在(有误判可能)
// 返回false:一定不存在
func (bf *BloomFilter) Contains(data string) bool {
for i := uint32(0); i < bf.k; i++ {
pos := bf.hash(data, i)
if !bf.bits[pos] {
return false
}
}
return true
}
// hash 计算hash值
func (bf *BloomFilter) hash(data string, seed uint32) uint32 {
h := fnv.New32a()
_, _ = h.Write([]byte(data))
// 添加seed实现多个hash函数
for i := uint32(0); i < seed; i++ {
_, _ = h.Write([]byte{byte(i)})
}
return h.Sum32() % bf.size
}
-168
View File
@@ -1,168 +0,0 @@
package core
import (
"fmt"
"testing"
)
/*
bloom_filter_test.go - BloomFilter 高价值测试
测试重点
1. 基本正确性 - Add后Contains返回true未添加的返回false
2. 误判率验证 - 实际误判率应接近理论值(1%)
3. 大规模数据 - 模拟真实ICMP去重场景
不测试
- 内部哈希实现细节
- 精确的数学公式验证
*/
// TestBloomFilter_BasicCorrectness 基本正确性测试
func TestBloomFilter_BasicCorrectness(t *testing.T) {
bf := NewBloomFilter(1000, 0.01)
// 添加元素后应该能找到
testData := []string{
"192.168.1.1",
"10.0.0.1",
"172.16.0.1",
}
for _, data := range testData {
bf.Add(data)
}
for _, data := range testData {
if !bf.Contains(data) {
t.Errorf("已添加的元素 %s 应该返回 true", data)
}
}
// 未添加的元素(大概率)返回false
notAdded := []string{
"8.8.8.8",
"1.1.1.1",
"255.255.255.255",
}
falsePositives := 0
for _, data := range notAdded {
if bf.Contains(data) {
falsePositives++
}
}
// 3个未添加元素全部误判的概率极低(<0.0001%
if falsePositives == len(notAdded) {
t.Error("所有未添加元素都返回true,布隆过滤器可能有问题")
}
}
// TestBloomFilter_FalsePositiveRate 误判率验证
//
// 对于 n=10000, p=0.01 的布隆过滤器:
// 实际误判率应该在 0.5% - 2% 之间(允许统计波动)
func TestBloomFilter_FalsePositiveRate(t *testing.T) {
n := 10000 // 添加的元素数
bf := NewBloomFilter(n, 0.01)
// 添加n个元素
for i := 0; i < n; i++ {
bf.Add(fmt.Sprintf("added_%d", i))
}
// 测试n个未添加的元素
falsePositives := 0
testCount := n
for i := 0; i < testCount; i++ {
if bf.Contains(fmt.Sprintf("not_added_%d", i)) {
falsePositives++
}
}
actualRate := float64(falsePositives) / float64(testCount)
// 允许的误判率范围:0.1% - 3%(考虑统计波动)
if actualRate > 0.03 {
t.Errorf("误判率过高: %.2f%% (期望 < 3%%)", actualRate*100)
}
t.Logf("实际误判率: %.2f%% (%d/%d)", actualRate*100, falsePositives, testCount)
}
// TestBloomFilter_LargeScale 大规模数据测试
//
// 模拟真实的ICMP去重场景:100万个IP地址
func TestBloomFilter_LargeScale(t *testing.T) {
if testing.Short() {
t.Skip("跳过大规模测试")
}
n := 1000000 // 100万
bf := NewBloomFilter(n, 0.01)
// 添加100万个元素
for i := 0; i < n; i++ {
bf.Add(fmt.Sprintf("192.168.%d.%d", i/256, i%256))
}
// 验证已添加的元素
sampleSize := 1000
for i := 0; i < sampleSize; i++ {
idx := i * (n / sampleSize)
data := fmt.Sprintf("192.168.%d.%d", idx/256, idx%256)
if !bf.Contains(data) {
t.Errorf("已添加的元素 %s 返回 false", data)
}
}
// 测试未添加元素的误判率
falsePositives := 0
for i := 0; i < sampleSize; i++ {
if bf.Contains(fmt.Sprintf("10.%d.%d.%d", i/65536, (i/256)%256, i%256)) {
falsePositives++
}
}
actualRate := float64(falsePositives) / float64(sampleSize)
if actualRate > 0.03 {
t.Errorf("大规模场景误判率过高: %.2f%%", actualRate*100)
}
t.Logf("100万元素场景误判率: %.2f%%", actualRate*100)
}
// TestBloomFilter_NoFalseNegative 验证无假阴性
//
// 布隆过滤器的核心保证:已添加的元素必定返回true
func TestBloomFilter_NoFalseNegative(t *testing.T) {
bf := NewBloomFilter(10000, 0.01)
// 添加5000个元素
added := make([]string, 5000)
for i := range added {
added[i] = fmt.Sprintf("element_%d", i)
bf.Add(added[i])
}
// 全部验证
for _, data := range added {
if !bf.Contains(data) {
t.Fatalf("假阴性!已添加的元素 %s 返回 false", data)
}
}
}
// TestBloomFilter_EmptyFilter 空过滤器测试
func TestBloomFilter_EmptyFilter(t *testing.T) {
bf := NewBloomFilter(100, 0.01)
// 空过滤器应该对任何查询返回false
testCases := []string{"anything", "192.168.1.1", ""}
for _, tc := range testCases {
if bf.Contains(tc) {
t.Errorf("空过滤器对 %q 返回 true", tc)
}
}
}
+651
View File
@@ -0,0 +1,651 @@
package core
import (
"math"
"sync"
"testing"
"time"
)
// =============================================================================
// computeRetries 边界
// =============================================================================
func TestComputeRetries_EdgeCases(t *testing.T) {
tests := []struct {
lossRate float64
wantMin int
wantMax int
desc string
}{
{-0.5, 1, 1, "负数丢包率: 视为零"},
{-1.0, 1, 1, "负一: 视为零"},
{0.0, 1, 1, "精确零"},
{0.001, 1, 1, "精确边界 0.001"},
{0.0009, 1, 1, "低于 0.001 边界"},
{0.0011, 1, 5, "高于 0.001 边界"},
{0.95, 5, 5, "精确边界 0.95"},
{0.949, 1, 5, "低于 0.95 边界"},
{0.951, 5, 5, "高于 0.95 边界"},
{1.0, 5, 5, "精确 1.0"},
{1.5, 5, 5, "超过 1.0"},
{100.0, 5, 5, "极大值"},
{math.SmallestNonzeroFloat64, 1, 1, "最小正浮点数"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := computeRetries(tt.lossRate, EnvWAN)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeRetries(%v) = %d, want [%d, %d]",
tt.lossRate, got, tt.wantMin, tt.wantMax)
}
if got < 1 || got > 5 {
t.Errorf("computeRetries(%v) = %d, 超出 [1,6] 范围", tt.lossRate, got)
}
})
}
}
func TestComputeRetries_NaN_Inf(t *testing.T) {
// 确保不 panic
for _, v := range []float64{math.NaN(), math.Inf(1), math.Inf(-1)} {
got := computeRetries(v, EnvWAN)
if got < 1 || got > 5 {
t.Errorf("computeRetries(%v) = %d, 超出 [1,6] 范围", v, got)
}
}
}
// =============================================================================
// computeICMPRate 边界
// =============================================================================
func TestComputeICMPRate_EdgeCases(t *testing.T) {
tests := []struct {
env NetworkEnv
fdLimit int
desc string
}{
{EnvLAN, 1, "fd=1: 极小"},
{EnvLAN, -1, "fd=负数: 应被忽略"},
{EnvLAN, 0, "fd=0: 未知"},
{EnvLAN, math.MaxInt32, "fd=极大"},
{NetworkEnv(99), 1024, "未知环境类型"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
net := &NetworkProfile{Env: tt.env}
sys := &SystemProfile{FDLimit: tt.fdLimit}
got := computeICMPRate(net, sys)
if got <= 0 || math.IsNaN(got) || math.IsInf(got, 0) {
t.Errorf("computeICMPRate(env=%v, fd=%d) = %v, 无效值", tt.env, tt.fdLimit, got)
}
})
}
}
// =============================================================================
// classifyEnv 精确边界值
// =============================================================================
func TestClassifyEnv_ExactBoundaries(t *testing.T) {
tests := []struct {
median time.Duration
lossRate float64
want NetworkEnv
desc string
}{
// RTT 边界
{4999 * time.Microsecond, 0.0, EnvLAN, "4.999ms → LAN"},
{5 * time.Millisecond, 0.0, EnvWAN, "精确 5ms → WAN"},
{49999 * time.Microsecond, 0.0, EnvWAN, "49.999ms → WAN"},
{50 * time.Millisecond, 0.0, EnvInternet, "精确 50ms → Internet"},
{199999 * time.Microsecond, 0.0, EnvInternet, "199.999ms → Internet"},
{200 * time.Millisecond, 0.0, EnvSlow, "精确 200ms → Slow"},
// 丢包率边界
{1 * time.Millisecond, 0.009, EnvLAN, "丢包 0.9% → LAN"},
{1 * time.Millisecond, 0.01, EnvWAN, "精确 1% → WAN (不满足 < 0.01)"},
{1 * time.Millisecond, 0.011, EnvWAN, "丢包 1.1% → WAN (超过 LAN 阈值)"},
{20 * time.Millisecond, 0.049, EnvWAN, "丢包 4.9% → WAN"},
{20 * time.Millisecond, 0.05, EnvInternet, "精确 5% → Internet (不满足 < 0.05)"},
{20 * time.Millisecond, 0.051, EnvInternet, "丢包 5.1% → Internet"},
{1 * time.Millisecond, 0.099, EnvInternet, "丢包 9.9% → Internet"},
{1 * time.Millisecond, 0.10, EnvInternet, "精确 10% → Internet (< 判断)"},
{1 * time.Millisecond, 0.101, EnvSlow, "丢包 10.1% → Slow"},
// 零值
{0, 0.0, EnvLAN, "零 RTT 零丢包 → LAN"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := classifyEnv(tt.median, tt.lossRate)
if got != tt.want {
t.Errorf("classifyEnv(median=%v, loss=%.4f) = %v, want %v",
tt.median, tt.lossRate, got, tt.want)
}
})
}
}
// =============================================================================
// classifyNetwork 边界
// =============================================================================
func TestClassifyNetwork_EdgeCases(t *testing.T) {
t.Run("单个 RTT 样本", func(t *testing.T) {
p := classifyNetwork([]time.Duration{5 * time.Millisecond}, 0, 1)
if p.Samples != 1 {
t.Errorf("samples = %d, want 1", p.Samples)
}
// stddev 应该是 0
if p.RTTStddev != 0 {
t.Errorf("单样本 stddev = %v, want 0", p.RTTStddev)
}
})
t.Run("所有 RTT 相同", func(t *testing.T) {
rtts := make([]time.Duration, 50)
for i := range rtts {
rtts[i] = 10 * time.Millisecond
}
p := classifyNetwork(rtts, 0, 50)
if p.RTTStddev != 0 {
t.Errorf("全相同 RTT stddev = %v, want 0", p.RTTStddev)
}
if p.RTTMedian != 10*time.Millisecond {
t.Errorf("median = %v, want 10ms", p.RTTMedian)
}
})
t.Run("极大 RTT 值", func(t *testing.T) {
rtts := []time.Duration{time.Hour, time.Hour, time.Hour}
p := classifyNetwork(rtts, 0, 3)
if p.Env != EnvSlow {
t.Errorf("env = %v, want Slow", p.Env)
}
})
t.Run("混合极端值", func(t *testing.T) {
rtts := []time.Duration{time.Microsecond, time.Hour}
p := classifyNetwork(rtts, 0, 2)
// 不 panic 就行
if p.Samples != 2 {
t.Errorf("samples = %d, want 2", p.Samples)
}
})
t.Run("全部失败无响应", func(t *testing.T) {
p := classifyNetwork(nil, 100, 100)
if p.Env != EnvWAN {
t.Errorf("env = %v, want WAN (default)", p.Env)
}
})
t.Run("failures > total (异常输入)", func(t *testing.T) {
rtts := []time.Duration{time.Millisecond}
p := classifyNetwork(rtts, 10, 5) // failures > total
// lossRate = 1 - 1/5 = 0.8, 不应 panic
if p.LossRate < 0 {
t.Errorf("lossRate = %.2f, 不应为负", p.LossRate)
}
})
t.Run("total=0", func(t *testing.T) {
p := classifyNetwork(nil, 0, 0)
// 不 panic
if p.Samples != 0 {
t.Errorf("samples = %d, want 0", p.Samples)
}
})
}
// =============================================================================
// RecommendConcurrency 边界
// =============================================================================
func TestRecommendConcurrency_EdgeCases(t *testing.T) {
tests := []struct {
env NetworkEnv
loss float64
userT int
explicit bool
desc string
}{
{EnvLAN, 0.0, 0, false, "userThreadNum=0"},
{EnvLAN, 0.0, 1, false, "userThreadNum=1"},
{EnvLAN, 0.0, -1, false, "userThreadNum 负数"},
{EnvLAN, 0.0, math.MaxInt32, false, "userThreadNum 极大"},
{EnvLAN, 0.99, 600, false, "99% 丢包"},
{EnvLAN, 1.0, 600, false, "100% 丢包"},
{EnvSlow, 0.0, 1, true, "慢速+显式+1"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
p := &NetworkProfile{Env: tt.env, LossRate: tt.loss, Samples: 10}
target, ceiling := p.RecommendConcurrency(tt.userT, tt.explicit)
// 不 panic,且 target >= 1clamp 保底 10 或 userT
if target < 0 || ceiling < 0 {
t.Errorf("target=%d ceiling=%d, 不应为负", target, ceiling)
}
if tt.explicit && ceiling != tt.userT && tt.userT > 0 {
t.Errorf("显式模式 ceiling=%d, want %d", ceiling, tt.userT)
}
t.Logf("env=%v loss=%.2f userT=%d explicit=%v → target=%d ceiling=%d",
tt.env, tt.loss, tt.userT, tt.explicit, target, ceiling)
})
}
}
// =============================================================================
// ScanMetrics 边界
// =============================================================================
func TestScanMetrics_EdgeCases(t *testing.T) {
t.Run("RTT=0", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(0)
// 不 panic
if m.Total() != 1 {
t.Errorf("Total = %d, want 1", m.Total())
}
})
t.Run("负数 RTT", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(-time.Millisecond)
// 不 panic,负数 RTT 应被忽略
if m.rttSamples.Load() != 0 {
t.Errorf("负数 RTT 不应计入采样: got %d", m.rttSamples.Load())
}
})
t.Run("极大 RTT", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(time.Hour)
if m.RTTFast() != time.Hour {
t.Errorf("首个样本 RTTFast = %v, want 1h", m.RTTFast())
}
})
t.Run("EMA 首个样本初始化", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(10 * time.Millisecond)
if m.rttFastNs.Load() != int64(10*time.Millisecond) {
t.Errorf("首个样本应直接设置 EMA: got %d", m.rttFastNs.Load())
}
})
t.Run("空 Snapshot", func(t *testing.T) {
m := &ScanMetrics{}
snap := m.Snapshot()
if snap.Total() != 0 {
t.Errorf("空 metrics Snapshot.Total = %d, want 0", snap.Total())
}
})
t.Run("RTTRatio 单侧为零", func(t *testing.T) {
m := &ScanMetrics{}
// 手动设置一个但不设另一个——不应该发生,但防御
m.rttFastNs.Store(1000)
m.rttSlowNs.Store(0)
m.rttSamples.Store(30)
ratio := m.RTTRatio()
if ratio != 1.0 {
t.Errorf("slow=0 时 ratio = %.2f, want 1.0", ratio)
}
})
t.Run("大量操作不溢出", func(t *testing.T) {
m := &ScanMetrics{}
for i := 0; i < 100000; i++ {
m.RecordConnect(time.Millisecond)
}
if m.Total() != 100000 {
t.Errorf("Total = %d, want 100000", m.Total())
}
ratio := m.RTTRatio()
if math.IsNaN(ratio) || math.IsInf(ratio, 0) {
t.Errorf("大量样本后 ratio = %v, 不应为 NaN/Inf", ratio)
}
})
}
// =============================================================================
// TuneConfig 边界
// =============================================================================
func TestTuneConfig_EdgeCases(t *testing.T) {
t.Run("RTTMedian=0 RTTStddev=0", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: 0, RTTStddev: 0, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// Timeout: median(0) + 4*stddev(0) = 0 → minTO = 0+200ms → clamp to 1s
if config.Timeout < time.Second {
t.Errorf("零 RTT Timeout = %v, 应该 >= 1s", config.Timeout)
}
})
t.Run("RTTStddev 远大于 RTTMedian", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvInternet, RTTMedian: 10 * time.Millisecond, RTTStddev: 5 * time.Second, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// Timeout = 10ms + 4*5s = 20.01s → clamp to 10s
if config.Timeout != 10*time.Second {
t.Errorf("极大 stddev Timeout = %v, 应该被 clamp 到 10s", config.Timeout)
}
})
t.Run("ThreadNum=0", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 0
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: time.Millisecond, RTTStddev: time.Millisecond, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// ModuleThreadNum = 0/30 = 0 → clamp to 5
if config.ModuleThreadNum < 5 {
t.Errorf("ThreadNum=0 时 ModuleThreadNum = %d, 应该 >= 5", config.ModuleThreadNum)
}
})
t.Run("多次调用 TuneConfig", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: time.Millisecond, RTTStddev: time.Millisecond, LossRate: 0.0, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
first := config.Timeout
// 第二次调用——已经调整过的值不等于默认值,应被视为"显式"
ep.TuneConfig(config, session)
second := config.Timeout
if first != second {
t.Errorf("多次调用 TuneConfig 不应重复调整: %v vs %v", first, second)
}
})
t.Run("fd limit = ThreadNum 精确值", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 1000}, // 1000 * 0.6 = 600
}
ep.TuneConfig(config, session)
// ThreadNum(600) == maxConcurrency(600), 不应触发约束
if config.ThreadNum != 600 {
t.Errorf("fd=1000 时 ThreadNum = %d, 不应被约束", config.ThreadNum)
}
})
t.Run("fd limit 精确低于 ThreadNum", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 999}, // 999 * 0.6 = 599
}
ep.TuneConfig(config, session)
if config.ThreadNum > 599 {
t.Errorf("fd=999 时 ThreadNum = %d, 应该 <= 599", config.ThreadNum)
}
})
}
// =============================================================================
// AdaptivePool 边界
// =============================================================================
func TestAdaptivePool_EdgeCases(t *testing.T) {
t.Run("target=1", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(1, 1, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// initial = max(1/4, 10) = 10 → 但 10 > target(1)... 看实现
// 实际上 initial = min(max(1/4, 10), 1) = 1... 不对
// initial = target/4 = 0, 但 < 10, 所以 initial = 10
// 但 initial > target(1)... initial = min(10, 1) = 1
// 看代码:if initial > target { initial = target }
if pool.Cap() != 1 {
t.Errorf("target=1 时 cap = %d, want 1", pool.Cap())
}
})
t.Run("target=0", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(0, 0, func(interface{}) {}, metrics)
// ants 可能拒绝 size=0
if err != nil {
t.Logf("target=0 正确返回错误: %v", err)
return
}
defer pool.Release()
t.Logf("target=0 cap = %d", pool.Cap())
})
t.Run("ceiling < target", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 50, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// initial = 100/4 = 25, 不超过 ceiling
if pool.Cap() > 50 {
t.Errorf("ceiling=50 但 cap = %d", pool.Cap())
}
})
t.Run("高频 Invoke 不 panic", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(10, 10, func(interface{}) {
time.Sleep(time.Millisecond)
}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(10)
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_ = pool.Invoke(nil)
}()
}
wg.Wait()
pool.Wait()
})
t.Run("assessHealth 零增量", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// 初始化 prevSnapshot 后不产生新数据
pool.prevSnapshot = metrics.Snapshot()
health := pool.assessHealth()
if health != HealthUnknown {
t.Errorf("零增量应返回 HealthUnknown, got %v", health)
}
})
}
// =============================================================================
// pickSamples 边界
// =============================================================================
func TestPickSamples_EdgeCases(t *testing.T) {
t.Run("maxSamples=0", func(t *testing.T) {
s := pickSamples([]string{"a", "b"}, 0)
if len(s) != 0 {
t.Errorf("maxSamples=0 应返回空, got %d", len(s))
}
})
t.Run("maxSamples=1", func(t *testing.T) {
s := pickSamples([]string{"a", "b", "c"}, 1)
if len(s) != 1 {
t.Errorf("maxSamples=1 应返回 1 个, got %d", len(s))
}
})
t.Run("hosts 等于 maxSamples", func(t *testing.T) {
hosts := []string{"a", "b", "c"}
s := pickSamples(hosts, 3)
if len(s) != 3 {
t.Errorf("应返回全部, got %d", len(s))
}
})
}
// =============================================================================
// isTimeoutError / isConnectionRefused 边界
// =============================================================================
func TestIsTimeoutError_EdgeCases(t *testing.T) {
if isTimeoutError(nil) {
t.Error("nil 不应判为 timeout")
}
}
func TestIsConnectionRefused_EdgeCases(t *testing.T) {
if isConnectionRefused(nil) {
t.Error("nil 不应判为 refused")
}
}
// =============================================================================
// NetworkEnv.String 覆盖
// =============================================================================
func TestNetworkEnv_String(t *testing.T) {
for _, env := range []NetworkEnv{EnvLAN, EnvWAN, EnvInternet, EnvSlow} {
s := env.String()
if s == "" {
t.Errorf("NetworkEnv(%d).String() = 空", env)
}
}
// 未知值
s := NetworkEnv(99).String()
if s == "" {
t.Error("未知 NetworkEnv.String() = 空")
}
}
// =============================================================================
// clampInt / clampDuration 边界
// =============================================================================
func TestClampInt(t *testing.T) {
tests := []struct {
v, min, max, want int
}{
{5, 1, 10, 5},
{0, 1, 10, 1},
{15, 1, 10, 10},
{-5, -10, -1, -5},
{5, 5, 5, 5}, // min == max == v
{3, 5, 5, 5}, // v < min == max
{10, 5, 5, 5}, // v > min == max
}
for _, tt := range tests {
got := clampInt(tt.v, tt.min, tt.max)
if got != tt.want {
t.Errorf("clampInt(%d, %d, %d) = %d, want %d", tt.v, tt.min, tt.max, got, tt.want)
}
}
}
func TestClampDuration(t *testing.T) {
got := clampDuration(5*time.Second, time.Second, 10*time.Second)
if got != 5*time.Second {
t.Errorf("got %v, want 5s", got)
}
got = clampDuration(0, time.Second, 10*time.Second)
if got != time.Second {
t.Errorf("got %v, want 1s", got)
}
got = clampDuration(time.Hour, time.Second, 10*time.Second)
if got != 10*time.Second {
t.Errorf("got %v, want 10s", got)
}
}
// =============================================================================
// isExplicit 边界
// =============================================================================
func TestIsExplicit(t *testing.T) {
config := makeDefaultConfig()
// 默认值 → 非显式
if isExplicit(config, "time") {
t.Error("默认 Timeout 不应视为显式")
}
if isExplicit(config, "mt") {
t.Error("默认 ModuleThreadNum 不应视为显式")
}
if isExplicit(config, "retry") {
t.Error("默认 MaxRetries 不应视为显式")
}
if isExplicit(config, "icmp-rate") {
t.Error("默认 ICMPRate 不应视为显式")
}
if isExplicit(config, "num") {
t.Error("默认 PocNum 不应视为显式")
}
// 未知 flag
if isExplicit(config, "nonexistent") {
t.Error("未知 flag 不应视为显式")
}
// ThreadNumExplicit
config.ThreadNumExplicit = true
if !isExplicit(config, "t") {
t.Error("ThreadNumExplicit=true 应视为显式")
}
config = makeDefaultConfig()
config.TimeoutExplicit = true
config.ModuleThreadNumExplicit = true
config.MaxRetriesExplicit = true
config.Network.ICMPRateExplicit = true
config.POC.NumExplicit = true
if !isExplicit(config, "time") || !isExplicit(config, "mt") ||
!isExplicit(config, "retry") || !isExplicit(config, "icmp-rate") ||
!isExplicit(config, "num") {
t.Error("显式标记为 true 时默认值也应视为显式")
}
}
+256
View File
@@ -0,0 +1,256 @@
package core
import (
"fmt"
"math"
"runtime"
"time"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// EnvironmentProfile 综合环境探测结果
type EnvironmentProfile struct {
Net NetworkProfile
System SystemProfile
}
// SystemProfile 系统能力信息
type SystemProfile struct {
FDLimit int // 文件描述符上限(0 表示未知)
NumCPU int
}
// ProbeSystem 探测系统能力(不需要网络目标)
func ProbeSystem() SystemProfile {
p := SystemProfile{
NumCPU: runtime.NumCPU(),
}
p.FDLimit = getFDLimit()
return p
}
// TuneConfig 根据探测结果调整 Config 中的参数
// 只调整用户未显式指定的参数
// 每个参数的推导都有明确的公式和探测依据
func (ep *EnvironmentProfile) TuneConfig(config *common.Config, session *common.ScanSession) {
net := &ep.Net
sys := &ep.System
// ---------- NetworkEnv ----------
config.DetectedNetworkEnv = int(net.Env)
// ---------- ThreadNum / ThreadCeiling ----------
if !isExplicit(config, "t") {
target, ceiling := net.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
old := config.ThreadNum
config.ThreadNum = target
config.ThreadCeiling = ceiling
session.LogDebug(fmt.Sprintf("ThreadNum: %d -> %d, Ceiling: %d (env=%s)", old, target, ceiling, net.Env))
} else {
config.ThreadCeiling = config.ThreadNum
}
// ---------- Timeout ----------
// 公式: median_rtt + 4 * stddev,下限 1s,上限 10s
// 依据: 与 AdaptiveTimeout 相同的统计原理(覆盖 99.9% 的正常连接)
if !isExplicit(config, "time") && net.Samples > 0 {
computed := net.RTTMedian + 4*net.RTTStddev
// 下限:连接建立至少需要 2 个 RTT(SYN + SYN-ACK+ 处理时间
minTO := net.RTTMedian*3 + 200*time.Millisecond
if computed < minTO {
computed = minTO
}
computed = clampDuration(computed, time.Second, 10*time.Second)
old := config.Timeout
config.Timeout = computed
session.LogDebug(fmt.Sprintf("Timeout: %v -> %v (RTT median=%v stddev=%v)",
old, computed, net.RTTMedian, net.RTTStddev))
}
// ---------- ModuleThreadNum ----------
// 公式: ThreadNum / 30,下限 5,上限 50
// 依据: 插件级并发(爆破等)不应超过端口扫描并发的 ~3%
// 单个服务的连接能力远低于 TCP SYN 扫描
// 公网服务通常有限流(MaxStartups 等),并发过高适得其反
if !isExplicit(config, "mt") {
computed := config.ThreadNum / 30
computed = clampInt(computed, 5, 50)
// 高丢包环境进一步压低,避免大量连接被丢弃浪费
if net.LossRate > 0.1 {
computed = computed * 2 / 3
if computed < 5 {
computed = 5
}
}
old := config.ModuleThreadNum
config.ModuleThreadNum = computed
session.LogDebug(fmt.Sprintf("ModuleThreadNum: %d -> %d (threadNum=%d)", old, computed, config.ThreadNum))
}
// ---------- MaxRetries ----------
// 公式: ceil(log(0.01) / log(loss_rate))
// 含义: 重试 N 次后仍然全部丢包的概率 < 1%
// 例: 丢包率 5% → N=2, 丢包率 20% → N=3, 丢包率 50% → N=7
// 下限 1(零丢包也至少试一次),上限 6(避免对不可达目标死磕)
if !isExplicit(config, "retry") {
if net.Samples > 0 {
computed := computeRetries(net.LossRate, net.Env)
old := config.MaxRetries
config.MaxRetries = computed
session.LogDebug(fmt.Sprintf("MaxRetries: %d -> %d (loss_rate=%.2f%%)", old, computed, net.LossRate*100))
} else if config.MaxRetries > 2 {
// 无网络探测数据(-np 跳过存活探测),降低默认重试避免对不可达主机死磕
old := config.MaxRetries
config.MaxRetries = 2
session.LogDebug(fmt.Sprintf("MaxRetries: %d -> %d (no network probe data)", old, config.MaxRetries))
}
}
// ---------- ICMPRate ----------
// 公式: 基于 fd limit 和网络环境
// 内网 fd 充裕: 0.5(高速发包)
// 公网或 fd 紧张: 0.1(默认保守)
// 依据: ICMP 发包速率受两个约束:网络带宽和本机 fd/socket 资源
if !isExplicit(config, "icmp-rate") && net.Samples > 0 {
computed := computeICMPRate(net, sys)
old := config.Network.ICMPRate
config.Network.ICMPRate = computed
session.LogDebug(fmt.Sprintf("ICMPRate: %.2f -> %.2f (env=%s fd=%d)", old, computed, net.Env, sys.FDLimit))
}
// ---------- PocNum ----------
// 公式: 与 ModuleThreadNum 一致
// 依据: POC 检测和凭据爆破的并发约束相同——都是对目标服务发起连接
if !isExplicit(config, "num") {
old := config.POC.Num
config.POC.Num = config.ModuleThreadNum
session.LogDebug(fmt.Sprintf("PocNum: %d -> %d (follows ModuleThreadNum)", old, config.POC.Num))
}
// ---------- DisablePing ----------
// 由 probeWithICMP 自动处理(尝试 → 失败 → 降级),无需在此干预
// 总结日志
if net.Samples > 0 {
session.LogInfo(i18n.Tr("env_tune_summary",
config.Timeout.Milliseconds(),
config.ModuleThreadNum,
config.MaxRetries,
fmt.Sprintf("%.2f", config.Network.ICMPRate),
config.POC.Num))
}
// fd limit 约束:总并发不应超过 fd limit 的 60%(留余量给系统)
if sys.FDLimit > 0 {
maxConcurrency := sys.FDLimit * 6 / 10
if config.ThreadNum > maxConcurrency {
session.LogInfo(i18n.Tr("env_fd_limit", config.ThreadNum, maxConcurrency, sys.FDLimit))
config.ThreadNum = maxConcurrency
}
if config.ThreadCeiling > maxConcurrency {
config.ThreadCeiling = maxConcurrency
}
}
}
// computeRetries 基于丢包率和网络环境计算重试次数
// 内网丢包异常,用更严格的目标概率(0.5%)和更低上限
// 公网/慢速丢包常见,放宽目标概率(2%)和更高上限
func computeRetries(lossRate float64, env NetworkEnv) int {
if lossRate <= 0.001 {
return 1
}
var targetProb float64
var maxRetries int
switch env {
case EnvLAN:
targetProb = 0.005
maxRetries = 4
case EnvWAN:
targetProb = 0.01
maxRetries = 5
default:
targetProb = 0.02
maxRetries = 6
}
if lossRate >= 0.95 {
return maxRetries
}
// P(N次全失败) = lossRate^N < targetProb
n := math.Ceil(math.Log(targetProb) / math.Log(lossRate))
return clampInt(int(n), 1, maxRetries)
}
// computeICMPRate 基于环境计算 ICMP 发包速率
func computeICMPRate(net *NetworkProfile, sys *SystemProfile) float64 {
// 基准:根据 RTT 估算网络可承受的速率
// RTT 越低,网络越快,可以发更快
var base float64
switch net.Env {
case EnvLAN:
base = 0.5
case EnvWAN:
base = 0.3
case EnvInternet:
base = 0.1
default:
base = 0.05
}
// fd 约束:fd limit 低时压低速率
if sys.FDLimit > 0 && sys.FDLimit < 1024 {
base = base * float64(sys.FDLimit) / 1024.0
if base < 0.02 {
base = 0.02
}
}
return base
}
// isExplicit 检查参数是否被用户显式指定。
// 显式标记来自 CLI flag.Visit;值比较保留 SDK/测试里直接构造 Config 的旧行为。
func isExplicit(config *common.Config, flagName string) bool {
switch flagName {
case "t":
return config.ThreadNumExplicit
case "time":
return config.TimeoutExplicit || config.Timeout != 3*time.Second
case "mt":
return config.ModuleThreadNumExplicit || config.ModuleThreadNum != 20
case "retry":
return config.MaxRetriesExplicit || config.MaxRetries != 3
case "icmp-rate":
return config.Network.ICMPRateExplicit || config.Network.ICMPRate != 0.1
case "num":
return config.POC.NumExplicit || config.POC.Num != 20
}
return false
}
func clampInt(v, min, max int) int {
if v < min {
return min
}
if v > max {
return max
}
return v
}
func clampDuration(v, min, max time.Duration) time.Duration {
if v < min {
return min
}
if v > max {
return max
}
return v
}
+374
View File
@@ -0,0 +1,374 @@
package core
import (
"math"
"testing"
"time"
"github.com/shadow1ng/fscan/common"
)
// =============================================================================
// 单元测试:computeRetries — 丢包率到重试次数的推导
// =============================================================================
func TestComputeRetries(t *testing.T) {
tests := []struct {
lossRate float64
wantMin int
wantMax int
desc string
}{
{0.0, 1, 1, "零丢包: 只需 1 次"},
{0.001, 1, 1, "极低丢包: 1 次"},
{0.05, 2, 2, "5% 丢包: 0.05^2=0.0025 < 0.01"},
{0.10, 2, 3, "10% 丢包: ceil(log(0.01)/log(0.1))=2, 但边界取 ceil 可能是 3"},
{0.20, 3, 3, "20% 丢包: 0.2^3=0.008 < 0.01"},
{0.30, 3, 4, "30% 丢包"},
{0.50, 5, 5, "50% 丢包: ceil(log(0.01)/log(0.5))=7 但上限 5"},
{0.80, 5, 5, "80% 丢包: 需要很多次但上限 5"},
{0.95, 5, 5, "95% 丢包: 触顶"},
{1.0, 5, 5, "100% 丢包: 触顶"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := computeRetries(tt.lossRate, EnvWAN)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeRetries(%.2f) = %d, want [%d, %d]",
tt.lossRate, got, tt.wantMin, tt.wantMax)
}
// 验证数学正确性:lossRate^got < 0.01
// 跳过:零丢包、极高丢包(触顶上限 6 时数学不满足,属于设计取舍)
if tt.lossRate > 0.001 && tt.lossRate < 0.45 {
prob := math.Pow(tt.lossRate, float64(got))
if prob >= 0.01 {
t.Errorf("lossRate=%.2f retries=%d: P(全失败)=%.4f >= 0.01, 重试不够",
tt.lossRate, got, prob)
}
}
})
}
}
// =============================================================================
// 单元测试:computeICMPRate
// =============================================================================
func TestComputeICMPRate(t *testing.T) {
tests := []struct {
env NetworkEnv
fdLimit int
wantMin float64
wantMax float64
desc string
}{
{EnvLAN, 65536, 0.4, 0.6, "内网高 fd: 高速"},
{EnvWAN, 65536, 0.2, 0.4, "局域网高 fd: 中速"},
{EnvInternet, 65536, 0.05, 0.15, "公网: 保守"},
{EnvSlow, 65536, 0.03, 0.08, "慢速: 极保守"},
{EnvLAN, 256, 0.01, 0.2, "内网低 fd: 受限"},
{EnvLAN, 0, 0.4, 0.6, "fd 未知: 按环境"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
net := &NetworkProfile{Env: tt.env}
sys := &SystemProfile{FDLimit: tt.fdLimit}
got := computeICMPRate(net, sys)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeICMPRate(env=%v, fd=%d) = %.3f, want [%.3f, %.3f]",
tt.env, tt.fdLimit, got, tt.wantMin, tt.wantMax)
}
})
}
}
// =============================================================================
// 集成测试:TuneConfig — 完整参数调整流程
// =============================================================================
func TestTuneConfig_LAN(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMin: 500 * time.Microsecond,
RTTMedian: 1 * time.Millisecond,
RTTP95: 3 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// Timeout: median(1ms) + 4*stddev(0.5ms) = 3ms → clamp to 1s 下限
if config.Timeout < time.Second || config.Timeout > 2*time.Second {
t.Errorf("LAN Timeout = %v, 内网应该在 1-2s", config.Timeout)
}
// MaxRetries: 零丢包 → 1
if config.MaxRetries != 1 {
t.Errorf("LAN MaxRetries = %d, 零丢包应该是 1", config.MaxRetries)
}
// ICMPRate: 内网应该比默认 0.1 高
if config.Network.ICMPRate <= 0.1 {
t.Errorf("LAN ICMPRate = %.2f, 应该 > 0.1", config.Network.ICMPRate)
}
// ModuleThreadNum: 基于 ThreadNum/30
if config.ModuleThreadNum < 5 {
t.Errorf("LAN ModuleThreadNum = %d, 应该 >= 5", config.ModuleThreadNum)
}
t.Logf("LAN 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
func TestTuneConfig_Internet(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvInternet,
RTTMin: 50 * time.Millisecond,
RTTMedian: 100 * time.Millisecond,
RTTP95: 250 * time.Millisecond,
RTTStddev: 40 * time.Millisecond,
LossRate: 0.08,
Samples: 25,
},
System: SystemProfile{FDLimit: 1024, NumCPU: 4},
}
ep.TuneConfig(config, session)
// Timeout: median(100ms) + 4*stddev(40ms) = 260ms → 但 minTO = 3*100+200 = 500ms
if config.Timeout < 500*time.Millisecond || config.Timeout > 5*time.Second {
t.Errorf("Internet Timeout = %v, 公网应该在 500ms-5s", config.Timeout)
}
// MaxRetries: 8% 丢包 → ceil(log(0.01)/log(0.08)) ≈ 2
if config.MaxRetries < 2 || config.MaxRetries > 3 {
t.Errorf("Internet MaxRetries = %d, 8%%丢包应该是 2-3", config.MaxRetries)
}
// ICMPRate: 公网应该偏低
if config.Network.ICMPRate > 0.2 {
t.Errorf("Internet ICMPRate = %.2f, 应该 <= 0.2", config.Network.ICMPRate)
}
t.Logf("Internet 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
func TestTuneConfig_SlowLossy(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvSlow,
RTTMin: 200 * time.Millisecond,
RTTMedian: 500 * time.Millisecond,
RTTP95: 2 * time.Second,
RTTStddev: 300 * time.Millisecond,
LossRate: 0.25,
Samples: 15,
},
System: SystemProfile{FDLimit: 512, NumCPU: 2},
}
ep.TuneConfig(config, session)
// Timeout: median(500ms) + 4*stddev(300ms) = 1700ms, minTO = 500*3+200 = 1700ms
if config.Timeout < time.Second {
t.Errorf("Slow Timeout = %v, 慢速网络应该 >= 1s", config.Timeout)
}
// MaxRetries: 25% 丢包 → ceil(log(0.01)/log(0.25)) ≈ 4
if config.MaxRetries < 3 || config.MaxRetries > 5 {
t.Errorf("Slow MaxRetries = %d, 25%%丢包应该是 3-5", config.MaxRetries)
}
// ICMPRate: 慢速 + 低 fd → 应该很低
if config.Network.ICMPRate > 0.1 {
t.Errorf("Slow ICMPRate = %.2f, 应该 <= 0.1", config.Network.ICMPRate)
}
t.Logf("Slow 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
// =============================================================================
// 集成测试:用户显式指定时不覆盖
// =============================================================================
func TestTuneConfig_ExplicitOverride(t *testing.T) {
config := makeDefaultConfig()
config.Timeout = 5 * time.Second // 用户设了 -time 5
config.ModuleThreadNum = 50 // 用户设了 -mt 50
config.MaxRetries = 1 // 用户设了 -retry 1
config.Network.ICMPRate = 0.8 // 用户设了 -icmp-rate 0.8
config.POC.Num = 100 // 用户设了 -num 100
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// 所有非默认值都不应被覆盖
if config.Timeout != 5*time.Second {
t.Errorf("用户 Timeout 被覆盖: %v", config.Timeout)
}
if config.ModuleThreadNum != 50 {
t.Errorf("用户 ModuleThreadNum 被覆盖: %d", config.ModuleThreadNum)
}
if config.MaxRetries != 1 {
t.Errorf("用户 MaxRetries 被覆盖: %d", config.MaxRetries)
}
if config.Network.ICMPRate != 0.8 {
t.Errorf("用户 ICMPRate 被覆盖: %.2f", config.Network.ICMPRate)
}
if config.POC.Num != 100 {
t.Errorf("用户 PocNum 被覆盖: %d", config.POC.Num)
}
}
func TestTuneConfig_ExplicitDefaultValues(t *testing.T) {
config := makeDefaultConfig()
config.TimeoutExplicit = true
config.ModuleThreadNumExplicit = true
config.MaxRetriesExplicit = true
config.Network.ICMPRateExplicit = true
config.POC.NumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
if config.Timeout != 3*time.Second {
t.Errorf("显式默认 Timeout 被覆盖: %v", config.Timeout)
}
if config.ModuleThreadNum != 20 {
t.Errorf("显式默认 ModuleThreadNum 被覆盖: %d", config.ModuleThreadNum)
}
if config.MaxRetries != 3 {
t.Errorf("显式默认 MaxRetries 被覆盖: %d", config.MaxRetries)
}
if config.Network.ICMPRate != 0.1 {
t.Errorf("显式默认 ICMPRate 被覆盖: %.2f", config.Network.ICMPRate)
}
if config.POC.Num != 20 {
t.Errorf("显式默认 PocNum 被覆盖: %d", config.POC.Num)
}
}
// =============================================================================
// 集成测试:fd limit 约束
// =============================================================================
func TestTuneConfig_FDLimitConstraint(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 256, NumCPU: 4},
}
ep.TuneConfig(config, session)
// 600 线程 > 256 * 0.6 = 153 → 应该被约束
maxExpected := 256 * 6 / 10
if config.ThreadNum > maxExpected {
t.Errorf("ThreadNum = %d, 应该 <= %d (fd_limit=256)", config.ThreadNum, maxExpected)
}
t.Logf("fd limit 约束: ThreadNum=%d (max=%d)", config.ThreadNum, maxExpected)
}
// =============================================================================
// 集成测试:零样本时不调整
// =============================================================================
func TestTuneConfig_NoSamples(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
origTimeout := config.Timeout
origRetry := config.MaxRetries
origICMP := config.Network.ICMPRate
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
if config.Timeout != origTimeout {
t.Errorf("零样本不应改 Timeout: %v -> %v", origTimeout, config.Timeout)
}
// 零样本时,默认重试降到 2(避免对不可达主机死磕)
if origRetry > 2 && config.MaxRetries != 2 {
t.Errorf("零样本应降 MaxRetries 至 2: %d -> %d", origRetry, config.MaxRetries)
}
if config.Network.ICMPRate != origICMP {
t.Errorf("零样本不应改 ICMPRate: %.2f -> %.2f", origICMP, config.Network.ICMPRate)
}
}
// =============================================================================
// 辅助
// =============================================================================
func makeDefaultConfig() *common.Config {
return &common.Config{
Timeout: 3 * time.Second,
ThreadNum: 600,
ModuleThreadNum: 20,
MaxRetries: 3,
Network: common.NetworkConfig{ICMPRate: 0.1},
POC: common.POCConfig{Num: 20},
Output: common.OutputConfig{LogLevel: "base,info,success"},
}
}
func makeTestSession(config *common.Config) *common.ScanSession {
return common.NewScanSession(config, common.NewState(), &common.FlagVars{})
}
+13
View File
@@ -0,0 +1,13 @@
//go:build !windows
package core
import "syscall"
func getFDLimit() int {
var lim syscall.Rlimit
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim); err != nil {
return 0
}
return int(lim.Cur)
}
+8
View File
@@ -0,0 +1,8 @@
//go:build windows
package core
// Windows 没有 RLIMIT_NOFILE,句柄上限由系统管理
func getFDLimit() int {
return 0
}
+22
View File
@@ -0,0 +1,22 @@
package core
import (
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/parsers"
)
const maxHostBatchSize = 65536
func targetHostBatchSize(config *common.Config) int {
size := parsers.DefaultHostBatchSize
if config != nil && config.ThreadNum > 0 {
threadWindow := config.ThreadNum * 8
if threadWindow > size {
size = threadWindow
}
}
if size > maxHostBatchSize {
return maxHostBatchSize
}
return size
}
+28
View File
@@ -0,0 +1,28 @@
package core
import (
"strings"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/parsers"
)
func loadHostExcludes(params *common.FlagVars) ([]string, error) {
if params == nil {
return nil, nil
}
excludes := make([]string, 0, 1)
if strings.TrimSpace(params.ExcludeHosts) != "" {
excludes = append(excludes, params.ExcludeHosts)
}
if strings.TrimSpace(params.ExcludeHostsFile) == "" {
return excludes, nil
}
lines, err := parsers.ReadLinesFromFile(params.ExcludeHostsFile)
if err != nil {
return nil, err
}
return append(excludes, lines...), nil
}
+29
View File
@@ -0,0 +1,29 @@
package core
import (
"os"
"reflect"
"testing"
"github.com/shadow1ng/fscan/common"
)
func TestLoadHostExcludesIncludesExcludeFile(t *testing.T) {
path := t.TempDir() + "/exclude.txt"
if err := os.WriteFile(path, []byte("192.168.1.2\n# comment\n192.168.1.3\n"), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
got, err := loadHostExcludes(&common.FlagVars{
ExcludeHosts: "192.168.1.1",
ExcludeHostsFile: path,
})
if err != nil {
t.Fatalf("loadHostExcludes error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.2", "192.168.1.3"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("loadHostExcludes = %#v, want %#v", got, want)
}
}
+76 -53
View File
@@ -2,11 +2,13 @@ package core
import (
"bytes"
"context"
"errors"
"fmt"
"net"
"os/exec"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
@@ -39,7 +41,9 @@ var pingErrorKeywords = []string{
// CheckLive 检测主机存活状态
// 支持 ICMP/Ping 探测,并在响应率过低时自动启用 TCP 补充探测
func CheckLive(hostslist []string, Ping bool, config *common.Config, state *common.State) []string {
func CheckLive(ctx context.Context, hostslist []string, Ping bool, session *common.ScanSession) []string {
config := session.Config
state := session.State
// 创建局部WaitGroup
var livewg sync.WaitGroup
@@ -52,7 +56,7 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
chanHosts := make(chan string, len(hostslist))
// 处理存活主机
go handleAliveHosts(chanHosts, hostslist, Ping, &aliveHosts, &aliveHostsMu, existHosts, config, &livewg)
go handleAliveHosts(chanHosts, hostslist, Ping, &aliveHosts, &aliveHostsMu, existHosts, config, session, &livewg)
// 根据Ping参数选择检测方式
if Ping {
@@ -68,7 +72,7 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
// TCP 补充探测:当 ICMP/Ping 响应率过低时自动启用
// 这对防火墙过滤 ICMP 的环境特别有用
aliveHosts = tcpSupplementaryProbe(hostslist, aliveHosts, config)
aliveHosts = tcpSupplementaryProbe(ctx, hostslist, aliveHosts, session)
// 输出存活统计信息
printAliveStats(aliveHosts, hostslist)
@@ -78,7 +82,11 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
// tcpSupplementaryProbe TCP 补充探测
// 当 ICMP 响应率过低时(<10%),对未响应主机进行 TCP 探测
func tcpSupplementaryProbe(allHosts []string, aliveHosts []string, config *common.Config) []string {
func tcpSupplementaryProbe(ctx context.Context, allHosts []string, aliveHosts []string, session *common.ScanSession) []string {
if session.Config.DisableTcpProbe || session.Config.Mode == "icmp" {
return aliveHosts
}
totalHosts := len(allHosts)
if totalHosts == 0 {
return aliveHosts
@@ -99,15 +107,15 @@ func tcpSupplementaryProbe(allHosts []string, aliveHosts []string, config *commo
}
// 提示用户正在进行 TCP 补充探测
common.LogInfo(i18n.Tr("tcp_probe_low_icmp_rate", fmt.Sprintf("%.1f%%", responseRate*100), len(unrespondedHosts)))
session.LogInfo(i18n.Tr("tcp_probe_low_icmp_rate", fmt.Sprintf("%.1f%%", responseRate*100), len(unrespondedHosts)))
// 执行 TCP 补充探测
tcpAliveHosts := runTcpProbeForHosts(unrespondedHosts, config)
tcpAliveHosts := runTcpProbeForHosts(ctx, unrespondedHosts, session)
// 合并结果
if len(tcpAliveHosts) > 0 {
aliveHosts = append(aliveHosts, tcpAliveHosts...)
common.LogInfo(i18n.Tr("tcp_probe_found", len(tcpAliveHosts)))
session.LogInfo(i18n.Tr("tcp_probe_found", len(tcpAliveHosts)))
}
return aliveHosts
@@ -123,7 +131,7 @@ func IsContain(items []string, item string) bool {
return false
}
func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, aliveHosts *[]string, aliveHostsMu *sync.Mutex, existHosts map[string]struct{}, config *common.Config, livewg *sync.WaitGroup) {
func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, aliveHosts *[]string, aliveHostsMu *sync.Mutex, existHosts map[string]struct{}, config *common.Config, session *common.ScanSession, livewg *sync.WaitGroup) {
for ip := range chanHosts {
if _, ok := existHosts[ip]; !ok && IsContain(hostslist, ip) {
existHosts[ip] = struct{}{}
@@ -148,12 +156,9 @@ func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, al
"protocol": protocol,
},
}
_ = common.SaveResult(result)
_ = session.SaveResult(result)
// 保留原有的控制台输出
if !config.Output.Silent {
common.LogInfo(i18n.Tr("host_alive", ip, protocol))
}
session.LogInfo(i18n.Tr("host_alive", ip, protocol))
}
livewg.Done()
}
@@ -178,9 +183,6 @@ func probeWithICMP(hostslist []string, chanHosts chan string, aliveHosts *[]stri
return
}
common.LogError(i18n.Tr("icmp_listen_failed", err))
common.LogInfo(i18n.GetText("trying_no_listen_icmp"))
// 尝试无监听ICMP探测
conn2, err := net.DialTimeout("ip4:icmp", "127.0.0.1", 3*time.Second)
if err == nil {
@@ -189,8 +191,6 @@ func probeWithICMP(hostslist []string, chanHosts chan string, aliveHosts *[]stri
return
}
common.LogError(i18n.Tr("icmp_connect_failed", err))
common.LogError(i18n.GetText("insufficient_privileges"))
common.LogInfo(i18n.GetText("switching_to_ping"))
// 降级使用ping探测
@@ -282,13 +282,13 @@ func waitAdaptive(hostslist []string, aliveHosts *[]string, aliveHostsMu *sync.M
// 条件1:所有主机都已响应,立即结束
if aliveCount >= totalHosts {
common.LogDebug(fmt.Sprintf("[ICMP] 全部响应,耗时 %v", elapsed.Round(time.Millisecond)))
common.LogDebug(i18n.Tr("icmp_debug_all_responded", elapsed.Round(time.Millisecond)))
break
}
// 条件2:超过最大等待时间,兜底结束
if elapsed >= maxWait {
common.LogDebug(fmt.Sprintf("[ICMP] 达到最大等待时间 %v,存活 %d/%d", maxWait, aliveCount, totalHosts))
common.LogDebug(i18n.Tr("icmp_debug_max_wait", maxWait, aliveCount, totalHosts))
break
}
@@ -301,8 +301,7 @@ func waitAdaptive(hostslist []string, aliveHosts *[]string, aliveHostsMu *sync.M
lastAliveCount = aliveCount
} else if time.Since(lastChangeTime) >= icmpStableThreshold {
// 连续 500ms 没有新响应,认为响应已稳定,提前结束
common.LogDebug(fmt.Sprintf("[ICMP] 响应稳定,提前结束,耗时 %v,存活 %d/%d",
elapsed.Round(time.Millisecond), aliveCount, totalHosts))
common.LogDebug(i18n.Tr("icmp_debug_stable_done", elapsed.Round(time.Millisecond), aliveCount, totalHosts))
break
}
} else {
@@ -321,8 +320,8 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
var endflag atomic.Bool
var listenerWg sync.WaitGroup
// 创建布隆过滤器用于去重(自动根据主机数量调整大小)
bloomFilter := NewBloomFilter(len(hostslist), 0.01)
// 去重集合:过滤重复的ICMP响应
seen := make(map[string]struct{}, len(hostslist))
// 启动监听协程
listenerWg.Add(1)
@@ -358,11 +357,10 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
if sourceIP != nil && !endflag.Load() {
ipStr := sourceIP.String()
// 使用布隆过滤器去重,过滤重复的ICMP响应和杂包
if bloomFilter.Contains(ipStr) {
if _, dup := seen[ipStr]; dup {
continue
}
bloomFilter.Add(ipStr)
seen[ipStr] = struct{}{}
livewg.Add(1)
select {
@@ -376,13 +374,22 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
}
}()
// 发送ICMP请求(应用令牌桶限速)
limiter := state.GetICMPLimiter(config.Network.ICMPRate)
// 发送ICMP请求(批量预构建 + 令牌桶限速)
// 预构建所有 ICMP 包和目标地址,减少发送循环中的开销
type icmpPacket struct {
data []byte
dst net.Addr
}
packets := make([]icmpPacket, 0, len(hostslist))
for _, host := range hostslist {
limiter.Wait(1) // 等待令牌,控制发包速率
dst, _ := net.ResolveIPAddr("ip", host)
IcmpByte := makemsg(host)
_, _ = conn.WriteTo(IcmpByte, dst)
dst, _ := common.DNSCache.ResolveIP(host)
packets = append(packets, icmpPacket{data: makemsg(host), dst: dst})
}
limiter := state.GetICMPLimiter(config.Network.ICMPRate)
for i := range packets {
limiter.Wait(1)
_, _ = conn.WriteTo(packets[i].data, packets[i].dst)
}
// 自适应等待响应
@@ -470,8 +477,12 @@ func icmpalive(host string) bool {
// RunPing 使用系统Ping命令并发探测主机存活
func RunPing(hostslist []string, chanHosts chan string, livewg *sync.WaitGroup) {
var wg sync.WaitGroup
// 限制并发数为50
limiter := make(chan struct{}, 50)
// 并发数根据主机数动态调整,上限 200
concurrency := len(hostslist)
if concurrency > 200 {
concurrency = 200
}
limiter := make(chan struct{}, concurrency)
// 并发探测
for _, host := range hostslist {
@@ -674,20 +685,34 @@ func ArrayCountValueTop(arrInit []string, length int, flag bool) (arrTop []strin
var tcpProbeCommonPorts = []int{80, 443, 22, 445}
// tcpProbeTimeout TCP 探测超时时间(较短,只做存活判断)
const tcpProbeTimeout = 2 * time.Second
const tcpProbeTimeout = 1 * time.Second
// tcpProbeThreshold TCP 补充探测触发阈值
// 当 ICMP 响应率低于此值时,自动启用 TCP 补充探测
const tcpProbeThreshold = 0.1 // 10%
// tcpProbeAlive 使用 TCP 探测主机是否存活
// 尝试连接常用端口,任一端口响应即认为存活
func tcpProbeAlive(host string) bool {
// tcpProbeAlive 使用 TCP 并行探测主机是否存活
// 同时连接所有常用端口,任一响应即返回
func tcpProbeAlive(ctx context.Context, session *common.ScanSession, host string) bool {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
result := make(chan bool, len(tcpProbeCommonPorts))
for _, port := range tcpProbeCommonPorts {
addr := fmt.Sprintf("%s:%d", host, port)
conn, err := common.WrapperTcpWithTimeout("tcp", addr, tcpProbeTimeout)
if err == nil {
_ = conn.Close()
go func(p int) {
addr := net.JoinHostPort(host, strconv.Itoa(p))
conn, err := session.DialTCP(ctx, "tcp", addr, tcpProbeTimeout)
if err == nil {
_ = conn.Close()
result <- true
return
}
result <- false
}(port)
}
for range tcpProbeCommonPorts {
if <-result {
return true
}
}
@@ -696,7 +721,7 @@ func tcpProbeAlive(host string) bool {
// runTcpProbeForHosts 对指定主机列表进行 TCP 补充探测
// 返回存活的主机列表
func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
func runTcpProbeForHosts(ctx context.Context, hosts []string, session *common.ScanSession) []string {
if len(hosts) == 0 {
return nil
}
@@ -705,10 +730,10 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
var mu sync.Mutex
aliveHosts := make([]string, 0)
// 并发控制,避免资源耗尽
concurrency := 50
if len(hosts) < concurrency {
concurrency = len(hosts)
// 并发控制,根据主机数动态调整,上限 200
concurrency := len(hosts)
if concurrency > 200 {
concurrency = 200
}
limiter := make(chan struct{}, concurrency)
@@ -722,7 +747,7 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
wg.Done()
}()
if tcpProbeAlive(h) {
if tcpProbeAlive(ctx, session, h) {
mu.Lock()
aliveHosts = append(aliveHosts, h)
mu.Unlock()
@@ -737,11 +762,9 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
"protocol": "TCP",
},
}
_ = common.SaveResult(result)
_ = session.SaveResult(result)
if !config.Output.Silent {
common.LogInfo(i18n.Tr("host_alive", h, "TCP"))
}
session.LogInfo(i18n.Tr("host_alive", h, "TCP"))
}
}(host)
}
+545
View File
@@ -0,0 +1,545 @@
package core
import (
"sync"
"sync/atomic"
"testing"
"time"
)
// =============================================================================
// 集成测试 1:探测 → 参数调整 → 线程池创建 完整链路
// 验证从 NetworkProfile 到 TuneConfig 到 AdaptivePool 的端到端数据流
// =============================================================================
func TestIntegration_ProbeToPool_LAN(t *testing.T) {
// 模拟内网探测结果
profile := classifyNetwork(
makeDurations([]int{1, 1, 2, 2, 2, 3, 3, 3, 4, 5}), // ms
0, 10,
)
if profile.Env != EnvLAN {
t.Fatalf("探测环境 = %v, want LAN", profile.Env)
}
// 构建 Config + TuneConfig
config := makeDefaultConfig()
session := makeTestSession(config)
sys := ProbeSystem()
ep := &EnvironmentProfile{Net: *profile, System: sys}
ep.TuneConfig(config, session)
// 验证参数被合理调整
if config.Timeout > 3*time.Second {
t.Errorf("内网 Timeout = %v, 不应 > 3s", config.Timeout)
}
if config.MaxRetries != 1 {
t.Errorf("内网零丢包 MaxRetries = %d, want 1", config.MaxRetries)
}
// 用调整后的参数创建线程池
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if pool.Cap() <= 0 {
t.Errorf("池容量 = %d, 应该 > 0", pool.Cap())
}
t.Logf("内网完整链路: Timeout=%v MT=%d Retry=%d ICMP=%.2f target=%d ceiling=%d poolCap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries,
config.Network.ICMPRate, target, ceiling, pool.Cap())
}
func TestIntegration_ProbeToPool_Internet(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 130, 140, 150}),
0, 10,
)
if profile.Env != EnvInternet {
t.Fatalf("探测环境 = %v, want Internet", profile.Env)
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 4096, NumCPU: 4}}
ep.TuneConfig(config, session)
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 公网并发应该明显低于默认 600
if target >= 600 {
t.Errorf("公网 target = %d, 应该 < 600", target)
}
t.Logf("公网完整链路: Timeout=%v MT=%d Retry=%d target=%d ceiling=%d poolCap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, target, ceiling, pool.Cap())
}
// =============================================================================
// 集成测试 2AdaptivePool + ScanMetrics 联动
// 验证:任务执行 → metrics 记录 → 池读取 metrics → 做出调整决策
// =============================================================================
func TestIntegration_PoolMetrics_HealthyTraffic(t *testing.T) {
metrics := &ScanMetrics{}
var taskCount atomic.Int64
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {
taskCount.Add(1)
}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 注入健康 metrics
for i := 0; i < 200; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 运行任务
var wg sync.WaitGroup
for i := 0; i < 200; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_ = pool.Invoke(nil)
}()
}
wg.Wait()
pool.Wait()
// 触发调整
pool.lastCheck.Store(0)
pool.adjust()
if pool.Cap() < 90 {
t.Errorf("健康流量池容量不应大幅下降: cap = %d", pool.Cap())
}
t.Logf("健康流量: tasks=%d connects=%d cap=%d",
taskCount.Load(), metrics.Snapshot().Connects, pool.Cap())
}
func TestIntegration_PoolMetrics_ExhaustedTraffic(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 直接向 metrics 注入大量资源耗尽事件(模拟扫描过程中的 fd 不足)
for i := 0; i < 200; i++ {
metrics.RecordExhausted()
}
// 手动触发调整(清除时间守卫)
pool.lastCheck.Store(0)
pool.adjust()
// 资源耗尽率 100% → 应该降速
if pool.Cap() >= 100 {
t.Errorf("资源耗尽后池应该降速: cap = %d", pool.Cap())
}
t.Logf("资源耗尽: exhausted=%d cap=%d", metrics.Snapshot().Exhausted, pool.Cap())
}
// =============================================================================
// 集成测试 3:慢启动 → 稳态 AIMD 过渡
// 验证慢启动阶段的翻倍行为和过渡到稳态的时机
// =============================================================================
func TestIntegration_SlowStartToSteady(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {
metrics.RecordConnect(time.Millisecond)
}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if !pool.inSlowStart {
t.Fatal("初始应该在慢启动状态")
}
initialCap := pool.Cap()
t.Logf("慢启动初始: cap=%d", initialCap)
// 喂入足够的健康 metrics
for i := 0; i < 100; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 模拟多次调整周期
caps := []int{initialCap}
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0) // 强制触发检查
pool.adjust()
caps = append(caps, pool.Cap())
}
// 验证:容量应该逐步增长
growing := false
for i := 1; i < len(caps); i++ {
if caps[i] > caps[i-1] {
growing = true
break
}
}
if !growing {
t.Errorf("慢启动期间容量没有增长: %v", caps)
}
// 最终应该退出慢启动
finalCap := pool.Cap()
if finalCap < initialCap {
t.Errorf("最终容量 %d < 初始 %d, 不合理", finalCap, initialCap)
}
t.Logf("慢启动过渡: %v, inSlowStart=%v", caps, pool.inSlowStart)
}
// =============================================================================
// 集成测试 4:拥塞 → 降速 → 恢复 完整周期
// =============================================================================
func TestIntegration_CongestionRecovery(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 200, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 直接到稳态,满容量
pool.inSlowStart = false
pool.tune(200)
// === 阶段 1: 正常运行 ===
for i := 0; i < 100; i++ {
metrics.RecordConnect(time.Millisecond)
}
pool.lastCheck.Store(0)
pool.adjust()
normalCap := pool.Cap()
t.Logf("正常阶段: cap=%d", normalCap)
// === 阶段 2: 突发拥塞(大量资源耗尽)===
for i := 0; i < 200; i++ {
metrics.RecordExhausted()
}
pool.lastCheck.Store(0)
pool.adjust()
congestedCap := pool.Cap()
if congestedCap >= normalCap {
t.Errorf("拥塞后应降速: normal=%d congested=%d", normalCap, congestedCap)
}
t.Logf("拥塞阶段: cap=%d (降幅 %d%%)", congestedCap, (normalCap-congestedCap)*100/normalCap)
// === 阶段 3: 恢复(大量成功连接)===
for i := 0; i < 500; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 多次调整模拟恢复过程
for i := 0; i < 20; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
recoveredCap := pool.Cap()
if recoveredCap <= congestedCap {
t.Errorf("恢复后应提速: congested=%d recovered=%d", congestedCap, recoveredCap)
}
// 恢复后不应超过 ceiling
if recoveredCap > 200 {
t.Errorf("恢复后不应超过 ceiling: cap=%d ceiling=200", recoveredCap)
}
t.Logf("恢复阶段: cap=%d", recoveredCap)
}
// =============================================================================
// 集成测试 5:RTT 趋势检测 → 池调整
// 验证 ScanMetrics 的 RTT EMA 趋势信号能正确传导到池的健康判断
// =============================================================================
func TestIntegration_RTTTrend_DrivesPoolAdjustment(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 建立基线:100 个 5ms RTT
for i := 0; i < 200; i++ {
metrics.RecordConnect(5 * time.Millisecond)
}
pool.lastCheck.Store(0)
pool.adjust()
baselineCap := pool.Cap()
// RTT 突增到 100ms20 倍)
for i := 0; i < 100; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
ratio := metrics.RTTRatio()
if ratio <= 1.0 {
t.Logf("RTT ratio = %.2f, EMA 可能还没追上(正常)", ratio)
}
// 多次调整看池是否响应
for i := 0; i < 5; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
afterRTTSpike := pool.Cap()
t.Logf("RTT 趋势: baseline_cap=%d after_spike=%d rtt_ratio=%.2f",
baselineCap, afterRTTSpike, ratio)
// 如果 ratio 足够高,池应该降速
if ratio > 2.0 && afterRTTSpike >= baselineCap {
t.Errorf("RTT ratio=%.2f 但池没有降速: %d -> %d", ratio, baselineCap, afterRTTSpike)
}
}
// =============================================================================
// 集成测试 6:不同网络环境下的参数一致性
// 验证同一组目标在不同环境下参数调整的合理递进关系
// =============================================================================
func TestIntegration_ParameterProgression(t *testing.T) {
environments := []struct {
name string
rtts []int // ms
loss int // failures out of 10
wantEnv NetworkEnv
}{
{"内网", []int{1, 1, 2, 2, 3, 3, 4, 4, 5, 5}, 0, EnvLAN},
{"局域网", []int{10, 15, 20, 25, 30, 35, 40, 45, 48, 49}, 0, EnvWAN},
{"公网", []int{60, 70, 80, 90, 100, 120, 140, 160, 180, 195}, 0, EnvInternet},
{"慢速", []int{200, 300, 400, 500, 600, 700, 800, 900, 1000, 1500}, 0, EnvSlow},
}
type params struct {
timeout time.Duration
mt int
retry int
icmpRate float64
}
var results []params
for _, env := range environments {
profile := classifyNetwork(makeDurations(env.rtts), env.loss, 10)
if profile.Env != env.wantEnv {
t.Errorf("%s: env = %v, want %v", env.name, profile.Env, env.wantEnv)
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
results = append(results, params{
timeout: config.Timeout,
mt: config.ModuleThreadNum,
retry: config.MaxRetries,
icmpRate: config.Network.ICMPRate,
})
t.Logf("%s: Timeout=%v MT=%d Retry=%d ICMP=%.2f",
env.name, config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate)
}
// 验证递进关系:从内网到慢速,Timeout 应递增
for i := 1; i < len(results); i++ {
if results[i].timeout < results[i-1].timeout {
t.Errorf("Timeout 不递增: %v (env[%d]) < %v (env[%d])",
results[i].timeout, i, results[i-1].timeout, i-1)
}
}
// ICMPRate 应递减(内网最高,慢速最低)
for i := 1; i < len(results); i++ {
if results[i].icmpRate > results[i-1].icmpRate {
t.Errorf("ICMPRate 不递减: %.2f (env[%d]) > %.2f (env[%d])",
results[i].icmpRate, i, results[i-1].icmpRate, i-1)
}
}
}
// =============================================================================
// 集成测试 7:用户显式 -t + 网络探测 完整流程
// 验证用户指定值作为 ceiling 但探测仍然影响其他参数
// =============================================================================
func TestIntegration_ExplicitThreadNum_WithProbe(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{100, 120, 140, 160, 180, 200, 220, 240, 260, 300}),
2, 12, // 部分丢包
)
config := makeDefaultConfig()
config.ThreadNum = 200
config.ThreadNumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 4096, NumCPU: 4},
}
ep.TuneConfig(config, session)
// ThreadNum 不应被修改(fd limit 允许范围内)
// 但 Timeout、ModuleThreadNum 等应根据探测调整
if config.Timeout == 3*time.Second {
t.Error("即使 -t 显式,Timeout 仍应根据探测调整")
}
// 创建池
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
if ceiling != 200 {
t.Errorf("显式 -t 200 的 ceiling = %d, want 200", ceiling)
}
if target > 200 {
t.Errorf("target = %d, 不应超过 ceiling 200", target)
}
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
t.Logf("显式 -t 200: Timeout=%v MT=%d Retry=%d target=%d ceiling=%d cap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, target, ceiling, pool.Cap())
}
// =============================================================================
// 集成测试 8AdaptiveTimeout + ScanMetrics 双 RTT 追踪
// 验证两个 RTT 追踪器独立工作不干扰
// =============================================================================
func TestIntegration_DualRTTTracking(t *testing.T) {
adaptiveTO := NewAdaptiveTimeout(3 * time.Second)
metrics := &ScanMetrics{}
// 喂入相同的 RTT 数据到两个追踪器
for i := 0; i < 50; i++ {
rtt := 10 * time.Millisecond
adaptiveTO.Record(rtt)
metrics.RecordConnect(rtt)
}
// AdaptiveTimeout 用于连接超时
toValue := adaptiveTO.Timeout()
// ScanMetrics 用于池健康判断
rttFast := metrics.RTTFast()
ratio := metrics.RTTRatio()
if toValue > 3*time.Second {
t.Errorf("AdaptiveTimeout 应该 < 初始值: %v", toValue)
}
if rttFast < 8*time.Millisecond || rttFast > 12*time.Millisecond {
t.Errorf("ScanMetrics RTTFast 应接近 10ms: %v", rttFast)
}
if ratio < 0.8 || ratio > 1.2 {
t.Errorf("稳定 RTT 的 ratio 应接近 1.0: %.2f", ratio)
}
t.Logf("双追踪: AdaptiveTO=%v, MetricsFast=%v, Ratio=%.2f", toValue, rttFast, ratio)
}
// =============================================================================
// 集成测试 9:丢包环境下 Retry + ModuleThreadNum 联动
// 验证高丢包同时影响重试和并发
// =============================================================================
func TestIntegration_LossyNetwork_RetryAndConcurrency(t *testing.T) {
lossRates := []float64{0.0, 0.05, 0.10, 0.20, 0.40}
type result struct {
loss float64
retry int
mt int
}
var results []result
for _, loss := range lossRates {
profile := &NetworkProfile{
Env: EnvInternet,
RTTMedian: 80 * time.Millisecond,
RTTStddev: 20 * time.Millisecond,
LossRate: loss,
Samples: 20,
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
results = append(results, result{loss, config.MaxRetries, config.ModuleThreadNum})
}
// 重试次数应随丢包率单调递增
for i := 1; i < len(results); i++ {
if results[i].retry < results[i-1].retry {
t.Errorf("Retry 不递增: loss=%.2f retry=%d < loss=%.2f retry=%d",
results[i].loss, results[i].retry, results[i-1].loss, results[i-1].retry)
}
}
// 高丢包时 ModuleThreadNum 应降低
if results[len(results)-1].mt >= results[0].mt {
t.Errorf("40%%丢包的 MT(%d) 应 < 0%%丢包的 MT(%d)",
results[len(results)-1].mt, results[0].mt)
}
for _, r := range results {
t.Logf("loss=%.0f%%: Retry=%d MT=%d", r.loss*100, r.retry, r.mt)
}
}
+13 -10
View File
@@ -1,6 +1,7 @@
package core
import (
"context"
"sync"
"github.com/shadow1ng/fscan/common"
@@ -16,17 +17,17 @@ type LocalScanStrategy struct {
// NewLocalScanStrategy 创建新的本地扫描策略
func NewLocalScanStrategy() *LocalScanStrategy {
return &LocalScanStrategy{
BaseScanStrategy: NewBaseScanStrategy("本地扫描", FilterLocal),
BaseScanStrategy: NewBaseScanStrategy(i18n.GetText("scan_strategy_local_name"), FilterLocal),
}
}
// LogPluginInfo 重写以只显示通过-local指定的插件
func (s *LocalScanStrategy) LogPluginInfo(config *common.Config) {
func (s *LocalScanStrategy) LogPluginInfo(config *common.Config, session *common.ScanSession) {
localPlugin := config.LocalPlugin
if localPlugin != "" {
common.LogInfo(i18n.Tr("local_plugin_info", localPlugin))
session.LogInfo(i18n.Tr("local_plugin_info", localPlugin))
} else {
common.LogError(i18n.GetText("local_plugin_not_specified"))
session.LogError(i18n.GetText("local_plugin_not_specified"))
}
}
@@ -41,32 +42,34 @@ func (s *LocalScanStrategy) Description() string {
}
// Execute 执行本地扫描策略
func (s *LocalScanStrategy) Execute(config *common.Config, state *common.State, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
func (s *LocalScanStrategy) Execute(ctx context.Context, session *common.ScanSession, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
config := session.Config
// 输出扫描开始信息
s.LogScanStart()
s.LogScanStart(session)
// 验证插件配置
if err := s.ValidateConfiguration(); err != nil {
common.LogError(err.Error())
session.LogError(err.Error())
return
}
// 验证本地插件是否存在
if config.LocalPlugin != "" {
if !plugins.Exists(config.LocalPlugin) {
common.LogError(i18n.Tr("local_plugin_not_found", config.LocalPlugin))
session.LogError(i18n.Tr("local_plugin_not_found", config.LocalPlugin))
return
}
}
// 输出插件信息
s.LogPluginInfo(config)
s.LogPluginInfo(config, session)
// 准备目标(本地扫描通常只有一个目标,即本机)
targets := s.PrepareTargets(info)
// 执行扫描任务
ExecuteScanTasks(config, state, targets, s, ch, wg)
ExecuteScanTasks(ctx, session, targets, s, ch, wg)
}
// PrepareTargets 准备本地扫描目标
+281
View File
@@ -0,0 +1,281 @@
package core
import (
"context"
"math"
"net"
"sort"
"strconv"
"sync"
"time"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// NetworkEnv 网络环境分类
type NetworkEnv int
const (
EnvLAN NetworkEnv = iota // 内网: RTT < 5ms, 丢包 < 1%
EnvWAN // 局域网/专线: RTT 5~50ms, 丢包 < 5%
EnvInternet // 公网: RTT 50~200ms
EnvSlow // 慢速/高丢包: RTT > 200ms 或 丢包 > 10%
)
func (e NetworkEnv) String() string {
switch e {
case EnvLAN:
return i18n.GetText("net_env_lan")
case EnvWAN:
return i18n.GetText("net_env_wan")
case EnvInternet:
return i18n.GetText("net_env_internet")
default:
return i18n.GetText("net_env_slow")
}
}
// NetworkProfile 网络探测结果
type NetworkProfile struct {
Env NetworkEnv
RTTMin time.Duration
RTTMedian time.Duration
RTTP95 time.Duration
RTTStddev time.Duration
LossRate float64
Samples int
}
// RecommendConcurrency 根据探测结果推荐并发参数
// 返回 (target, ceiling)
// - target: 推荐的目标并发数
// - ceiling: 允许的最大并发数
//
// 如果用户显式指定了 -tceiling = 用户值,target 取 min(推荐值, 用户值)
// 如果用户未指定,target 和 ceiling 均为推荐值
func (p *NetworkProfile) RecommendConcurrency(userThreadNum int, explicit bool) (target, ceiling int) {
// 基于网络环境的缩放因子
var factor float64
switch p.Env {
case EnvLAN:
factor = 1.5
case EnvWAN:
factor = 1.0
case EnvInternet:
factor = 0.4
case EnvSlow:
factor = 0.15
}
recommended := int(float64(userThreadNum) * factor)
if recommended < 10 {
recommended = 10
}
// 丢包率高时进一步压缩
if p.LossRate > 0.05 {
recommended = int(float64(recommended) * (1.0 - p.LossRate))
if recommended < 10 {
recommended = 10
}
}
if explicit {
ceiling = userThreadNum
target = recommended
if target > ceiling {
target = ceiling
}
} else {
target = recommended
ceiling = recommended
}
return
}
// probePorts 探测用的端口列表(高响应率的常见端口)
var probePorts = []int{80, 443, 22, 445, 8080, 3389, 21, 8443}
func networkProbeAddress(host string, port int) string {
return net.JoinHostPort(host, strconv.Itoa(port))
}
// ProbeNetwork 探测目标网络环境
// 从 hosts 中抽样,用低并发 TCP 连接测量 RTT 和丢包率
// 整个过程控制在数秒内完成
func ProbeNetwork(ctx context.Context, hosts []string, session *common.ScanSession) *NetworkProfile {
if len(hosts) == 0 {
return defaultProfile()
}
// 抽样:均匀分布,最多 10 个
samples := pickSamples(hosts, 10)
probeTimeout := session.Config.Timeout
if probeTimeout > time.Second {
probeTimeout = time.Second
}
if probeTimeout < 500*time.Millisecond {
probeTimeout = 500 * time.Millisecond
}
var (
mu sync.Mutex
rtts []time.Duration
failures int
total int
)
sem := make(chan struct{}, 10)
var wg sync.WaitGroup
for _, host := range samples {
for _, port := range probePorts {
select {
case <-ctx.Done():
goto done
default:
}
total++
wg.Add(1)
sem <- struct{}{}
go func(h string, p int) {
defer func() { <-sem; wg.Done() }()
addr := networkProbeAddress(h, p)
start := time.Now()
conn, err := session.DialTCP(ctx, "tcp", addr, probeTimeout)
rtt := time.Since(start)
mu.Lock()
defer mu.Unlock()
if err != nil {
// 连接拒绝也是有效的 RTT 样本(说明对端可达)
if isConnectionRefused(err) {
rtts = append(rtts, rtt)
}
failures++
} else {
_ = conn.Close()
rtts = append(rtts, rtt)
}
}(host, port)
}
}
done:
wg.Wait()
return classifyNetwork(rtts, failures, total)
}
func classifyNetwork(rtts []time.Duration, failures, total int) *NetworkProfile {
if len(rtts) == 0 {
return defaultProfile()
}
sort.Slice(rtts, func(i, j int) bool { return rtts[i] < rtts[j] })
n := len(rtts)
median := rtts[n/2]
p95idx := int(float64(n) * 0.95)
if p95idx >= n {
p95idx = n - 1
}
p95 := rtts[p95idx]
// 标准差
var sum float64
for _, r := range rtts {
sum += float64(r)
}
mean := sum / float64(n)
var variance float64
for _, r := range rtts {
d := float64(r) - mean
variance += d * d
}
stddev := time.Duration(math.Sqrt(variance / float64(n)))
// 丢包率:只计算超时的(非 refused),但简化为 1 - 有效响应数/总数
lossRate := 1.0 - float64(n)/float64(total)
if lossRate < 0 {
lossRate = 0
}
// 分类
env := classifyEnv(median, lossRate)
return &NetworkProfile{
Env: env,
RTTMin: rtts[0],
RTTMedian: median,
RTTP95: p95,
RTTStddev: stddev,
LossRate: lossRate,
Samples: n,
}
}
func classifyEnv(median time.Duration, lossRate float64) NetworkEnv {
switch {
case lossRate > 0.10:
return EnvSlow
case median < 5*time.Millisecond && lossRate < 0.01:
return EnvLAN
case median < 50*time.Millisecond && lossRate < 0.05:
return EnvWAN
case median < 200*time.Millisecond:
return EnvInternet
default:
return EnvSlow
}
}
func defaultProfile() *NetworkProfile {
return &NetworkProfile{
Env: EnvWAN,
RTTMedian: 10 * time.Millisecond,
LossRate: 0,
Samples: 0,
}
}
// pickSamples 均匀抽样
func pickSamples(hosts []string, maxSamples int) []string {
if maxSamples <= 0 {
return nil
}
n := len(hosts)
if n <= maxSamples {
return hosts
}
step := n / maxSamples
samples := make([]string, 0, maxSamples)
for i := 0; i < n && len(samples) < maxSamples; i += step {
samples = append(samples, hosts[i])
}
return samples
}
func isConnectionRefused(err error) bool {
if err == nil {
return false
}
// connection refused 通常包含 "refused" 关键词
// 在不同 OS 上表现一致
return containsFold(err.Error(), "refused")
}
// isTimeoutError 判断是否为超时错误
func isTimeoutError(err error) bool {
if err == nil {
return false
}
if ne, ok := err.(net.Error); ok {
return ne.Timeout()
}
return containsFold(err.Error(), "timeout") || containsFold(err.Error(), "deadline")
}
+187
View File
@@ -0,0 +1,187 @@
package core
import (
"testing"
"time"
)
// =============================================================================
// 单元测试:classifyEnv — 网络环境分类
// =============================================================================
func TestClassifyEnv(t *testing.T) {
tests := []struct {
median time.Duration
lossRate float64
wantEnv NetworkEnv
desc string
}{
{1 * time.Millisecond, 0.0, EnvLAN, "1ms 零丢包 → 内网"},
{3 * time.Millisecond, 0.005, EnvLAN, "3ms 0.5%丢包 → 内网"},
{5 * time.Millisecond, 0.0, EnvWAN, "5ms 零丢包 → 局域网边界"},
{20 * time.Millisecond, 0.02, EnvWAN, "20ms 2%丢包 → 局域网"},
{50 * time.Millisecond, 0.03, EnvInternet, "50ms 3%丢包 → 公网边界"},
{100 * time.Millisecond, 0.05, EnvInternet, "100ms 5%丢包 → 公网"},
{300 * time.Millisecond, 0.05, EnvSlow, "300ms → 慢速"},
{50 * time.Millisecond, 0.15, EnvSlow, "50ms 15%丢包 → 高丢包归类慢速"},
{1 * time.Millisecond, 0.20, EnvSlow, "低延迟但高丢包 → 慢速"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := classifyEnv(tt.median, tt.lossRate)
if got != tt.wantEnv {
t.Errorf("classifyEnv(median=%v, loss=%.2f) = %v, want %v",
tt.median, tt.lossRate, got, tt.wantEnv)
}
})
}
}
// =============================================================================
// 单元测试:classifyNetwork — 从 RTT 样本推导 profile
// =============================================================================
func TestClassifyNetwork(t *testing.T) {
t.Run("内网 RTT 分布", func(t *testing.T) {
rtts := makeDurations([]int{1, 1, 1, 2, 2, 2, 3, 3, 4, 5}) // ms
p := classifyNetwork(rtts, 0, 10)
if p.Env != EnvLAN {
t.Errorf("env = %v, want LAN", p.Env)
}
if p.RTTMedian > 5*time.Millisecond {
t.Errorf("median = %v, want < 5ms", p.RTTMedian)
}
if p.LossRate != 0 {
t.Errorf("lossRate = %.2f, want 0", p.LossRate)
}
})
t.Run("公网 RTT 分布(低丢包)", func(t *testing.T) {
rtts := makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 150, 200, 300}) // ms
p := classifyNetwork(rtts, 0, 10) // 无丢包
if p.Env != EnvInternet {
t.Errorf("env = %v, want Internet", p.Env)
}
if p.LossRate != 0 {
t.Errorf("lossRate = %.2f, want 0", p.LossRate)
}
})
t.Run("高丢包归类为慢速", func(t *testing.T) {
rtts := makeDurations([]int{60, 70, 80, 90, 100}) // ms, 5 responded
p := classifyNetwork(rtts, 5, 10) // 50% loss
if p.Env != EnvSlow {
t.Errorf("env = %v, want Slow (高丢包)", p.Env)
}
})
t.Run("零样本降级", func(t *testing.T) {
p := classifyNetwork(nil, 5, 5)
if p.Env != EnvWAN {
t.Errorf("env = %v, want WAN (default)", p.Env)
}
if p.Samples != 0 {
t.Errorf("samples = %d, want 0", p.Samples)
}
})
}
// =============================================================================
// 单元测试:RecommendConcurrency
// =============================================================================
func TestRecommendConcurrency(t *testing.T) {
tests := []struct {
env NetworkEnv
lossRate float64
userT int
explicit bool
wantTMin int
wantTMax int
wantCeil int
desc string
}{
{EnvLAN, 0.0, 600, false, 800, 1000, -1, "内网自动: ×1.5"},
{EnvWAN, 0.0, 600, false, 550, 650, -1, "局域网自动: ×1.0"},
{EnvInternet, 0.0, 600, false, 200, 280, -1, "公网自动: ×0.4"},
{EnvSlow, 0.0, 600, false, 80, 100, -1, "慢速自动: ×0.15"},
{EnvInternet, 0.0, 200, true, 70, 100, 200, "公网显式: target<ceiling"},
{EnvLAN, 0.0, 100, true, 100, 160, 100, "内网显式: ceiling=用户值"},
{EnvInternet, 0.15, 600, false, 170, 240, -1, "公网高丢包: 进一步压缩"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
p := &NetworkProfile{Env: tt.env, LossRate: tt.lossRate, Samples: 10}
target, ceiling := p.RecommendConcurrency(tt.userT, tt.explicit)
if target < tt.wantTMin || target > tt.wantTMax {
t.Errorf("target = %d, want [%d, %d]", target, tt.wantTMin, tt.wantTMax)
}
if tt.explicit && ceiling != tt.wantCeil {
t.Errorf("ceiling = %d, want %d", ceiling, tt.wantCeil)
}
})
}
}
// =============================================================================
// 单元测试:pickSamples
// =============================================================================
func TestPickSamples(t *testing.T) {
hosts := make([]string, 100)
for i := range hosts {
hosts[i] = "host"
}
s := pickSamples(hosts, 10)
if len(s) != 10 {
t.Errorf("pickSamples(100, 10) = %d items, want 10", len(s))
}
s = pickSamples(hosts[:5], 10)
if len(s) != 5 {
t.Errorf("pickSamples(5, 10) = %d items, want 5", len(s))
}
s = pickSamples(nil, 10)
if len(s) != 0 {
t.Errorf("pickSamples(nil, 10) = %d items, want 0", len(s))
}
}
func TestNetworkProbeAddressUsesJoinHostPort(t *testing.T) {
tests := []struct {
host string
port int
want string
}{
{"127.0.0.1", 80, "127.0.0.1:80"},
{"::1", 443, "[::1]:443"},
{"2001:db8::1", 22, "[2001:db8::1]:22"},
}
for _, tt := range tests {
if got := networkProbeAddress(tt.host, tt.port); got != tt.want {
t.Fatalf("networkProbeAddress(%q, %d) = %q, want %q", tt.host, tt.port, got, tt.want)
}
}
}
// =============================================================================
// 辅助
// =============================================================================
func makeDurations(ms []int) []time.Duration {
ds := make([]time.Duration, len(ms))
for i, m := range ms {
ds[i] = time.Duration(m) * time.Millisecond
}
return ds
}
+549
View File
@@ -0,0 +1,549 @@
package core
import (
"sync/atomic"
"testing"
"time"
)
// =============================================================================
// 优化 1target/ceiling 分离
// =============================================================================
func TestOpt1_TargetCeilingSeparation_TuneConfig(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
if config.ThreadCeiling <= 0 {
t.Fatalf("ThreadCeiling 未被设置: %d", config.ThreadCeiling)
}
// 内网 factor=1.5,非显式 → target=ceiling=recommended
// 但 ceiling 应该 >= target
if config.ThreadCeiling < config.ThreadNum {
t.Errorf("Ceiling(%d) < ThreadNum(%d)", config.ThreadCeiling, config.ThreadNum)
}
t.Logf("target=%d, ceiling=%d", config.ThreadNum, config.ThreadCeiling)
}
func TestOpt1_TargetCeilingSeparation_ExplicitT(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 200
config.ThreadNumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvInternet,
RTTMedian: 100 * time.Millisecond,
RTTStddev: 30 * time.Millisecond,
LossRate: 0.0,
Samples: 20,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// 用户显式指定 -t → ceiling = threadNum = 200
if config.ThreadCeiling != 200 {
t.Errorf("显式 -t 200: ceiling=%d, want 200", config.ThreadCeiling)
}
if config.ThreadNum != 200 {
t.Errorf("显式 -t 200: threadNum=%d, want 200", config.ThreadNum)
}
}
func TestOpt1_PoolUsesCeiling(t *testing.T) {
metrics := &ScanMetrics{}
target, ceiling := 50, 200
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(target)
// 注入健康 metrics 让池增长
for i := 0; i < 200; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 多次 adjust,池应能增长超过 target 但不超过 ceiling
for i := 0; i < 30; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
finalCap := pool.Cap()
if finalCap <= target {
t.Errorf("池应能超过 target(%d): cap=%d", target, finalCap)
}
if finalCap > ceiling {
t.Errorf("池不应超过 ceiling(%d): cap=%d", ceiling, finalCap)
}
t.Logf("target=%d, ceiling=%d, finalCap=%d", target, ceiling, finalCap)
}
func TestOpt1_FDLimitConstraintsBothFields(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 1000
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 256, NumCPU: 4},
}
ep.TuneConfig(config, session)
maxFD := 256 * 6 / 10
if config.ThreadNum > maxFD {
t.Errorf("ThreadNum(%d) 超过 fd 限制(%d)", config.ThreadNum, maxFD)
}
if config.ThreadCeiling > maxFD {
t.Errorf("ThreadCeiling(%d) 超过 fd 限制(%d)", config.ThreadCeiling, maxFD)
}
}
// =============================================================================
// 优化 2RTT 漂移微调 target
// =============================================================================
func TestOpt2_RTTDriftReducesTarget(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 400, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(200)
// 建立基线:slow EMA 锚定在 1ms 附近
for i := 0; i < 500; i++ {
metrics.RecordConnect(1 * time.Millisecond)
}
origTarget := atomic.LoadInt32(&pool.target)
// RTT 突增到 100ms100 倍),大量喂入让 fast EMA 拉开差距
for i := 0; i < 1000; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
ratio := metrics.RTTRatio()
t.Logf("RTT ratio after spike: %.2f", ratio)
if ratio <= 3.0 {
t.Skipf("RTT ratio=%.2fEMA 差距不够大,跳过", ratio)
}
// 需要足够的新 metrics 让 assessHealth 的 deltaTotal >= 30
for i := 0; i < 50; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
// 多次 adjust 触发 maybeReduceTarget
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0)
pool.prevSnapshot = MetricsSnapshot{} // 重置快照让 delta 足够
pool.adjust()
}
newTarget := atomic.LoadInt32(&pool.target)
if newTarget >= origTarget {
t.Errorf("RTT 漂移后 target 应降低: %d -> %d (ratio=%.2f)", origTarget, newTarget, ratio)
}
// 不应低于 ceiling/5
minTarget := atomic.LoadInt32(&pool.ceiling) / 5
if newTarget < minTarget {
t.Errorf("target(%d) 低于下限(%d)", newTarget, minTarget)
}
t.Logf("RTT drift: ratio=%.2f, target %d -> %d (min=%d)", ratio, origTarget, newTarget, minTarget)
}
func TestOpt2_NoReductionWhenStable(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 400, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(200)
// 稳定 RTT
for i := 0; i < 200; i++ {
metrics.RecordConnect(10 * time.Millisecond)
}
origTarget := atomic.LoadInt32(&pool.target)
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
newTarget := atomic.LoadInt32(&pool.target)
if newTarget != origTarget {
t.Errorf("稳定 RTT 不应改变 target: %d -> %d", origTarget, newTarget)
}
}
// =============================================================================
// 优化 3assessHealth 阈值跟 NetworkEnv 关联
// =============================================================================
func TestOpt3_LANTighterThresholds(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, EnvLAN)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 10% exhaust rate — 对 LAN 来说应该是 Congested(阈值 8%
for i := 0; i < 100; i++ {
if i < 10 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
if pool.Cap() >= 100 {
t.Errorf("LAN 10%% exhaust 应触发降速: cap=%d", pool.Cap())
}
t.Logf("LAN tight threshold: cap=%d (from 100)", pool.Cap())
}
func TestOpt3_InternetLooseThresholds(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, EnvInternet)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 10% exhaust rate — 对 Internet 来说不算 Congested(阈值 25%),应是 Stressed
for i := 0; i < 100; i++ {
if i < 10 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(10 * time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
capAfter := pool.Cap()
// Internet 对 10% exhaust 只是 Stressed(×0.85),不是 Congested(×0.5
if capAfter < 80 {
t.Errorf("Internet 10%% exhaust 不应大幅降速: cap=%d", capAfter)
}
t.Logf("Internet loose threshold: cap=%d (from 100)", capAfter)
}
func TestOpt3_EnvAffectsHealthDecision(t *testing.T) {
envs := []struct {
env NetworkEnv
name string
}{
{EnvLAN, "LAN"},
{EnvWAN, "WAN"},
{EnvInternet, "Internet"},
}
var caps []int
for _, e := range envs {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, e.env)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
pool.inSlowStart = false
pool.tune(100)
// 相同的 12% exhaust rate
for i := 0; i < 100; i++ {
if i < 12 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
caps = append(caps, pool.Cap())
pool.Release()
t.Logf("%s: cap=%d (12%% exhaust)", e.name, caps[len(caps)-1])
}
// LAN 反应最激烈(cap 最低),Internet 最宽容(cap 最高)
if caps[0] >= caps[2] {
t.Errorf("LAN cap(%d) 应 < Internet cap(%d) for same exhaust rate", caps[0], caps[2])
}
}
// =============================================================================
// 优化 4:去掉 semaphoreants 池天然反压
// =============================================================================
func TestOpt4_SemaphoreRemoved(t *testing.T) {
// 验证 portScanTask 结构体不再有 semaphore 字段
// 如果 semaphore 被加回来,这段代码编译就会报 "unknown field"
_ = portScanTask{
host: "127.0.0.1",
port: 80,
addr: "127.0.0.1:80",
}
t.Log("portScanTask 无 semaphore 字段,反压由 ants pool 统一管理")
}
// =============================================================================
// 优化 5:扩充探测端口
// =============================================================================
func TestOpt5_ProbePortsExpanded(t *testing.T) {
if len(probePorts) < 5 {
t.Errorf("probePorts 只有 %d 个,应该扩充到至少 5 个", len(probePorts))
}
// 验证包含关键端口
required := map[int]bool{80: false, 443: false, 22: false}
for _, p := range probePorts {
if _, ok := required[p]; ok {
required[p] = true
}
}
for port, found := range required {
if !found {
t.Errorf("probePorts 缺少关键端口 %d", port)
}
}
// 验证没有重复
seen := make(map[int]bool)
for _, p := range probePorts {
if seen[p] {
t.Errorf("probePorts 有重复端口 %d", p)
}
seen[p] = true
}
t.Logf("probePorts = %v (%d 个)", probePorts, len(probePorts))
}
// =============================================================================
// 优化 6computeRetries 环境自适应
// =============================================================================
func TestOpt6_RetriesEnvAware(t *testing.T) {
lossRate := 0.3 // 30% 丢包
lanRetry := computeRetries(lossRate, EnvLAN)
wanRetry := computeRetries(lossRate, EnvWAN)
inetRetry := computeRetries(lossRate, EnvInternet)
// LAN 目标概率更严格(0.5%),应该重试更多;但上限更低(4)
// Internet 目标概率更宽松(2%),应该重试更少;但上限更高(6)
t.Logf("30%% loss: LAN=%d, WAN=%d, Internet=%d", lanRetry, wanRetry, inetRetry)
if lanRetry < 1 || lanRetry > 4 {
t.Errorf("LAN retry=%d, 应在 [1,4]", lanRetry)
}
if wanRetry < 1 || wanRetry > 5 {
t.Errorf("WAN retry=%d, 应在 [1,5]", wanRetry)
}
if inetRetry < 1 || inetRetry > 6 {
t.Errorf("Internet retry=%d, 应在 [1,6]", inetRetry)
}
}
func TestOpt6_RetriesMaxByEnv(t *testing.T) {
// 高丢包率,各环境应返回各自上限
lanMax := computeRetries(0.99, EnvLAN)
wanMax := computeRetries(0.99, EnvWAN)
inetMax := computeRetries(0.99, EnvInternet)
if lanMax != 4 {
t.Errorf("LAN max retry=%d, want 4", lanMax)
}
if wanMax != 5 {
t.Errorf("WAN max retry=%d, want 5", wanMax)
}
if inetMax != 6 {
t.Errorf("Internet max retry=%d, want 6", inetMax)
}
}
func TestOpt6_RetriesMathCorrectness(t *testing.T) {
envs := []struct {
env NetworkEnv
targetProb float64
name string
}{
{EnvLAN, 0.005, "LAN"},
{EnvWAN, 0.01, "WAN"},
{EnvInternet, 0.02, "Internet"},
}
for _, e := range envs {
for _, loss := range []float64{0.05, 0.10, 0.20, 0.30} {
retries := computeRetries(loss, e.env)
prob := 1.0
for i := 0; i < retries; i++ {
prob *= loss
}
// 重试后全失败概率应 < targetProb(除非被 clamp 了)
if prob >= e.targetProb && retries < 4 {
t.Errorf("%s loss=%.0f%% retries=%d: P=%.6f >= %.3f",
e.name, loss*100, retries, prob, e.targetProb)
}
}
}
}
// =============================================================================
// 端到端集成:全链路验证
// =============================================================================
func TestOptAll_EndToEnd_LANToPool(t *testing.T) {
// 模拟内网探测 → TuneConfig → 创建池 → 池根据 env 自适应
profile := classifyNetwork(
makeDurations([]int{1, 1, 2, 2, 2, 3, 3, 3, 4, 5}),
0, 10,
)
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 65536, NumCPU: 8}}
ep.TuneConfig(config, session)
// 验证 env 被存储
if config.DetectedNetworkEnv != int(EnvLAN) {
t.Errorf("DetectedNetworkEnv=%d, want %d(LAN)", config.DetectedNetworkEnv, int(EnvLAN))
}
// 验证 ceiling 合理
if config.ThreadCeiling < config.ThreadNum {
t.Errorf("ceiling(%d) < target(%d)", config.ThreadCeiling, config.ThreadNum)
}
// 创建池并验证 env 传递
netEnv := NetworkEnv(config.DetectedNetworkEnv)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(config.ThreadNum, config.ThreadCeiling, func(interface{}) {}, metrics, netEnv)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if pool.networkEnv != EnvLAN {
t.Errorf("池的 networkEnv=%v, want LAN", pool.networkEnv)
}
t.Logf("端到端 LAN: target=%d ceiling=%d env=%v maxRetry=%d",
config.ThreadNum, config.ThreadCeiling, netEnv, config.MaxRetries)
}
func TestOptAll_EndToEnd_InternetToPool(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 130, 140, 150}),
0, 10,
)
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 4096, NumCPU: 4}}
ep.TuneConfig(config, session)
if config.DetectedNetworkEnv != int(EnvInternet) {
t.Errorf("DetectedNetworkEnv=%d, want %d(Internet)", config.DetectedNetworkEnv, int(EnvInternet))
}
// 公网 target 应明显低于默认 600
if config.ThreadNum >= 600 {
t.Errorf("公网 threadNum=%d, 应 < 600", config.ThreadNum)
}
// ceiling 应 == target(非显式模式)
if config.ThreadCeiling != config.ThreadNum {
t.Errorf("非显式模式 ceiling(%d) != target(%d)", config.ThreadCeiling, config.ThreadNum)
}
netEnv := NetworkEnv(config.DetectedNetworkEnv)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(config.ThreadNum, config.ThreadCeiling, func(interface{}) {}, metrics, netEnv)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 注入 12% exhaustInternet 环境应只是 Stressed 而不是 Congested
pool.inSlowStart = false
pool.tune(config.ThreadNum)
for i := 0; i < 100; i++ {
if i < 12 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(80 * time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
// cap 不应被砍到一半以下(Stressed 只降 15%
if pool.Cap() < config.ThreadNum*7/10 {
t.Errorf("Internet 12%% exhaust 降速过猛: %d -> %d", config.ThreadNum, pool.Cap())
}
t.Logf("端到端 Internet: target=%d ceiling=%d cap_after_stress=%d",
config.ThreadNum, config.ThreadCeiling, pool.Cap())
}
+102
View File
@@ -0,0 +1,102 @@
package core
import (
"errors"
"fmt"
"net"
"strings"
"testing"
)
// =============================================================================
// Benchmark: containsFold vs strings.ToLower + strings.Contains
// =============================================================================
func BenchmarkContainsFold(b *testing.B) {
err := errors.New("connection reset by peer: 192.168.1.1:445")
b.ResetTimer()
for i := 0; i < b.N; i++ {
containsFold(err.Error(), "connection reset")
}
}
func BenchmarkStringsToLowerContains(b *testing.B) {
err := errors.New("connection reset by peer: 192.168.1.1:445")
b.ResetTimer()
for i := 0; i < b.N; i++ {
strings.Contains(strings.ToLower(err.Error()), "connection reset")
}
}
// =============================================================================
// Benchmark: fmt.Sprintf vs net.JoinHostPort + fmtPort
// =============================================================================
func BenchmarkFmtSprintfAddr(b *testing.B) {
host := "192.168.1.1"
port := 445
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = fmt.Sprintf("%s:%d", host, port)
}
}
func BenchmarkJoinHostPortFmtPort(b *testing.B) {
host := "192.168.1.1"
port := 445
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = net.JoinHostPort(host, fmtPort(port))
}
}
func BenchmarkFmtPort(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = fmtPort(445)
}
}
// =============================================================================
// Benchmark: readFromConn buffer pre-allocation
// =============================================================================
func BenchmarkAppendFromNil(b *testing.B) {
data := []byte("HTTP/1.1 200 OK\r\nServer: nginx")
chunk := data[:10]
b.ResetTimer()
for i := 0; i < b.N; i++ {
var result []byte
result = append(result, chunk...)
result = append(result, chunk...)
_ = result
}
}
func BenchmarkAppendPreAllocated(b *testing.B) {
data := []byte("HTTP/1.1 200 OK\r\nServer: nginx")
chunk := data[:10]
b.ResetTimer()
for i := 0; i < b.N; i++ {
result := make([]byte, 0, 4096)
result = append(result, chunk...)
result = append(result, chunk...)
_ = result
}
}
// =============================================================================
// Benchmark: AdaptiveTimeout computation under lock vs outside lock
// =============================================================================
func BenchmarkAdaptiveTimeoutComputation(b *testing.B) {
at := NewAdaptiveTimeout(3000 * 1000000) // 3s in ns
// Warm up: add 64 samples
for i := 0; i < 64; i++ {
at.Record(10 * 1000000) // 10ms in ns
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = at.Timeout()
}
}
+460 -148
View File
@@ -1,8 +1,10 @@
package core
import (
"context"
"fmt"
"net"
"strconv"
"strings"
"sync"
"sync/atomic"
@@ -34,31 +36,59 @@ var resourceExhaustedPatterns = []string{
"no buffer space available",
"cannot assign requested address",
"connection reset by peer",
"发包受限",
i18n.GetText("network_rate_limited_pattern"),
"rate limited",
}
// closedPatterns 连接已关闭的错误模式
var closedPatterns = []string{
"broken pipe",
"connection reset",
"connection refused",
"use of closed network connection",
"connection was forcibly closed",
}
// proxyErrorTexts 代理错误响应文本模式
var proxyErrorTexts = []string{
"connection refused",
"host unreachable",
"network unreachable",
"connection timed out",
"proxy error",
"gateway error",
"bad gateway",
"502",
"503",
}
// resultCollector 结果收集器,用于并发安全地收集扫描结果
// 使用 map 实现:O(1) 的添加和删除,无顺序依赖问题
type resultCollector struct {
mu sync.Mutex
addrs map[string]struct{}
mu sync.Mutex
addrs map[string]struct{}
stream chan<- string
}
// newResultCollector 创建结果收集器
func newResultCollector() *resultCollector {
func newResultCollector(stream chan<- string) *resultCollector {
return &resultCollector{
addrs: make(map[string]struct{}),
addrs: make(map[string]struct{}),
stream: stream,
}
}
// Add 添加一个扫描结果
func (c *resultCollector) Add(addr string) {
c.mu.Lock()
if _, dup := c.addrs[addr]; dup {
c.mu.Unlock()
return
}
c.addrs[addr] = struct{}{}
c.mu.Unlock()
if c.stream != nil {
c.stream <- addr
}
}
// GetAll 获取所有结果
func (c *resultCollector) GetAll() []string {
c.mu.Lock()
result := make([]string, 0, len(c.addrs))
@@ -71,9 +101,9 @@ func (c *resultCollector) GetAll() []string {
// portScanTask 端口扫描任务(轻量级,用于滑动窗口调度)
type portScanTask struct {
host string
port int
semaphore chan struct{} // 完成时释放窗口槽位
host string
port int
addr string // 预格式化的 host:port,避免 fmt.Sprintf 热路径分配
}
// failedPortInfo 失败端口信息
@@ -110,16 +140,34 @@ func (f *failedPortCollector) Count() int {
// EnhancedPortScan 高性能端口扫描函数
// 使用滑动窗口调度 + 自适应线程池 + 流式迭代器
func EnhancedPortScan(hosts []string, ports string, timeout int64, config *common.Config, state *common.State) []string {
common.LogDebug(fmt.Sprintf("[PortScan] 开始: %d个主机, 线程数=%d", len(hosts), config.ThreadNum))
// stream: 可选,非 nil 时每发现开放端口立即发送 addr,扫描结束后关闭
func EnhancedPortScan(ctx context.Context, hosts []string, ports string, timeout int64, session *common.ScanSession, stream chan<- string) []string {
config := session.Config
state := session.State
session.LogDebug(i18n.Tr("port_scan_debug_start", len(hosts), config.ThreadNum))
// 大规模扫描预筛:跨多个 /24 时先做网段探活,跳过空网段
if len(hosts) > subnetProbeThreshold {
hosts = probeSubnets(ctx, hosts, time.Duration(timeout)*time.Second, session)
if len(hosts) == 0 {
session.LogInfo(i18n.GetText("port_scan_no_alive_subnet"))
if stream != nil {
close(stream)
}
return nil
}
}
// 解析端口和排除端口
portList := parsers.ParsePort(ports)
if len(portList) == 0 {
common.LogError(i18n.Tr("invalid_port", ports))
session.LogError(i18n.Tr("invalid_port", ports))
if stream != nil {
close(stream)
}
return nil
}
common.LogDebug(fmt.Sprintf("[PortScan] 端口解析完成: %d个端口", len(portList)))
session.LogDebug(i18n.Tr("port_scan_debug_ports_parsed", len(portList)))
// 使用config中的排除端口配置
excludePorts := parsers.ParsePort(config.Target.ExcludePorts)
@@ -129,80 +177,89 @@ func EnhancedPortScan(hosts []string, ports string, timeout int64, config *commo
}
// 检查代理可靠性,如果存在全回显问题则警告
if common.IsProxyEnabled() && !common.IsProxyReliable() {
common.LogError("检测到代理存在全回显问题,端口扫描结果可能不准确")
if session.ProxyEnabled() && !session.ProxyReliable() {
session.LogError(i18n.GetText("proxy_echo_warning"))
}
// 创建流式迭代器(O(1) 内存,端口喷洒策略)
iter := NewSocketIterator(hosts, portList, exclude)
totalTasks := iter.Total()
common.LogDebug(fmt.Sprintf("[PortScan] 总任务数: %d", totalTasks))
session.LogDebug(i18n.Tr("port_scan_debug_total_tasks", totalTasks))
// 使用传入的配置
// 并发参数(已由 EnvironmentProfile.TuneConfig 调整过)
threadNum := config.ThreadNum
// 大规模扫描警告和线程数自动调整
// 大规模扫描额外约束
if totalTasks > 100000 {
common.LogInfo(fmt.Sprintf("大规模扫描: %d 个目标 (%d主机 × %d端口)", totalTasks, len(hosts), len(portList)))
// 如果任务数超过100万且线程数大于300,自动降低线程数
session.LogInfo(i18n.Tr("large_scan_notice", totalTasks, len(hosts), len(portList)))
if totalTasks > 1000000 && threadNum > 300 {
oldThreadNum := threadNum
threadNum = 300
common.LogInfo(fmt.Sprintf("自动调整线程数: %d -> %d (大规模扫描优化)", oldThreadNum, threadNum))
session.LogInfo(i18n.Tr("large_scan_thread_adjusted", oldThreadNum, threadNum))
}
}
// 初始化端口扫描进度条
if totalTasks > 0 && config.Output.ShowProgress {
description := fmt.Sprintf("端口扫描中(%d线程)", threadNum)
common.InitProgressBar(int64(totalTasks), description)
description := i18n.Tr("port_scan_progress_description", threadNum)
common.InitProgressBar(totalTasks, description)
}
common.LogDebug("[PortScan] 进度条初始化完成")
session.LogDebug(i18n.GetText("port_scan_debug_progress_ready"))
// 初始化并发控制
to := time.Duration(timeout) * time.Second
var count int64
collector := newResultCollector()
adaptiveTO := NewAdaptiveTimeout(to)
metrics := &ScanMetrics{}
var count atomic.Int64
collector := newResultCollector(stream)
failedCollector := &failedPortCollector{}
var wg sync.WaitGroup
common.LogDebug(fmt.Sprintf("[PortScan] 开始创建线程池, size=%d", threadNum))
// 创建自适应线程池(支持动态调整)
pool, err := NewAdaptivePool(threadNum, func(task interface{}) {
ceiling := config.ThreadCeiling
if ceiling < threadNum {
ceiling = threadNum
}
netEnv := NetworkEnv(config.DetectedNetworkEnv)
session.LogDebug(i18n.Tr("port_scan_debug_pool_create", threadNum))
pool, err := NewAdaptivePool(threadNum, ceiling, func(task interface{}) {
taskInfo, ok := task.(portScanTask)
if !ok {
return
}
defer func() {
<-taskInfo.semaphore // 释放窗口槽位
wg.Done()
}()
defer wg.Done()
addr := fmt.Sprintf("%s:%d", taskInfo.host, taskInfo.port)
scanSinglePort(taskInfo.host, taskInfo.port, addr, to, &count, collector, failedCollector, config, state)
scanSinglePort(ctx, taskInfo.host, taskInfo.port, taskInfo.addr, adaptiveTO, metrics, &count, collector, failedCollector, session)
common.UpdateProgressBar(1)
}, state)
}, metrics, netEnv)
if err != nil {
common.LogError(i18n.Tr("thread_pool_create_failed", err))
session.LogError(i18n.Tr("thread_pool_create_failed", err))
if stream != nil {
close(stream)
}
return nil
}
common.LogDebug("[PortScan] 线程池创建成功")
session.LogDebug(i18n.GetText("port_scan_debug_pool_created"))
defer pool.Release()
common.LogDebug("[PortScan] 开始滑动窗口调度")
// 滑动窗口调度:维护固定数量的"飞行中"任务
slidingWindowSchedule(iter, pool, &wg, threadNum)
common.LogDebug("[PortScan] 滑动窗口调度完成")
session.LogDebug(i18n.GetText("port_scan_debug_schedule_start"))
// 滑动窗口调度
slidingWindowSchedule(iter, pool, &wg)
session.LogDebug(i18n.GetText("port_scan_debug_schedule_done"))
// 收集结果
aliveAddrs := collector.GetAll()
// 关闭流式通知 channel
if stream != nil {
close(stream)
}
// 完成端口扫描进度条
if common.IsProgressActive() {
common.FinishProgressBar()
}
common.LogInfo(i18n.Tr("port_scan_complete", count))
session.LogInfo(i18n.Tr("port_scan_complete", count.Load()))
// 检查扫描失败率,如果过高则警告用户
resourceErrors := state.GetResourceExhaustedCount()
@@ -213,58 +270,69 @@ func EnhancedPortScan(hosts []string, ports string, timeout int64, config *commo
if failureRate > 20 {
// 失败率超过20%,严重警告
common.LogError(i18n.Tr("scan_failure_rate_high", fmt.Sprintf("%.1f%%", failureRate), failedCount, totalTasks))
common.LogError(i18n.GetText("scan_failure_reason"))
common.LogError(i18n.Tr("scan_reduce_threads_suggestion", threadNum))
session.LogError(i18n.Tr("scan_failure_rate_high", fmt.Sprintf("%.1f%%", failureRate), failedCount, totalTasks))
session.LogError(i18n.GetText("scan_failure_reason"))
session.LogError(i18n.Tr("scan_reduce_threads_suggestion", threadNum))
} else if failureRate > 5 {
// 失败率5-20%,一般警告
common.LogInfo(i18n.Tr("scan_partial_failure", fmt.Sprintf("%.1f%%", failureRate), failedCount, totalTasks))
common.LogInfo(i18n.Tr("scan_reduce_threads_accuracy", threadNum))
session.LogInfo(i18n.Tr("scan_partial_failure", fmt.Sprintf("%.1f%%", failureRate), failedCount, totalTasks))
session.LogInfo(i18n.Tr("scan_reduce_threads_accuracy", threadNum))
}
}
if resourceErrors > 0 {
common.LogError(i18n.Tr("resource_exhausted_warning", resourceErrors))
session.LogError(i18n.Tr("resource_exhausted_warning", resourceErrors))
}
return aliveAddrs
}
// slidingWindowSchedule 滑动窗口调度器
// 核心思想:维护固定数量的"飞行中"任务,一个完成立即补充新的
// 优势:避免任务队列堆积,内存使用恒定
func slidingWindowSchedule(iter *SocketIterator, pool *AdaptivePool, wg *sync.WaitGroup, windowSize int) {
// 使用信号量控制窗口大小
semaphore := make(chan struct{}, windowSize)
// ants.PoolWithFunc.Invoke 在池满时阻塞,天然提供反压,无需额外 semaphore
func slidingWindowSchedule(iter *SocketIterator, pool *AdaptivePool, wg *sync.WaitGroup) {
for {
host, port, ok := iter.Next()
if !ok {
break
}
// 获取窗口槽位(阻塞直到有空位)
semaphore <- struct{}{}
wg.Add(1)
task := portScanTask{
host: host,
port: port,
semaphore: semaphore,
host: host,
port: port,
addr: net.JoinHostPort(host, fmtPort(port)),
}
if err := pool.Invoke(task); err != nil {
wg.Done()
}
_ = pool.Invoke(task)
}
// 等待所有任务完成
wg.Wait()
}
// fmtPort 无分配的端口号格式化
func fmtPort(port int) string {
if port < 0 || port > 65535 {
return "0"
}
// 预分配足够大的缓冲区
var buf [6]byte
i := len(buf)
for port > 0 || i == len(buf) {
i--
buf[i] = byte(port%10) + '0'
port /= 10
}
return string(buf[i:])
}
// connectWithRetry 带重试的TCP连接 - 只对资源耗尽错误重试
func connectWithRetry(addr string, timeout time.Duration, maxRetries int, state *common.State) (net.Conn, error) {
func connectWithRetry(ctx context.Context, session *common.ScanSession, addr string, timeout time.Duration, maxRetries int) (net.Conn, error) {
var lastErr error
for attempt := 0; attempt < maxRetries; attempt++ {
conn, err := common.WrapperTcpWithTimeout("tcp", addr, timeout)
conn, err := session.DialTCP(ctx, "tcp", addr, timeout)
if err == nil {
return conn, nil
@@ -278,11 +346,11 @@ func connectWithRetry(addr string, timeout time.Duration, maxRetries int, state
}
// 记录资源耗尽错误
state.IncrementResourceExhaustedCount()
session.State.IncrementResourceExhaustedCount()
// 指数退避:第1次等50ms,第2次等150ms
// 指数退避:200ms → 600ms → 1200ms
if attempt < maxRetries-1 {
waitTime := time.Duration(50*(attempt+1)) * time.Millisecond
waitTime := time.Duration(200*(1<<uint(attempt))) * time.Millisecond
time.Sleep(waitTime)
}
}
@@ -298,7 +366,7 @@ func isResourceExhaustedError(err error) bool {
errStr := err.Error()
for _, pattern := range resourceExhaustedPatterns {
if strings.Contains(errStr, pattern) {
if containsFold(errStr, pattern) {
return true
}
}
@@ -306,14 +374,54 @@ func isResourceExhaustedError(err error) bool {
return false
}
// containsFold 忽略大小写的子串匹配,避免 strings.ToLower 分配
func containsFold(s, substr string) bool {
if len(substr) == 0 {
return true
}
if len(substr) > len(s) {
return false
}
for i := 0; i <= len(s)-len(substr); i++ {
if matchFold(s[i:i+len(substr)], substr) {
return true
}
}
return false
}
// matchFold 忽略大小写逐字节比较
func matchFold(a, b string) bool {
if len(a) != len(b) {
return false
}
for i := 0; i < len(a); i++ {
ca, cb := a[i], b[i]
if ca >= 'A' && ca <= 'Z' {
ca += 'a' - 'A'
}
if cb >= 'A' && cb <= 'Z' {
cb += 'a' - 'A'
}
if ca != cb {
return false
}
}
return true
}
// buildServiceLogMessage 构建服务识别的日志信息
// 格式: addr service [Product:xxx ||Version:xxx] Banner:(xxx)
// 格式: addr-or-url service [Product:xxx ||Version:xxx] Banner:(xxx)
func buildServiceLogMessage(addr string, serviceInfo *ServiceInfo, isWeb bool) string {
var msg strings.Builder
msg.WriteString(fmt.Sprintf("%-21s", addr))
displayTarget := addr
if isWeb {
displayTarget = buildWebServiceURL(addr, serviceInfo)
}
fmt.Fprintf(&msg, "%-30s", displayTarget)
if serviceInfo.Name != "unknown" {
msg.WriteString(fmt.Sprintf(" %-8s", serviceInfo.Name))
fmt.Fprintf(&msg, " %-8s", serviceInfo.Name)
}
// 构建 [Product:xxx ||Version:xxx] 格式
@@ -325,44 +433,101 @@ func buildServiceLogMessage(addr string, serviceInfo *ServiceInfo, isWeb bool) s
info = append(info, fmt.Sprintf("Version:%s", serviceInfo.Version))
}
if len(info) > 0 {
msg.WriteString(fmt.Sprintf(" [%s]", strings.Join(info, " ||")))
fmt.Fprintf(&msg, " [%s]", strings.Join(info, " ||"))
}
// Banner 信息
if len(serviceInfo.Banner) > 0 {
banner := strings.TrimSpace(serviceInfo.Banner)
if len(banner) > 80 {
banner = banner[:80] + "..."
}
msg.WriteString(fmt.Sprintf(" Banner:(%s)", banner))
banner = truncateString(banner, 80)
fmt.Fprintf(&msg, " Banner:(%s)", banner)
}
return msg.String()
}
// scanSinglePort 扫描单个端口并进行服务识别(重构后的简洁版本)
func scanSinglePort(host string, port int, addr string, timeout time.Duration, count *int64, collector *resultCollector, failedCollector *failedPortCollector, config *common.Config, state *common.State) {
// 步骤1:建立连接
conn, err := connectWithRetry(addr, timeout, 3, state)
func truncateString(s string, maxRunes int) string {
if maxRunes < 0 {
return s
}
for i := range s {
if maxRunes == 0 {
return s[:i] + "..."
}
maxRunes--
}
return s
}
func buildWebServiceURL(addr string, serviceInfo *ServiceInfo) string {
protocol := "http"
serviceName := ""
if serviceInfo != nil {
serviceName = strings.ToLower(serviceInfo.Name)
}
if strings.Contains(serviceName, "https") || strings.Contains(serviceName, "ssl") || strings.Contains(serviceName, "tls") {
protocol = "https"
}
host, port, err := net.SplitHostPort(addr)
if err != nil {
return fmt.Sprintf("%s://%s", protocol, addr)
}
if protocol == "http" && port == "80" {
return fmt.Sprintf("http://%s", urlHost(host))
}
if protocol == "https" && port == "443" {
return fmt.Sprintf("https://%s", urlHost(host))
}
return fmt.Sprintf("%s://%s", protocol, net.JoinHostPort(host, port))
}
func urlHost(host string) string {
if strings.Contains(host, ":") && !strings.HasPrefix(host, "[") {
return "[" + host + "]"
}
return host
}
// scanSinglePort 扫描单个端口并进行服务识别(重构后的简洁版本)
func scanSinglePort(ctx context.Context, host string, port int, addr string, adaptiveTO *AdaptiveTimeout, metrics *ScanMetrics, count *atomic.Int64, collector *resultCollector, failedCollector *failedPortCollector, session *common.ScanSession) {
config := session.Config
timeout := adaptiveTO.Timeout()
// 步骤1:建立连接
start := time.Now()
conn, err := connectWithRetry(ctx, session, addr, timeout, 2)
if err != nil {
rtt := time.Since(start)
switch {
case isResourceExhaustedError(err):
metrics.RecordExhausted()
case isTimeoutError(err):
metrics.RecordTimeout()
default:
metrics.RecordRefused(rtt)
}
handleConnectionFailure(err, host, port, addr, failedCollector)
return
}
rtt := time.Since(start)
metrics.RecordConnect(rtt)
adaptiveTO.Record(rtt)
// 步骤1.5:代理连接深度验证(防止透明代理/全回显代理的假连接问题)
valid, verifyMethod := verifyProxyConnectionDeep(conn, addr)
valid, verifyMethod := verifyProxyConnectionDeep(conn, addr, session)
if !valid {
common.LogDebug(fmt.Sprintf("代理验证失败 %s: %s", addr, verifyMethod))
session.LogDebug(i18n.Tr("proxy_verify_failed", addr, verifyMethod))
_ = conn.Close()
return
}
// 步骤1.6:如果使用了代理且进行了数据交互,需要重建连接
// 因为验证阶段可能读取了Banner或发送了HTTP GET探测,污染了连接状态
if common.IsProxyEnabled() && verifyMethod != "direct" {
if session.ProxyEnabled() && verifyMethod != "direct" {
_ = conn.Close()
// 重新建立干净的连接用于服务识别
conn, err = connectWithRetry(addr, timeout, 3, state)
conn, err = connectWithRetry(ctx, session, addr, timeout, 2)
if err != nil {
handleConnectionFailure(err, host, port, addr, failedCollector)
return
@@ -370,31 +535,36 @@ func scanSinglePort(host string, port int, addr string, timeout time.Duration, c
}
// 步骤2:记录开放端口
atomic.AddInt64(count, 1)
collector.Add(addr)
saveOpenPort(host, port)
count.Add(1)
saveOpenPort(session, host, port)
// 步骤3:服务识别(Scanner负责关闭连接,包括探测中可能创建的新连接)
scanner := NewSmartPortInfoScanner(host, port, conn, timeout, config)
scanner := NewSmartPortInfoScanner(ctx, host, port, conn, timeout, config, session)
// 服务探测超时自适应:用 RTT 采样值约束读超时上限
// 下限 500ms:服务处理需要时间,不能太激进
if rttTO := adaptiveTO.Timeout(); rttTO < timeout {
maxMS := int(rttTO.Milliseconds()) * 6
if maxMS < 500 {
maxMS = 500
}
scanner.info.maxReadTimeoutMS = maxMS
}
defer scanner.Close()
serviceInfo, _ := scanner.SmartIdentify()
// 步骤4:处理结果
processServiceResult(host, port, addr, serviceInfo, config)
processServiceResult(ctx, host, port, addr, serviceInfo, config, session)
collector.Add(addr)
}
// handleConnectionFailure 处理连接失败
// 只收集资源耗尽类错误,timeout 是正常的扫描结果(防火墙 drop)不计入失败
func handleConnectionFailure(err error, host string, port int, addr string, failedCollector *failedPortCollector) {
if isResourceExhaustedError(err) || isTimeoutError(err) {
if isResourceExhaustedError(err) {
failedCollector.Add(host, port, addr)
}
}
// isTimeoutError 判断是否为超时错误
func isTimeoutError(err error) bool {
return err != nil && strings.Contains(err.Error(), "i/o timeout")
}
// verifyProxyConnectionDeep 深度验证代理连接是否真正可用
// 防止透明代理/全回显代理的假连接问题
// 返回: (是否有效, 验证方式)
@@ -403,9 +573,10 @@ func isTimeoutError(err error) bool {
// 1. 快速 Banner 检测 (100ms) - 大部分服务会主动发送数据
// 2. 轻量探测 (发送 \r\n) - 触发某些服务响应,同时不污染协议状态
// 3. 短超时等待 (500ms) - 平衡准确性和性能
func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
// 如果没有使用代理,跳过验证
if !common.IsProxyEnabled() {
func verifyProxyConnectionDeep(conn net.Conn, addr string, session *common.ScanSession) (bool, string) {
// 无代理或SOCKS5代理:跳过深度验证
// SOCKS5协议层已验证连接可达性,连接成功即端口开放
if !session.ProxyEnabled() || session.IsSOCKS5Proxy() {
return true, "direct"
}
@@ -420,7 +591,7 @@ func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
if n > 0 {
if isProxyErrorResponse(buf[:n]) {
common.LogDebug(fmt.Sprintf("代理返回错误响应 %s", addr))
common.LogDebug(i18n.Tr("proxy_error_response", addr))
return false, "proxy_error"
}
return true, "banner"
@@ -437,7 +608,7 @@ func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
_ = conn.SetWriteDeadline(time.Time{})
if writeErr != nil && isConnectionClosed(writeErr) {
common.LogDebug(fmt.Sprintf("探测写入失败 %s: %v", addr, writeErr))
common.LogDebug(i18n.Tr("proxy_probe_write_failed", addr, writeErr))
return false, "write_failed"
}
@@ -449,7 +620,7 @@ func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
if n > 0 {
if isProxyErrorResponse(buf[:n]) {
common.LogDebug(fmt.Sprintf("代理探测返回错误 %s", addr))
common.LogDebug(i18n.Tr("proxy_probe_error_response", addr))
return false, "proxy_error"
}
return true, "probe"
@@ -457,10 +628,10 @@ func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
// 阶段4: 最终判断
if readErr != nil {
errLower := strings.ToLower(readErr.Error())
errStr := readErr.Error()
for _, pattern := range proxyFailurePatterns {
if strings.Contains(errLower, pattern) {
common.LogDebug(fmt.Sprintf("代理连接被拒绝 %s: %v", addr, readErr))
if containsFold(errStr, pattern) {
common.LogDebug(i18n.Tr("proxy_connection_rejected", addr, readErr))
return false, "proxy_reject"
}
}
@@ -470,7 +641,7 @@ func verifyProxyConnectionDeep(conn net.Conn, addr string) (bool, string) {
// 在透明代理环境下,ProxyReliable 检测可能被污染,不可信
// 因此采用更保守的策略:无响应一律判定为关闭
// 这样可以避免透明代理导致的全端口误报问题
common.LogDebug(fmt.Sprintf("代理连接无响应,判定为端口关闭 %s", addr))
common.LogDebug(i18n.Tr("proxy_no_response_closed", addr))
return false, "no_response"
}
@@ -492,21 +663,9 @@ func isProxyErrorResponse(data []byte) bool {
}
// 检查常见的代理错误文本
dataStr := strings.ToLower(string(data))
proxyErrorTexts := []string{
"connection refused",
"host unreachable",
"network unreachable",
"connection timed out",
"proxy error",
"gateway error",
"bad gateway",
"502",
"503",
}
dataStr := string(data)
for _, errText := range proxyErrorTexts {
if strings.Contains(dataStr, errText) {
if containsFold(dataStr, errText) {
return true
}
}
@@ -520,17 +679,9 @@ func isConnectionClosed(err error) bool {
return false
}
errStr := strings.ToLower(err.Error())
closedPatterns := []string{
"broken pipe",
"connection reset",
"connection refused",
"use of closed network connection",
"connection was forcibly closed",
}
errStr := err.Error()
for _, pattern := range closedPatterns {
if strings.Contains(errStr, pattern) {
if containsFold(errStr, pattern) {
return true
}
}
@@ -539,8 +690,8 @@ func isConnectionClosed(err error) bool {
}
// saveOpenPort 保存开放端口结果
func saveOpenPort(host string, port int) {
_ = common.SaveResult(&output.ScanResult{
func saveOpenPort(session *common.ScanSession, host string, port int) {
_ = session.SaveResult(&output.ScanResult{
Time: time.Now(),
Type: output.TypePort,
Target: host,
@@ -549,34 +700,59 @@ func saveOpenPort(host string, port int) {
})
}
// correctServiceByBanner 根据 banner 特征校正被 nmap 指纹误匹配的服务名
func correctServiceByBanner(info *ServiceInfo) {
if info == nil || info.Banner == "" {
return
}
banner := strings.ToLower(info.Banner)
// MySQL 握手包包含认证插件名,nmap 随机 salt 可能导致误匹配
if strings.Contains(banner, "mysql_native_password") || strings.Contains(banner, "caching_sha2_password") {
info.Name = "mysql"
}
}
// processServiceResult 处理服务识别结果
func processServiceResult(host string, port int, addr string, serviceInfo *ServiceInfo, config *common.Config) {
func processServiceResult(ctx context.Context, host string, port int, addr string, serviceInfo *ServiceInfo, config *common.Config, session *common.ScanSession) {
if serviceInfo == nil {
// 服务识别失败,尝试 HTTP 回退探测
if !tryHTTPFallbackDetection(host, port, addr, config) {
common.LogInfo(i18n.Tr("port_open", addr))
if !tryHTTPFallbackDetection(ctx, host, port, addr, config, session) {
session.LogInfo(i18n.Tr("port_open", addr))
}
return
}
// Banner 校正:nmap 指纹库可能将 MySQL 握手包的随机 salt 误匹配为其他服务
correctServiceByBanner(serviceInfo)
// 缓存指纹识别结果,供插件按服务类型匹配(解决非标准端口问题)
CacheServiceInfo(host, port, serviceInfo)
// 保存并输出服务信息
details := buildServiceDetails(port, serviceInfo)
isWeb := IsWebServiceByFingerprint(serviceInfo)
if isWeb {
details["is_web"] = true
MarkAsWebService(host, port, serviceInfo)
// 指纹既不匹配 webKeywords 也不匹配 nonWebKeywords(不确定区间)
// 补做一次 HTTP 探测,覆盖自定义 HTTP 框架等漏网场景
if !isWeb && !isDefinitelyNonWeb(serviceInfo) {
if tryHTTPFallbackDetection(ctx, host, port, addr, config, session) {
isWeb = true
}
}
_ = common.SaveResult(&output.ScanResult{
if isWeb {
details["is_web"] = true
}
_ = session.SaveResult(&output.ScanResult{
Time: time.Now(),
Type: output.TypeService,
Target: fmt.Sprintf("%s:%d", host, port),
Target: net.JoinHostPort(host, strconv.Itoa(port)),
Status: "identified",
Details: details,
})
common.LogInfo(buildServiceLogMessage(addr, serviceInfo, isWeb))
session.LogInfo(buildServiceLogMessage(addr, serviceInfo, isWeb))
}
// buildServiceDetails 构建服务详情 map
@@ -613,10 +789,10 @@ func buildServiceDetails(port int, info *ServiceInfo) map[string]interface{} {
}
// tryHTTPFallbackDetection 尝试HTTP回退探测,返回是否成功识别为HTTP服务
func tryHTTPFallbackDetection(host string, port int, addr string, config *common.Config) bool {
func tryHTTPFallbackDetection(ctx context.Context, host string, port int, addr string, config *common.Config, session *common.ScanSession) bool {
// 使用WebDetection进行HTTP协议探测
webDetector := GetWebPortDetector()
if !webDetector.DetectHTTPServiceOnly(host, port, config) {
if !webDetector.DetectHTTPServiceOnlyContext(ctx, host, port, config, session) {
return false
}
@@ -636,14 +812,150 @@ func tryHTTPFallbackDetection(host string, port int, addr string, config *common
"is_web": true,
"detected_by": "http_probe",
}
_ = common.SaveResult(&output.ScanResult{
_ = session.SaveResult(&output.ScanResult{
Time: time.Now(),
Type: output.TypeService,
Target: fmt.Sprintf("%s:%d", host, port),
Target: net.JoinHostPort(host, strconv.Itoa(port)),
Status: "identified",
Details: details,
})
common.LogInfo(i18n.Tr("port_open_http", addr))
session.LogInfo(i18n.Tr("port_open_http", addr))
return true
}
// =============================================================================
// 网段预筛 — 大规模扫描时跳过空 /24 网段
// =============================================================================
// subnetProbeThreshold 触发网段预筛的主机数阈值(超过 1 个 /24)
const subnetProbeThreshold = 256
// subnetProbePorts 逐主机探活用的端口(轮换)
var subnetProbePorts = []int{80, 443, 22, 445, 3389, 8080, 3306, 6379}
// gatewayProbePorts 网关启发式探测端口(网关常开的服务)
var gatewayProbePorts = []int{22, 80, 443, 23, 8080, 161, 53, 3389}
// gatewayOffsets 网关候选地址偏移量
var gatewayOffsets = []string{".1", ".254"}
// subnetProbeTimeout 每个探测的超时
const subnetProbeTimeout = 1500 * time.Millisecond
// subnetProbeConcurrency 网段探活全局并发数
const subnetProbeConcurrency = 500
// probeSubnets 对每个 /24 网段做探活,返回属于存活网段的主机列表
// 两阶段策略:
//
// 阶段 1(快速):对每个子网的 .1/.254 网关做多端口探测,命中即标记存活
// 阶段 2(兜底):未命中的子网,逐主机单端口轮换扫描
func probeSubnets(ctx context.Context, hosts []string, timeout time.Duration, session *common.ScanSession) []string {
// 按 /24 分组
subnets := make(map[string][]string)
for _, h := range hosts {
prefix := subnetPrefix(h)
if prefix != "" {
subnets[prefix] = append(subnets[prefix], h)
}
}
if len(subnets) <= 1 {
return hosts
}
session.LogInfo(i18n.Tr("subnet_prefilter_start", len(subnets), len(hosts)))
aliveSubnets := sync.Map{}
var wg sync.WaitGroup
limiter := make(chan struct{}, subnetProbeConcurrency)
// ── 阶段 1:网关启发式 ──────────────────────────────────
// 对每个子网的 .1 和 .254 打多个端口,命中率高且速度极快
for prefix := range subnets {
for _, suffix := range gatewayOffsets {
gw := prefix + suffix
for _, port := range gatewayProbePorts {
wg.Add(1)
limiter <- struct{}{}
go func(pfx, addr string) {
defer func() { <-limiter; wg.Done() }()
conn, err := session.DialTCP(ctx, "tcp", addr, subnetProbeTimeout)
if err == nil {
_ = conn.Close()
aliveSubnets.Store(pfx, true)
}
}(prefix, net.JoinHostPort(gw, strconv.Itoa(port)))
}
}
}
wg.Wait()
// 统计阶段 1 命中
gwHits := 0
aliveSubnets.Range(func(_, _ interface{}) bool { gwHits++; return true })
// ── 阶段 2:逐主机兜底(仅对网关未命中的子网)──────────
for prefix, subnetHosts := range subnets {
if _, alive := aliveSubnets.Load(prefix); alive {
continue // 网关已命中,跳过
}
for i, host := range subnetHosts {
select {
case <-ctx.Done():
goto done
default:
}
if _, alive := aliveSubnets.Load(prefix); alive {
break
}
port := subnetProbePorts[i%len(subnetProbePorts)]
wg.Add(1)
limiter <- struct{}{}
go func(pfx, h string, p int) {
defer func() { <-limiter; wg.Done() }()
conn, err := session.DialTCP(ctx, "tcp", net.JoinHostPort(h, strconv.Itoa(p)), subnetProbeTimeout)
if err == nil {
_ = conn.Close()
aliveSubnets.Store(pfx, true)
}
}(prefix, host, port)
}
}
done:
wg.Wait()
// 统计
aliveCount := 0
aliveSubnets.Range(func(_, _ interface{}) bool { aliveCount++; return true })
if aliveCount == 0 {
return nil
}
result := make([]string, 0, len(hosts))
for _, h := range hosts {
if _, alive := aliveSubnets.Load(subnetPrefix(h)); alive {
result = append(result, h)
}
}
skipped := len(subnets) - aliveCount
session.LogInfo(i18n.Tr("subnet_prefilter_done", aliveCount, gwHits, skipped, len(result)))
return result
}
// subnetPrefix 提取 IP 的 /24 前缀(如 "10.1.1"
func subnetPrefix(ip string) string {
lastDot := strings.LastIndex(ip, ".")
if lastDot <= 0 {
return ""
}
return ip[:lastDot]
}
+185 -2
View File
@@ -2,6 +2,8 @@ package core
import (
"fmt"
"sort"
"strings"
"testing"
)
@@ -212,6 +214,152 @@ func TestFormatAddress(t *testing.T) {
}
}
func TestBuildWebServiceURLIPv6(t *testing.T) {
tests := []struct {
name string
addr string
serviceInfo *ServiceInfo
want string
}{
{
name: "http default port",
addr: "[2001:db8::1]:80",
serviceInfo: &ServiceInfo{
Name: "http",
},
want: "http://[2001:db8::1]",
},
{
name: "https default port",
addr: "[2001:db8::1]:443",
serviceInfo: &ServiceInfo{
Name: "https",
},
want: "https://[2001:db8::1]",
},
{
name: "http non-default port",
addr: "[2001:db8::1]:8080",
serviceInfo: &ServiceInfo{
Name: "http",
},
want: "http://[2001:db8::1]:8080",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := buildWebServiceURL(tt.addr, tt.serviceInfo); got != tt.want {
t.Fatalf("buildWebServiceURL(%q) = %q, want %q", tt.addr, got, tt.want)
}
})
}
}
func TestPortScanCollectorsAndHelpers(t *testing.T) {
t.Run("result collector deduplicates and streams", func(t *testing.T) {
stream := make(chan string, 2)
collector := newResultCollector(stream)
collector.Add("127.0.0.1:80")
collector.Add("127.0.0.1:80")
collector.Add("127.0.0.1:443")
got := collector.GetAll()
sort.Strings(got)
expected := []string{"127.0.0.1:443", "127.0.0.1:80"}
if !stringSlicesEqual(got, expected) {
t.Fatalf("collector results = %v, want %v", got, expected)
}
close(stream)
var streamed []string
for addr := range stream {
streamed = append(streamed, addr)
}
sort.Strings(streamed)
if !stringSlicesEqual(streamed, expected) {
t.Fatalf("streamed results = %v, want %v", streamed, expected)
}
})
t.Run("failed collector counts", func(t *testing.T) {
var collector failedPortCollector
collector.Add("127.0.0.1", 80, "127.0.0.1:80")
collector.Add("127.0.0.1", 443, "127.0.0.1:443")
if got := collector.Count(); got != 2 {
t.Fatalf("failed count = %d, want 2", got)
}
})
t.Run("proxy and closed error helpers", func(t *testing.T) {
if !isProxyErrorResponse([]byte{0x05, 0x01, 0x00, 0x01}) {
t.Fatal("SOCKS5 failure reply should be proxy error")
}
if !isProxyErrorResponse([]byte("HTTP/1.1 502 Bad Gateway\r\n\r\n")) {
t.Fatal("HTTP proxy error text should be detected")
}
if isProxyErrorResponse(nil) || isProxyErrorResponse([]byte{0x05, 0x00}) {
t.Fatal("empty or success response should not be proxy error")
}
if !isConnectionClosed(fmt.Errorf("use of closed network connection")) {
t.Fatal("closed connection error should be detected")
}
if isConnectionClosed(nil) || isConnectionClosed(fmt.Errorf("temporary timeout")) {
t.Fatal("non-closed error should not be detected")
}
})
t.Run("service details and subnet prefix", func(t *testing.T) {
details := buildServiceDetails(8443, &ServiceInfo{
Name: "https",
Version: "1.2.3",
Banner: " hello \r\n",
Extras: map[string]string{
"vendor_product": "nginx",
"os": "linux",
"info": "tls",
"empty": "",
"ignored": "value",
},
})
expected := map[string]interface{}{
"port": 8443,
"service": "https",
"version": "1.2.3",
"banner": "hello",
"product": "nginx",
"os": "linux",
"info": "tls",
}
for key, want := range expected {
if got := details[key]; got != want {
t.Fatalf("details[%s] = %#v, want %#v (all=%#v)", key, got, want, details)
}
}
if _, ok := details["ignored"]; ok {
t.Fatalf("unexpected ignored extra in details: %#v", details)
}
if got := subnetPrefix("192.168.1.25"); got != "192.168.1" {
t.Fatalf("subnetPrefix IPv4 = %q", got)
}
if got := subnetPrefix("localhost"); got != "" {
t.Fatalf("subnetPrefix hostname = %q, want empty", got)
}
})
}
func stringSlicesEqual(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
// =============================================================================
// 排除端口逻辑测试(从EnhancedPortScan:28-32行提取)
// =============================================================================
@@ -478,7 +626,31 @@ func TestBuildServiceLogMessage(t *testing.T) {
Extras: map[string]string{},
},
isWeb: true,
wantContain: []string{"192.168.1.1:80", "http", "1.1"},
wantContain: []string{"http://192.168.1.1", "http", "1.1"},
},
{
name: "非标准端口HTTP服务显示URL",
addr: "192.168.1.1:8080",
serviceInfo: &ServiceInfo{
Name: "http",
Version: "1.1",
Banner: "",
Extras: map[string]string{},
},
isWeb: true,
wantContain: []string{"http://192.168.1.1:8080", "http", "1.1"},
},
{
name: "HTTPS服务显示HTTPS URL",
addr: "192.168.1.1:443",
serviceInfo: &ServiceInfo{
Name: "https",
Version: "1.1",
Banner: "",
Extras: map[string]string{},
},
isWeb: true,
wantContain: []string{"https://192.168.1.1", "https", "1.1"},
},
{
name: "带Banner的SSH服务",
@@ -548,6 +720,17 @@ func TestBuildServiceLogMessage(t *testing.T) {
}
}
func TestBuildServiceLogMessageTruncatesBannerByRune(t *testing.T) {
result := buildServiceLogMessage("10.0.0.1:22", &ServiceInfo{
Name: "ssh",
Banner: strings.Repeat("界", 85),
Extras: map[string]string{},
}, false)
if !strings.Contains(result, strings.Repeat("界", 80)+"...") {
t.Fatalf("truncated banner is not rune-safe: %q", result)
}
}
// contains 检查字符串是否包含子串
func contains(s, substr string) bool {
return len(s) >= len(substr) && (s == substr || len(substr) == 0 ||
@@ -662,7 +845,7 @@ func TestIsResourceExhaustedError_EdgeCases(t *testing.T) {
{
name: "大小写混合",
err: fmt.Errorf("Too Many Open Files"),
expected: false, // 当前实现区分大小写
expected: true, // containsFold 不区分大小写
},
{
name: "错误信息包含但不完全匹配",
+4 -2
View File
@@ -4,6 +4,8 @@ import (
"fmt"
"regexp"
"strings"
"github.com/shadow1ng/fscan/common/i18n"
)
// BytesToRegexSafeString 将字节切片转换为 Go regexp 安全的正则表达式模式字符串
@@ -13,7 +15,7 @@ func BytesToRegexSafeString(b []byte) string {
for _, c := range b {
if c < 32 || c >= 128 {
// 控制字符和高位字节转换为 \x{NN} 格式
result.WriteString(fmt.Sprintf("\\x{%02x}", c))
fmt.Fprintf(&result, "\\x{%02x}", c)
} else {
result.WriteByte(c)
}
@@ -43,7 +45,7 @@ func (p *Probe) parseMatchDirective(data, prefix string, isSoft bool) (Match, er
// 分割文本获取pattern和版本信息
textSplited := strings.Split(directive.DirectiveStr, directive.Delimiter)
if len(textSplited) == 0 {
return match, fmt.Errorf("无效的%s指令格式", prefix)
return match, fmt.Errorf("%s", i18n.Tr("portfinger_match_directive_invalid", prefix))
}
pattern := textSplited[0]

Some files were not shown because too many files have changed in this diff Show More