170 Commits
Author SHA1 Message Date
ZacharyZcR 95cc12e753 Merge pull request #604 from shadow1ng/dev
发布 / auto-tag (push) Canceled after 0s
测试构建 / 代码检查 (push) Canceled after 0s
发布 / release (push) Canceled after 0s
测试构建 / 单元测试和构建 (push) Canceled after 0s
测试构建 / 构建验证 (push) Canceled after 0s
release: v2.2.1
2026-08-26 04:43:42 +08:00
ZacharyZcR 75f4265098 Merge remote-tracking branch 'origin/main' into dev
测试构建 / 代码检查 (push) Canceled after 0s
测试构建 / 单元测试和构建 (push) Canceled after 0s
测试构建 / 构建验证 (push) Canceled after 0s
2026-08-26 04:34:51 +08:00
ZacharyZcR 5bda99528b docs: add v2.2.1 release notes 2026-08-26 04:31:59 +08:00
ZacharyZcR a1ff55ef55 fix: resolve recent service scan regressions 2026-08-26 04:19:58 +08:00
ZacharyZcR 1418f6d8ce Disable default global scan timeout 2026-08-25 23:50:30 +08:00
ZacharyZcR 3ef7a1beee feat: expand internal network poc coverage
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-07-16 02:04:36 +08:00
ZacharyZcR 621b2c2f24 feat: add curated internal network pocs 2026-07-16 01:45:09 +08:00
ZacharyZcR 61ae87d171 chore: bump dev version to 2.2.1 2026-07-16 01:08:05 +08:00
逸航 9d0010927e fix: 修复高并发下自适应超时过低导致开放端口漏扫 (#598)
* fix: 修复高并发下自适应超时过低导致开放端口漏扫 (#503)

扫描本机/低 RTT 目标时,AdaptiveTimeout 在 10 次采样后迅速收敛到 100ms 下限。高并发(600+ 线程)下 TCP 握手尾延迟可能超过 100ms,加上超时错误不会重试,导致开放端口被误判为关闭。

- AdaptiveTimeout 下限从 100ms 提升至 max(500ms, maxTimeout/5)
- connectWithRetry 对超时错误用完整超时重试一次
- slidingWindowSchedule 任务丢弃时记录日志,便于排查漏扫

* refactor: 按 review 意见移除无条件超时重试,补充 minTO 下限测试

根据 #598 review 反馈:

1. 移除 connectWithRetry 中 timeout->full maxTO 无条件重试
   - filtered/无响应端口占超时大头,盲目重试只烧时间
   - #503 主场景靠 minTO 抬升已足够覆盖
2. 移除不再使用的 MaxTimeout() 方法和 port_scan_timeout_retry i18n 条目
3. AdaptiveTimeout 收敛测试补充 minTO 下限断言(3s->600ms)
2026-07-16 01:07:27 +08:00
ZacharyZcR bf036fd9b2 Merge pull request #594 from shadow1ng/dev
发布 / auto-tag (push) Has been cancelled
测试构建 / 代码检查 (push) Has been cancelled
发布 / release (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
Release v2.2.0
2026-07-10 13:57:26 +08:00
ZacharyZcR fdf836f003 chore: prepare v2.2.0 release
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-07-09 20:41:10 +08:00
ZacharyZcR 075bf646dc fix: 全局超时改为动态估算,替代硬编码阈值表
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
根据 hostCount × portCount / threads 计算端口扫描耗时,
结合开放率估算插件扫描耗时,加 20% 余量,上限 2h。
新增 EstimateHostCount 快速统计 CIDR/range/文件中的主机数。
2026-06-27 16:50:53 +08:00
ZacharyZcR 1980504007 fix: 大规模扫描全局超时过短导致提前终止 (#588)
4 万 IP 全端口扫描默认 -gt 180s 完全不够用,3 分钟后报
"解析目标失败: context deadline exceeded" 误导用户。

1. 自适应全局超时:用户未显式指定 -gt 时,根据端口数和是否有
   hosts 文件自动调大超时(最高 24h),并输出调整日志
2. 修正超时错误信息:context deadline exceeded 不再包装为
   "解析目标失败",改为提示用户调大 -gt 或设为 0 禁用
2026-06-27 16:42:32 +08:00
ZacharyZcR a3ccc2827b fix: Telnet 弱口令误报,Cisco MOTD 横幅触发 shell prompt 误判 (Closes #590)
isShellPrompt 使用 Contains 匹配 # $ > 单字符,Cisco IOS MOTD 横幅中
的装饰线(###)和文本内容会误触发,导致未发送凭据就判定认证成功。

重写 isShellPrompt 改为行尾匹配,排除全同字符装饰线;
performTelnetAuth 等待 login prompt 阶段移除 isShellPrompt 检查,
未授权检测由 testUnauthAccess 专门负责。
2026-06-27 16:35:18 +08:00
ZacharyZcR ed45d0ead5 fix: 结果文件中 POC 漏洞只显示 vulnerable 不显示漏洞名称 (Closes #591)
POC 扫描结果存入 details["vulnerability_name"],
但 TXT/CSV/NDJSON 三种输出格式只读 details["vulnerability"],
key 不匹配导致漏洞名丢失,退化为显示 status 字段 "vulnerable"。
三种 writer 统一兼容两种 key。
2026-06-27 16:23:52 +08:00
ZacharyZcR 4922122530 fix: 修复 #591 POC 对 HTTPS 端口误用 HTTP + #592 空指针 panic
1. buildTargetURL 对 443/8443 等已知 TLS 端口默认使用 https scheme,
   webtitle 触发 POC 扫描前将检测到的协议写回 info.URL,
   避免对 HTTPS 服务发送 HTTP 请求导致 EOF
2. GetInfo 添加 probe nil 检查,防止探针初始化失败时空指针 panic
3. 删除 test-nuclei-example.yaml 测试模板,避免 robots.txt 误报
2026-06-27 16:23:52 +08:00
ZacharyZcR 4976cb1f6b feat: 添加 -nsp 参数禁用网段预筛
大规模扫描时 probeSubnets 会自动跳过空 /24 网段,
部分场景下用户需要关闭此优化以扫描全部目标。
新增 -nsp (no subnet probe) 参数控制。
2026-06-27 16:23:51 +08:00
ZacharyZcR ed2f947722 fix: POC 扫描遇到非 HTTP 服务时不再输出错误日志
扫描非 HTTP 端口时 Go net/http 返回 malformed HTTP status code 等
transport 级错误,属于正常现象,不应作为 error 输出。
在 executeRule 和 clustersend 两个调用点统一过滤 transport 错误,
返回 false, nil 表示"目标不可达 = 无漏洞"。
2026-06-27 15:36:42 +08:00
ZacharyZcR 34638954b8 fix: 消除 state_test.go SA2001 lint 警告
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-17 12:51:43 +08:00
ZacharyZcR 65e64e8967 fix: Oracle TNS Resend 重试 + ANO 格式修正,扩展集成测试至 22 协议
Oracle raw TNS 修复:
- connect 阶段支持 Resend 包重试(Oracle 18c+ 需要)
- ANO 请求补齐加密/完整性算法列表和 auth UB2 字段
- ANO length 字段修正为包含 magic 的完整长度
- Oracle 18c ANO 仍不兼容(字节级匹配 go-ora 但被拒绝),爆破标记 SKIP

新增集成测试协议:
ActiveMQ, Zookeeper, Rsync, VNC, SNMP, Oracle(服务检测), Cassandra, Neo4j, Kafka, SMTP, LDAP

VNC 修复:换用支持 RFB 3.8 的 debian-xfce-vnc 镜像
2026-06-17 12:51:43 +08:00
ZacharyZcR 1c6f3b80d0 fix: Cassandra CQL 协议头缺少 flags 字节 + version 方向位错误
cqlSend 写 8 字节头(缺 flags),实际 CQL v4 需要 9 字节。
version byte 0x84 是 response 方向,request 应为 0x04。

同时扩展集成测试至 17 个协议:新增 Memcached、Elasticsearch、
MSSQL、RabbitMQ、MQTT、LDAP、Cassandra、Neo4j、Kafka、SMTP。
2026-06-17 12:51:42 +08:00
ZacharyZcR 9b8e4f3f3b fix: MongoDB SCRAM 认证因 BSON 键序随机而失败
Go map 遍历顺序不确定,导致 buildBSON 输出的命令文档中
saslStart/saslContinue 不一定是第一个键,MongoDB 拒绝执行。

引入有序 []mongoKV 类型,SASL 命令改用 orderedDoc() 构造。
同时新增 6 协议集成测试框架(Docker Compose + go test -tags integration)。
2026-06-17 12:51:42 +08:00
ZacharyZcR 0612255893 test: 补充单元测试覆盖率 29.9% → 36.6%
新建 18 个测试文件,追加 30 个已有测试文件,覆盖协议解析、
错误分类、CEL 表达式求值、YAML 反序列化、字节编码等纯函数。
2026-06-17 12:51:41 +08:00
ZacharyZcR d7dbccab76 merge dev into main for v2.2.0-rc.1 re-release
发布 / release (push) Has been cancelled
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-15 12:46:01 +08:00
ZacharyZcR 6eff1d5ccf fix: 外部审查 8 项修复 + 国密 TLS 按需回退
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- UserAgent 默认值回退 + 注册 -ua flag (#2)
- README 编译命令 main.go → . (#3)
- README 版本号同步 rc.1 (#4)
- Client.go gmtls stdout 劫持删除 (#5)
- ms17010 smb1GetResponse size<32 越界 panic (#6)
- SSH 拨号超时统一 ModuleTimeout (#8)
- AddPorts 死字段删除 (#9)
- 国密 TLS 按需回退:标准 TLS 握手失败时仅在错误为
  cipher/protocol 不兼容时尝试国密,跳过超时/拒绝等连接级错误
2026-06-15 04:46:25 +08:00
ZacharyZcR 2f7d2d49c6 fix: redis exploit 超时改用配置值 & 清理死代码
- redis exploit 硬编码 30s deadline 改为 config.ModuleTimeout(),与同文件其他超时一致
- 删除 BaseScanStrategy.LogPluginInfo 残留死代码(全是空操作)
- .gitignore 补充 fscan_cli/fscan_web/embed-agent 构建产物
2026-06-14 23:58:56 +08:00
ZacharyZcR 35f3cf1960 docs: release notes 移除实测验证章节
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-14 23:01:39 +08:00
ZacharyZcR 065ba6fae7 docs: 更新 v2.2.0-rc.1 release notes,覆盖全部 29 个 commit 的变更 2026-06-14 23:00:26 +08:00
ZacharyZcR 6d61b661f4 fix: 修复实机测试发现的可靠性问题 (v2.2.0-rc.1)
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- UDP 插件在 -p 指定端口时被跳过
- Redis exploit 无超时保护 / readReply 吞没非超时错误
- service_probe 连接丢失后静默成功
- SNMP 探测成功但终端无输出
- SSH 爆破不稳定 (并发过高 + 自适应超时过短 + 限流误判)
- 进度条 isActive 竞态

新增 Config.ModuleTimeout() 协议级超时下限 (≥3s)
新增 ErrorTypeThrottle 限流错误分类
2026-06-14 22:23:52 +08:00
ZacharyZcR 3babff6863 fix: -hash 支持 LM:NT 格式 & -debug 日志文件修复
1. -hash 支持标准的 LM:NT 格式 (如 aad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0)
   之前只接受纯 32 字符 NTLM hash,LM:NT 格式报 invalid hash length

2. -debug 日志文件写入修复(已在上一个 commit 中)
2026-06-14 22:23:51 +08:00
ZacharyZcR d7071b7b8e fix: -debug 日志文件写入失败,applyLogLevel 重建 Logger 时丢失 DebugLogFile 配置 2026-06-14 22:23:51 +08:00
ZacharyZcR 6ae37b8892 fix: -nopoc 禁用POC时不再输出错误日志 2026-06-14 22:23:51 +08:00
ZacharyZcR 2b202aa298 fix: 让 -gt 全局超时参数真正生效
-gt 参数之前是死代码:flag 定义了 GlobalTimeout 但从未被使用。
现在在 RunScan 中用它创建带 deadline 的 context,
超时后所有扫描任务(端口扫描、插件执行)被取消。
2026-06-14 22:23:51 +08:00
ZacharyZcR 6d7e6cd394 fix: 修复3个实测输出问题
1. 静默模式NDJSON banner截断至200字符
   Redis INFO响应~5KB导致JSONL单行过长,截断后加...后缀

2. CSV漏洞Type列补全
   ResultTypeVuln的fillDetail未设type字段,CSV Vulns列为空
   统一设为"vulnerability"

3. ICMP权限不足告警精简
   4行告警(listen失败/连接失败/权限不足/切换ping)合并为1行
2026-06-14 22:23:51 +08:00
ZacharyZcR 626d8f79bb fix: 修复3个实测发现的问题
1. -pwd 支持逗号分隔多个密码
   之前 -pwd "123,456,root" 被当作单个密码,SSH root:123 无法匹配
   现在逗号分隔为独立密码,空格保留(可能是密码的一部分)

2. -nobr 禁用爆破时仍检测 Redis 未授权访问
   未授权访问是服务探测不是爆破,不应被 -nobr 跳过
   将未授权检测移到 DisableBrute 判断之前

3. 指定端口时跳过 UDP 插件调度
   -p 80 只扫 HTTP 时不需要 SNMP/BACnet/DNS 等 UDP 探测
   仅在默认端口扫描时才分发 UDP 插件
   效果: -p 80 从 9 秒降到 3 秒
2026-06-14 22:23:51 +08:00
ZacharyZcR a52e93e84c fix: 非终端输出时禁用进度条,防止ANSI控制码覆盖扫描结果 2026-06-14 22:23:50 +08:00
ZacharyZcR 04cae2e42d fix: 彻底解决UDP插件阻塞导致扫描无法结束的问题
根因分析(通过 goroutine dump 定位):
1. UDP conn.Write 在 WSL2 上可能永久阻塞(SetDeadline 对 Write 不生效)
2. SNMP community 爆破混入通用密码字典(57个),串行 × 10s超时 = 10分钟

修复:
- 提取 udpProbe() 公共函数,用 context timeout + conn.Close 双保险
  超时后强制关闭连接,中断阻塞的 Write/Read
- BACnet/DNS/IPMI/TFTP 统一使用 udpProbe()
- SNMP probe 使用 goroutine + context select 保护
- SNMP community 列表不再混入通用密码字典(8个专用 community 足够)
- SNMP 后续 community 爆破用 3s 短超时 + 连续失败 3 次快速退出

效果: 同样的扫描从无限卡死 → 9秒完成
2026-06-14 22:23:50 +08:00
ZacharyZcR 4169eb6ee0 fix: UDP插件改用 conn.SetDeadline 替代类型断言 2026-06-14 22:23:50 +08:00
ZacharyZcR 63631d6cdf fix: 所有UDP插件添加ReadDeadline防止无限阻塞
SNMP/BACnet/DNS/IPMI/TFTP 的 conn.Read() 在目标不响应时
无限阻塞(goroutine 泄漏),导致整个扫描无法结束。

在 Write 前设置 ReadDeadline 确保超时后返回。
2026-06-14 22:23:50 +08:00
ZacharyZcR 4f6bb28138 fix: 修复6个运行时问题
1. 抑制 gmtls 库的 handshake error stdout 噪声
   gmtls/conn.go:1304 硬编码了 fmt.Println,在调用时临时重定向 os.Stdout

2. MySQL 3306 服务名误识别为 genetec-5400
   nmap 指纹库将 MySQL 握手包的随机 salt 误匹配,通过 banner 特征校正

3. 管道输出时自动禁用 ANSI 控制码
   检测 stdout 是否为终端,非终端时自动启用 NoColor

4. 进度条完成消息措辞精确化
   去掉冗余冒号,保持信息简洁一致

5. URL 模式跳过不必要的 TLS 探测
   用户已通过 -u 显式指定 http:// 协议时直接使用,不再做 TLS 握手

6. 无网络探测数据时降低默认重试次数
   -np 跳过存活探测后,将默认重试从 3 降到 2,加速不可达主机的超时
2026-06-14 22:23:49 +08:00
ZacharyZcR 46a6d812a4 fix: DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式
有 transport 字段但 rules 是数组的 POC(如 apache-httpd-cve-2021-40438)
属于 fscan 格式,不应被 xray 分支兜底。移除错误的 fallback return,
让这类 POC 正确落入 fscan 格式检测分支。

修复前: 388个POC成功380个,失败8个
修复后: 388个POC成功388个,失败0个
2026-06-14 22:23:49 +08:00
ZacharyZcR d4f4e65dec refactor: 4项架构优化 — CEL缓存/POC隔离/服务缓存/结果统一
1. CEL 表达式编译缓存
   - 新增 CelProgCache,同一 POC 的所有规则/参数组合共享编译后的 Program
   - clusterpoc 热路径上消除重复的 Compile+Program 调用

2. POC 全局状态消除
   - allPocs/pocLoaded 全局变量改为 pocStore 按 PocPath 缓存
   - 不同 PocPath 的扫描独立加载,Web API 并发场景不再互相覆盖

3. serviceCache 下沉到 per-session State
   - 服务识别缓存从包级全局 map 迁移到 State.serviceCache (sync.Map)
   - BaseScanStrategy 通过 SetState 注入 session state
   - 消除多个并发扫描之间的服务识别缓存串台

4. POC 结果输出路径统一
   - 提取 buildVulnDetails/buildVulnLogMsg/saveVulnResult 三个公共函数
   - CheckMultiPoc 和 recordVulnerabilityResult 共用统一的结果构造逻辑
   - 消除 details 字段名不一致和日志格式差异
2026-06-14 22:23:49 +08:00
ZacharyZcR a115499793 fix+perf: 修复10个bug & 10项性能优化
Bug修复:
- clustersend CEL结果判断从字符串比较改为类型断言
- Nuclei DSL matcher安全降级为false避免误报
- clusterpoc发现漏洞后返回true修正语义
- reverseCheck加10s超时防止ceye API阻塞
- doSearch/bmatches正则编译结果缓存到sync.Map
- evalset CEL求值失败时存空字符串而非原始表达式
- CEL wait()函数加nil Reverse指针检查防panic
- MongoDB readMongoMsg应用timeout参数设置读超时
- TXTWriter.Close确保Sync失败后仍调用file.Close

性能优化:
- 指纹regex缓存从RWMutex+map改为sync.Map消除锁竞争
- CaseInsensitive指纹词加载时预小写化避免匹配时分配
- 版本提取FindAllStringSubmatch限制返回数量
- i18n.Tr用strconv.Itoa替代Sprintf减少分配
- POC加载用atomic.Bool+DCLP消除热路径锁
- 结果缓冲map预分配容量减少rehash
- HTTP连接池参数随并发数动态调整
- getRuleHash去除反射+Headers排序保证确定性dedup
- POC并发加载用channel替代Mutex收集结果
2026-06-14 22:23:49 +08:00
ZacharyZcR 8402be98e3 优化自适应扫描系统 & 修复 POC 调度问题
自适应扫描优化:
- target/ceiling 分离,自适应池可向上探索而非锁死在 target
- assessHealth 阈值按网络环境区分(LAN 收紧 / Internet 放宽)
- RTT 漂移时动态压低 target,配合 AIMD 双重降速
- 去掉 semaphore 双层流控,由 ants pool 统一反压
- 探测端口从 3 个扩充到 8 个,减少 RTT 采样偏差
- computeRetries 按环境调整目标概率和上限

Bug 修复:
- AdaptivePool.Wait() 加 10 分钟超时,防止 goroutine 卡死时永久挂起
- CEL 环境初始化失败后允许重试(sync.Once → sync.Mutex + 标志位)
- CAS 自旋加 runtime.Gosched() 退避,减少高并发下 CPU 空转
- -full 模式下 web 插件跳过 IsMarkedWebService 检查 #588
- 不确定服务补做 HTTP 回退探测,覆盖自定义框架漏网场景
- POC sets 纯字面量值跳过 CEL 编译,消除大量误报错误日志
2026-06-14 22:23:48 +08:00
ZacharyZcR c49c23c7f0 Harden scan robustness and tests 2026-06-14 22:23:48 +08:00
ZacharyZcR 5ad914a1bb feat: 统一服务缓存 + 指纹驱动插件匹配
将 webServiceCache 扩展为通用 serviceCache,所有指纹识别结果
统一缓存,插件匹配时端口不命中则回退到服务名称匹配。

删除多余的 service_cache.go,复用已有的 ServiceInfo 体系。
补充 nil 防御、Explicit 标记、大量单元/集成/回归测试。
2026-06-14 22:23:47 +08:00
ZacharyZcR 2ab7c4d9b2 fix: 非标准端口的服务无法匹配对应插件 #588
端口扫描识别到 8881 上运行 SSH,但 SSH 插件只注册了 [22,2222,2200,22222],
端口不匹配导致插件不执行。

新增服务名称缓存:端口扫描阶段记录 host:port → serviceName,
插件匹配时端口不命中则回退到服务名称匹配。
2026-06-14 22:23:46 +08:00
ZacharyZcR 5c251b123d fix: 移除误导性的"无可用插件"日志 #588
扫描开始前的插件预检基于端口列表静态匹配,不代表实际扫描中插件
不会执行。移除"无可用插件"提示,避免用户误以为插件未工作。
2026-06-14 22:23:46 +08:00
ZacharyZcR 800cc30794 Fix credential cleanup and explicit tuning flags 2026-06-14 22:23:46 +08:00
ZacharyZcR 8e3cac303d fix: webtitle HTTP 请求失败时重试,修复批量扫描 POC 缺失 #587
批量扫描(-hf)时并发压力导致 HTTP 请求瞬时失败,getWebTitle
直接返回 error,跳过指纹识别和 POC 触发。

加入指数退避重试(200ms→400ms),最多 3 次,复用 config.MaxRetries。
2026-06-14 22:23:45 +08:00
ZacharyZcR 4b79cb7a18 ci: 升级 CI Go 版本到 1.25 2026-06-14 22:23:45 +08:00
ZacharyZcR 9d38874a03 fix: 降级 modernc.org/sqlite 到 v1.39.0 适配 CI 2026-06-14 22:23:45 +08:00
ZacharyZcR 88c7e4f2be feat: 自适应并发调度 — 网络探测 + AIMD + 参数智能推导
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据
推导 6 个关键参数,替代硬编码默认值:

- Timeout: median_RTT + 4σ(覆盖 99.9% 正常连接)
- ModuleThreadNum: target_concurrency / 30
- MaxRetries: ceil(log(0.01)/log(loss_rate))(全失败概率 <1%)
- ICMPRate: 环境基准 × fd 系数
- PocNum: 跟随 ModuleThreadNum
- DisablePing: 已有 ICMP 权限降级机制

线程池从单信号(资源耗尽率)升级为 AIMD + 慢启动:
- 慢启动:target/4 起步,500ms 翻倍
- 稳态 AIMD:健康 +5%,拥塞 ×0.5
- 双信号:资源耗尽率 + RTT 趋势(双 EMA)

用户 -t 显式指定时作为 ceiling,探测仍调整其他参数。

测试:单元 + 边界 + 集成 + 真实网络,core 包 580+ 用例全通过。
2026-06-14 22:23:45 +08:00
ZacharyZcR e0468ecd35 feat: Web 版独立入口 + SQLite 持久化存储
- 拆分 main.go 为 main_cli.go 和 main_web.go,Web 版不再包含 CLI 参数解析
- Web 版直接启动 HTTP 服务,通过 -port/-lang 控制,无需 -web flag
- 结果存储从内存 map 替换为 SQLite(modernc.org/sqlite,纯 Go 零 CGO)
- 数据库文件 ~/.fscan/results.db,进程重启后结果不丢失
- 修复结果分布面板跟随 tab 筛选联动的问题
2026-06-14 22:23:44 +08:00
ZacharyZcR d0295dcb92 fix: 修复 SSH 扫描 goroutine 泄漏
ssh.NewClientConn 不接受 context,context 取消后底层 TCP 连接未关闭,
导致 readLoop goroutine 永久阻塞在 conn.Read 上。大规模扫描时泄漏数万
goroutine。

- doSSHAuth 新增 goroutine 监听 context 取消并关闭底层连接
- TestSingleCredential 移除 5 秒超时放弃逻辑,改为持续等待清理
2026-06-14 22:23:44 +08:00
ZacharyZcR ade9cd1bff 修复默认扫描 POC 结果缺失 #586 2026-06-14 22:23:44 +08:00
ZacharyZcR 6d91b544de 显示 Web 服务识别 URL 2026-06-14 22:23:44 +08:00
ZacharyZcR 42092d8664 fix: 修复实机测试发现的可靠性问题 (v2.2.0-rc.1)
- UDP 插件在 -p 指定端口时被跳过
- Redis exploit 无超时保护 / readReply 吞没非超时错误
- service_probe 连接丢失后静默成功
- SNMP 探测成功但终端无输出
- SSH 爆破不稳定 (并发过高 + 自适应超时过短 + 限流误判)
- 进度条 isActive 竞态

新增 Config.ModuleTimeout() 协议级超时下限 (≥3s)
新增 ErrorTypeThrottle 限流错误分类
2026-06-14 22:14:02 +08:00
ZacharyZcR bc46e90d89 fix: -hash 支持 LM:NT 格式 & -debug 日志文件修复
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
1. -hash 支持标准的 LM:NT 格式 (如 aad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0)
   之前只接受纯 32 字符 NTLM hash,LM:NT 格式报 invalid hash length

2. -debug 日志文件写入修复(已在上一个 commit 中)
2026-06-14 09:50:10 +08:00
ZacharyZcR 67f2251da3 fix: -debug 日志文件写入失败,applyLogLevel 重建 Logger 时丢失 DebugLogFile 配置 2026-06-14 09:48:03 +08:00
ZacharyZcR e6c5e5a9a8 fix: -nopoc 禁用POC时不再输出错误日志
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-13 23:09:47 +08:00
ZacharyZcR 0356485595 fix: 让 -gt 全局超时参数真正生效
-gt 参数之前是死代码:flag 定义了 GlobalTimeout 但从未被使用。
现在在 RunScan 中用它创建带 deadline 的 context,
超时后所有扫描任务(端口扫描、插件执行)被取消。
2026-06-13 23:04:22 +08:00
ZacharyZcR b085df1878 fix: 修复3个实测输出问题
1. 静默模式NDJSON banner截断至200字符
   Redis INFO响应~5KB导致JSONL单行过长,截断后加...后缀

2. CSV漏洞Type列补全
   ResultTypeVuln的fillDetail未设type字段,CSV Vulns列为空
   统一设为"vulnerability"

3. ICMP权限不足告警精简
   4行告警(listen失败/连接失败/权限不足/切换ping)合并为1行
2026-06-13 22:55:02 +08:00
ZacharyZcR 8fb66e8e2f fix: 修复3个实测发现的问题
1. -pwd 支持逗号分隔多个密码
   之前 -pwd "123,456,root" 被当作单个密码,SSH root:123 无法匹配
   现在逗号分隔为独立密码,空格保留(可能是密码的一部分)

2. -nobr 禁用爆破时仍检测 Redis 未授权访问
   未授权访问是服务探测不是爆破,不应被 -nobr 跳过
   将未授权检测移到 DisableBrute 判断之前

3. 指定端口时跳过 UDP 插件调度
   -p 80 只扫 HTTP 时不需要 SNMP/BACnet/DNS 等 UDP 探测
   仅在默认端口扫描时才分发 UDP 插件
   效果: -p 80 从 9 秒降到 3 秒
2026-06-13 22:35:48 +08:00
ZacharyZcR 272b0e28c8 fix: 非终端输出时禁用进度条,防止ANSI控制码覆盖扫描结果 2026-06-13 22:21:03 +08:00
ZacharyZcR 28686f845d fix: 彻底解决UDP插件阻塞导致扫描无法结束的问题
根因分析(通过 goroutine dump 定位):
1. UDP conn.Write 在 WSL2 上可能永久阻塞(SetDeadline 对 Write 不生效)
2. SNMP community 爆破混入通用密码字典(57个),串行 × 10s超时 = 10分钟

修复:
- 提取 udpProbe() 公共函数,用 context timeout + conn.Close 双保险
  超时后强制关闭连接,中断阻塞的 Write/Read
- BACnet/DNS/IPMI/TFTP 统一使用 udpProbe()
- SNMP probe 使用 goroutine + context select 保护
- SNMP community 列表不再混入通用密码字典(8个专用 community 足够)
- SNMP 后续 community 爆破用 3s 短超时 + 连续失败 3 次快速退出

效果: 同样的扫描从无限卡死 → 9秒完成
2026-06-13 22:07:03 +08:00
ZacharyZcR 06ba595e32 fix: UDP插件改用 conn.SetDeadline 替代类型断言 2026-06-13 21:52:26 +08:00
ZacharyZcR 1a7770530d fix: 所有UDP插件添加ReadDeadline防止无限阻塞
SNMP/BACnet/DNS/IPMI/TFTP 的 conn.Read() 在目标不响应时
无限阻塞(goroutine 泄漏),导致整个扫描无法结束。

在 Write 前设置 ReadDeadline 确保超时后返回。
2026-06-13 21:42:49 +08:00
ZacharyZcR 2a9a3c36e2 fix: 修复6个运行时问题
1. 抑制 gmtls 库的 handshake error stdout 噪声
   gmtls/conn.go:1304 硬编码了 fmt.Println,在调用时临时重定向 os.Stdout

2. MySQL 3306 服务名误识别为 genetec-5400
   nmap 指纹库将 MySQL 握手包的随机 salt 误匹配,通过 banner 特征校正

3. 管道输出时自动禁用 ANSI 控制码
   检测 stdout 是否为终端,非终端时自动启用 NoColor

4. 进度条完成消息措辞精确化
   去掉冗余冒号,保持信息简洁一致

5. URL 模式跳过不必要的 TLS 探测
   用户已通过 -u 显式指定 http:// 协议时直接使用,不再做 TLS 握手

6. 无网络探测数据时降低默认重试次数
   -np 跳过存活探测后,将默认重试从 3 降到 2,加速不可达主机的超时
2026-06-13 19:53:28 +08:00
ZacharyZcR 70cce742e1 fix: DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式
有 transport 字段但 rules 是数组的 POC(如 apache-httpd-cve-2021-40438)
属于 fscan 格式,不应被 xray 分支兜底。移除错误的 fallback return,
让这类 POC 正确落入 fscan 格式检测分支。

修复前: 388个POC成功380个,失败8个
修复后: 388个POC成功388个,失败0个
2026-06-13 19:31:16 +08:00
ZacharyZcR 02ad8f5334 refactor: 4项架构优化 — CEL缓存/POC隔离/服务缓存/结果统一
1. CEL 表达式编译缓存
   - 新增 CelProgCache,同一 POC 的所有规则/参数组合共享编译后的 Program
   - clusterpoc 热路径上消除重复的 Compile+Program 调用

2. POC 全局状态消除
   - allPocs/pocLoaded 全局变量改为 pocStore 按 PocPath 缓存
   - 不同 PocPath 的扫描独立加载,Web API 并发场景不再互相覆盖

3. serviceCache 下沉到 per-session State
   - 服务识别缓存从包级全局 map 迁移到 State.serviceCache (sync.Map)
   - BaseScanStrategy 通过 SetState 注入 session state
   - 消除多个并发扫描之间的服务识别缓存串台

4. POC 结果输出路径统一
   - 提取 buildVulnDetails/buildVulnLogMsg/saveVulnResult 三个公共函数
   - CheckMultiPoc 和 recordVulnerabilityResult 共用统一的结果构造逻辑
   - 消除 details 字段名不一致和日志格式差异
2026-06-13 19:24:57 +08:00
ZacharyZcR 6a1636112f fix+perf: 修复10个bug & 10项性能优化
Bug修复:
- clustersend CEL结果判断从字符串比较改为类型断言
- Nuclei DSL matcher安全降级为false避免误报
- clusterpoc发现漏洞后返回true修正语义
- reverseCheck加10s超时防止ceye API阻塞
- doSearch/bmatches正则编译结果缓存到sync.Map
- evalset CEL求值失败时存空字符串而非原始表达式
- CEL wait()函数加nil Reverse指针检查防panic
- MongoDB readMongoMsg应用timeout参数设置读超时
- TXTWriter.Close确保Sync失败后仍调用file.Close

性能优化:
- 指纹regex缓存从RWMutex+map改为sync.Map消除锁竞争
- CaseInsensitive指纹词加载时预小写化避免匹配时分配
- 版本提取FindAllStringSubmatch限制返回数量
- i18n.Tr用strconv.Itoa替代Sprintf减少分配
- POC加载用atomic.Bool+DCLP消除热路径锁
- 结果缓冲map预分配容量减少rehash
- HTTP连接池参数随并发数动态调整
- getRuleHash去除反射+Headers排序保证确定性dedup
- POC并发加载用channel替代Mutex收集结果
2026-06-13 18:46:14 +08:00
ZacharyZcR 45ebe7040e 优化自适应扫描系统 & 修复 POC 调度问题
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
自适应扫描优化:
- target/ceiling 分离,自适应池可向上探索而非锁死在 target
- assessHealth 阈值按网络环境区分(LAN 收紧 / Internet 放宽)
- RTT 漂移时动态压低 target,配合 AIMD 双重降速
- 去掉 semaphore 双层流控,由 ants pool 统一反压
- 探测端口从 3 个扩充到 8 个,减少 RTT 采样偏差
- computeRetries 按环境调整目标概率和上限

Bug 修复:
- AdaptivePool.Wait() 加 10 分钟超时,防止 goroutine 卡死时永久挂起
- CEL 环境初始化失败后允许重试(sync.Once → sync.Mutex + 标志位)
- CAS 自旋加 runtime.Gosched() 退避,减少高并发下 CPU 空转
- -full 模式下 web 插件跳过 IsMarkedWebService 检查 #588
- 不确定服务补做 HTTP 回退探测,覆盖自定义框架漏网场景
- POC sets 纯字面量值跳过 CEL 编译,消除大量误报错误日志
2026-06-13 12:39:24 +08:00
ZacharyZcR 15a7670ba2 Harden scan robustness and tests 2026-06-13 07:55:37 +08:00
ZacharyZcR 1595c92aed feat: 统一服务缓存 + 指纹驱动插件匹配
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
将 webServiceCache 扩展为通用 serviceCache,所有指纹识别结果
统一缓存,插件匹配时端口不命中则回退到服务名称匹配。

删除多余的 service_cache.go,复用已有的 ServiceInfo 体系。
补充 nil 防御、Explicit 标记、大量单元/集成/回归测试。
2026-06-12 19:49:07 +08:00
ZacharyZcR 517133f72f fix: 非标准端口的服务无法匹配对应插件 #588
端口扫描识别到 8881 上运行 SSH,但 SSH 插件只注册了 [22,2222,2200,22222],
端口不匹配导致插件不执行。

新增服务名称缓存:端口扫描阶段记录 host:port → serviceName,
插件匹配时端口不命中则回退到服务名称匹配。
2026-06-12 19:31:56 +08:00
ZacharyZcR 0918eb38a6 fix: 移除误导性的"无可用插件"日志 #588
扫描开始前的插件预检基于端口列表静态匹配,不代表实际扫描中插件
不会执行。移除"无可用插件"提示,避免用户误以为插件未工作。
2026-06-12 16:36:27 +08:00
ZacharyZcR 5b7e72e56e Fix credential cleanup and explicit tuning flags 2026-06-12 15:30:44 +08:00
ZacharyZcR 52f872b8d1 fix: webtitle HTTP 请求失败时重试,修复批量扫描 POC 缺失 #587
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
批量扫描(-hf)时并发压力导致 HTTP 请求瞬时失败,getWebTitle
直接返回 error,跳过指纹识别和 POC 触发。

加入指数退避重试(200ms→400ms),最多 3 次,复用 config.MaxRetries。
2026-06-12 12:23:54 +08:00
ZacharyZcR 66b175d623 ci: 升级 CI Go 版本到 1.25 2026-06-12 11:41:49 +08:00
ZacharyZcR 6b5bc191ca fix: 降级 modernc.org/sqlite 到 v1.39.0 适配 CI 2026-06-12 10:11:34 +08:00
ZacharyZcR f883944b2b feat: 自适应并发调度 — 网络探测 + AIMD + 参数智能推导
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据
推导 6 个关键参数,替代硬编码默认值:

- Timeout: median_RTT + 4σ(覆盖 99.9% 正常连接)
- ModuleThreadNum: target_concurrency / 30
- MaxRetries: ceil(log(0.01)/log(loss_rate))(全失败概率 <1%)
- ICMPRate: 环境基准 × fd 系数
- PocNum: 跟随 ModuleThreadNum
- DisablePing: 已有 ICMP 权限降级机制

线程池从单信号(资源耗尽率)升级为 AIMD + 慢启动:
- 慢启动:target/4 起步,500ms 翻倍
- 稳态 AIMD:健康 +5%,拥塞 ×0.5
- 双信号:资源耗尽率 + RTT 趋势(双 EMA)

用户 -t 显式指定时作为 ceiling,探测仍调整其他参数。

测试:单元 + 边界 + 集成 + 真实网络,core 包 580+ 用例全通过。
2026-06-12 09:46:02 +08:00
ZacharyZcR 683707fcd4 feat: Web 版独立入口 + SQLite 持久化存储
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- 拆分 main.go 为 main_cli.go 和 main_web.go,Web 版不再包含 CLI 参数解析
- Web 版直接启动 HTTP 服务,通过 -port/-lang 控制,无需 -web flag
- 结果存储从内存 map 替换为 SQLite(modernc.org/sqlite,纯 Go 零 CGO)
- 数据库文件 ~/.fscan/results.db,进程重启后结果不丢失
- 修复结果分布面板跟随 tab 筛选联动的问题
2026-06-12 05:17:26 +08:00
ZacharyZcR b94e8bc4ca fix: 修复 SSH 扫描 goroutine 泄漏
ssh.NewClientConn 不接受 context,context 取消后底层 TCP 连接未关闭,
导致 readLoop goroutine 永久阻塞在 conn.Read 上。大规模扫描时泄漏数万
goroutine。

- doSSHAuth 新增 goroutine 监听 context 取消并关闭底层连接
- TestSingleCredential 移除 5 秒超时放弃逻辑,改为持续等待清理
2026-06-12 03:59:02 +08:00
ZacharyZcR 4198c1abc8 修复默认扫描 POC 结果缺失 #586
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-06-04 14:41:49 +08:00
ZacharyZcR 68f990d20b 显示 Web 服务识别 URL 2026-06-04 14:26:20 +08:00
ZacharyZcR 72042111c6 Merge pull request #580 from shadow1ng/dev
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
v2.2.0-rc Release
2026-06-01 08:54:24 +08:00
ZacharyZcR 08a87b3128 ci: 完善发版流程
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- release workflow 自动读取 .github/release-notes/<tag>.md 覆盖 body
- 移除手动 prerelease input(goreleaser auto 模式根据 tag 后缀判断)
- 新增 RELEASE.md 发版流程文档
2026-06-01 08:42:17 +08:00
ZacharyZcR cc1af631f0 fix: CI 测试排除路径 mylib → libs 2026-06-01 08:40:14 +08:00
ZacharyZcR 5078fbc975 fix: 更新 lint 排除路径 mylib → libs 2026-06-01 08:39:49 +08:00
ZacharyZcR 354ad2e24b chore: 移除 fscan-lite (C 版本)
功能过于单一(仅 TCP 端口扫描),与 fscan 品牌定位不符。
fscan-nolocal 已是精简版的正确方案。

- 删除 fscan-lite 源码和构建脚本
- goreleaser 移除 lite 构建步骤
- CI 不再安装 mingw 交叉编译工具
2026-06-01 08:37:04 +08:00
ZacharyZcR 942f0fa6c3 fix: web API 版本号改为动态获取,不再硬编码 2026-06-01 08:34:42 +08:00
ZacharyZcR 37b77f9c86 docs: 添加 v2.2.0-rc Release Notes 草稿
用于发版时 gh release create --notes-file 引用。
2026-06-01 08:28:59 +08:00
ZacharyZcR a02eed0518 refactor: mylib 重命名为 libs
更清晰的目录命名,libs/ 存放内嵌的独立协议实现库。
2026-06-01 08:22:13 +08:00
ZacharyZcR 019d13c9d4 chore: 版本号更新为 2.2.0-rc
- README/README_EN 版本标记更新
- globals.go version 常量更新
- 移除 Issue 模板中不可用的安全报告链接
2026-06-01 08:20:01 +08:00
ZacharyZcR c43982f834 chore: 更新 Issue 模板
- 版本号更新至 2.2.0-rc / 2.1.3 / 2.1.2
- 移除 placeholder 预填充内容,避免用户不删不改直接提交
- 新增 SDK、代理、mips 等选项
- macOS/Ubuntu 版本更新
- 新增安全漏洞报告入口指引
- 模板顶部增加"未填写有效信息会被关闭"提示
2026-06-01 08:17:00 +08:00
ZacharyZcR d6d323854a refactor: 完成全局状态到 session 的完整迁移
将 plugins/services、plugins/local、plugins/web、webscan 层的日志输出、
漏洞结果保存和 TCP 计数器从全局 common.Log*/GetGlobalState() 迁移到
session 实例方法,确保 SDK 并发扫描时各实例完全隔离。

- 50 个文件,所有插件日志走 session.Log*
- DoRequest 加入 session 参数,计数器走 session.State
- POC 执行器通过 POCContext.Session 传递
- 仅保留 init() 和 CEL runtime 等无 session 场景的全局回退
2026-06-01 08:13:23 +08:00
ZacharyZcR 569d21a8bc fix: harden address parsing edge cases 2026-06-01 04:03:46 +08:00
ZacharyZcR 8ec96bfe6d fix: harden scan edge cases 2026-06-01 03:32:13 +08:00
ZacharyZcR 8b558b4f12 fix: align atomic counters on arm 2026-06-01 03:03:40 +08:00
ZacharyZcR 3e4e2db722 feat: stream large host scans 2026-06-01 02:55:28 +08:00
ZacharyZcR ac755a9b4a test: avoid expanding massive CIDRs 2026-06-01 02:33:10 +08:00
ZacharyZcR ea17b5b1bc feat: allow multiple added passwords
Fixes #584
2026-06-01 02:06:15 +08:00
ZacharyZcR bfc0e0fb52 fix: support SOCKS5 proxy auth
Fixes #585
2026-06-01 01:59:08 +08:00
ZacharyZcR 40b0eee654 Remove SimpleMaxHosts limit to fix large CIDR scanning
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
Closes #583
2026-05-27 16:16:42 +08:00
ZacharyZcR 8d30ee334c Add native protocol service plugins
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-23 15:51:50 +08:00
ZacharyZcR 73cbe803c4 Expand i18n coverage 2026-05-23 15:18:40 +08:00
ZacharyZcR 9ed6cc95b6 replace string builder sprintf with fprintf
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-23 07:53:28 +08:00
ZacharyZcR a0f0379c6c silence ci warnings and staticcheck suggestions 2026-05-23 07:50:49 +08:00
ZacharyZcR 3c7823355d tighten scan session and HTTP paths 2026-05-23 07:46:53 +08:00
ZacharyZcR 1a714f6a0c reduce scanner client fingerprints 2026-05-23 07:20:24 +08:00
ZacharyZcR 0046817c2e fix mssql login client metadata 2026-05-23 07:17:55 +08:00
ZacharyZcR e0080555b4 fix: 跳过snmp社区字符串暴力破解中的冗余public探测
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-19 21:00:47 +08:00
ZacharyZcR bc2a8f653a fix: 修复新服务扫描插件的健壮性问题
- ipmi: 删除未使用的 encoding/binary 导入
- rmi: TCP读取改用 io.ReadFull 避免分片导致的解析错误
- jdwp: handshake响应读取改用 io.ReadFull 避免分片误判
- nfs: v4协议回退时使用新连接避免残留数据污染
- snmp: 修正timeout计算与其他插件保持一致
2026-05-19 20:46:22 +08:00
ZacharyZcR a92df59dfe fix: NFS plugin RPC probe + UDP dispatch in auto mode
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
NFS: use RPC NULL call to detect NFS service before MOUNT EXPORT
UDP dispatch: query plugin registry directly in auto mode instead of
GetPlugins which excludes UDP from FilterService
2026-05-19 00:37:35 +08:00
ZacharyZcR 0a28db7371 feat: add NFS, RMI and IPMI plugins
NFS (2049/TCP): Sun RPC EXPORT call, lists shared directories
RMI (1099/TCP): Java RMI handshake, detects exposed JMX/RMI endpoints
IPMI (623/UDP): RMCP ping + channel auth capabilities probe

All pure stdlib, zero new dependencies.
2026-05-19 00:01:58 +08:00
ZacharyZcR 778ba019d5 feat: add IMAP and POP3 plugins for mail server detection
IMAP (143/993): banner grab + LOGIN brute force
POP3 (110/995): banner grab + USER/PASS brute force
Both registered as safe default plugins with auth-check capability.
2026-05-18 23:29:52 +08:00
ZacharyZcR f43e3e3d0a feat: add JDWP plugin for Java debug port detection
Detects exposed JDWP (Java Debug Wire Protocol) services on ports
5005/8000/8787/5050. Performs protocol handshake and reports as VULN
since an exposed debug port allows remote code execution.
2026-05-18 23:22:15 +08:00
ZacharyZcR a1588a321f feat: SDK agent integration + UDP plugin framework + SNMP plugin
SDK enhancements for endpoint agent embedding:
- ScanWithController for pause/resume and live stats
- OnProgress callback for periodic progress reporting
- TaskID injection into every scan result
- ScanController with goroutine-safe pause/resume/stats
- Multi-target stats aggregation (race-free)

UDP plugin infrastructure:
- PluginTypeUDP registry with dedicated dispatch path
- DialUDP on ScanSession with rate limiting and packet counting
- UDP plugins bypass TCP port scan, probe targets directly
- FilterService excludes UDP plugins from TCP port matching

SNMP plugin (first UDP plugin):
- SNMPv2c GetRequest probe for sysDescr detection
- Community string brute force (public/private/community/etc)
- Pure stdlib implementation (encoding/asn1)
- Registered as safe default plugin on port 161/UDP

Tests: 95.7% SDK coverage, race-free, 50+ new test cases
2026-05-18 23:11:39 +08:00
ZacharyZcR eb4fa38fea merge sdk architecture polish 2026-05-18 21:41:01 +08:00
ZacharyZcR c15aa3488b polish embedded scanner SDK contracts 2026-05-18 21:40:52 +08:00
ZacharyZcR 218811c38d Merge pull request #582 from shadow1ng/feature/sdk-library
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
Add embeddable scanner SDK
2026-05-18 18:03:12 +08:00
ZacharyZcR 5a884ca6ad fix lint issues before merge 2026-05-18 17:59:00 +08:00
ZacharyZcR 5942d3bbcb preserve sdk host language state 2026-05-18 17:43:00 +08:00
ZacharyZcR c0a9cfd8f5 use config proxy state for credential prechecks 2026-05-18 17:13:19 +08:00
ZacharyZcR c16aa04e28 isolate session network checks 2026-05-18 17:11:24 +08:00
ZacharyZcR 856eeccd78 respect per-call session dial timeouts 2026-05-18 16:35:58 +08:00
ZacharyZcR 13f7997d16 isolate scan strategy runtime state 2026-05-18 16:33:47 +08:00
ZacharyZcR adb3ac5b74 add session-aware scan logging 2026-05-18 16:19:34 +08:00
ZacharyZcR 8de7570268 allow concurrent embedded scans 2026-05-18 16:11:41 +08:00
ZacharyZcR 3dde0c6a8e move plugin safety metadata to registry 2026-05-18 15:57:18 +08:00
ZacharyZcR d4ed0867c9 polish scanner SDK API and docs 2026-05-18 15:49:57 +08:00
ZacharyZcR 6605c93dd9 improve embedded scanner runtime 2026-05-18 14:53:00 +08:00
ZacharyZcR 6bfa05cb45 add embeddable scanner SDK 2026-05-18 14:41:15 +08:00
ZacharyZcR bce0a718ea slim mssql credential check 2026-05-18 14:00:45 +08:00
ZacharyZcR 40c3f1423e slim oracle credential check 2026-05-18 13:13:40 +08:00
ZacharyZcR bfd64f78ed 瘦身: Kafka/MongoDB/Cassandra用raw TCP替代重型依赖
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- kafka: 移除IBM/sarama(45MB), 自实现SASL PLAIN+ApiVersions协议(~150行)
- mongodb: 移除mongo-driver(25MB), 自实现OP_MSG+saslStart认证(~180行)
- cassandra: 移除gocql(1.2MB), 自实现CQLv4 STARTUP+SASL PLAIN(~130行)
- 同时移除间接依赖: pierrec/lz4, klauspost/compress, eapache/snappy等

二进制: 47MB → 40MB (-15%), 移除~55MB压缩依赖
全部13个测试包通过
2026-05-18 06:37:05 +08:00
ZacharyZcR 7063027acf 性能优化: 热路径零分配, 自适应池CAS无锁化, 锁外计算
- port_scan: fmt.Sprintf→JoinHostPort+fmtPort零分配地址格式化 (2.5x)
- port_scan: strings.ToLower→containsFold零分配大小写不敏感匹配 (2.5x)
- port_scan: slidingWindowSchedule修复semaphore泄漏bug
- service_probe: readFromConn预分配4KB缓冲区消除扩容
- adaptive_pool: maybeAdjust用atomic CAS代替持锁检查, 99%免锁
- adaptive_timeout: Timeout锁外计算均值/标准差, 只锁缓存更新
- 新增perf_bench_test.go基准测试验证所有优化
2026-05-18 06:11:10 +08:00
ZacharyZcR 639298b7c8 修复全部SA1006: fmt.Errorf(i18n)统一使用"%s"前缀 2026-05-18 05:17:40 +08:00
ZacharyZcR f427f04d35 修复SA1006: fmt.Errorf(i18n.Tr)改用fmt.Errorf("%s", ...) 2026-05-18 05:08:34 +08:00
ZacharyZcR 05e383d8bf 修复lint: errcheck返回值检查, SA1006动态格式化字符串
- scanner_core: 3处InitializeGlobalVScan()返回值用_显式丢弃
- flag/initialize/network: fmt.Errorf(i18n.Tr(...))改为fmt.Errorf("%s", i18n.Tr(...))
2026-05-18 05:07:02 +08:00
ZacharyZcR a560531e70 修复CI: 覆盖率门禁改用awk整数比较, lint过滤简化
- 覆盖率检查移除bc依赖,改用awk int()整数比较
- lint过滤用单条grep -vE替代管道链,更健壮
- 测试命令预计算包列表避免子shell问题
2026-05-18 04:57:39 +08:00
ZacharyZcR 77827bef66 修复凭证测试器计数器、消除panic、补齐i18n
- credential_tester: testCredentialWithRetry返回ErrorType,修复网络错误计数器永久不递增的bug
- scanner: os.Exit(1)改为return,defer Cleanup可正常执行
- probe_parser: 5处panic改为error返回,调用链透传到init()
- common库: parsers/initialize/network/session共17处硬编码中文改用i18n
- services插件: 18个文件115处硬编码中文改用i18n
- locale: 补齐service/parser/network相关~25个中英文键
2026-05-18 04:55:20 +08:00
ZacharyZcR c266912dcb 修复10个代码缺陷: panic防护, 死锁, 密钥泄漏, 错误吞没
- eval_random: randomInt参数max<=min时不再panic,返回CEL错误
- scanner: 长驻插件nil/panic时兜底发送ready通道,消除死锁
- poc_executor: Ceye API密钥改为环境变量CEYE_API/CEYE_DOMAIN
- Eval: ParseResponse加入oResp.Request nil检查
- Eval: reverseCheck中http.NewRequest错误不再忽略
- poc_executor: clusterpoc中CEL表达式求值错误记录日志
- winwmi: PowerShell执行失败完整记录错误信息
- sshkey: authorized_keys读取失败处理错误
- minidump: Scan结束后释放系统DLL句柄
- Windows插件: PE文件错误消息改用i18n
2026-05-18 03:47:42 +08:00
ZacharyZcR 2f2b30763c 修复硬编码中文,迁移废弃的 uuid 依赖
- flag.go: -local/-perf 参数改用 i18n.GetText,checkParameterConflicts 改用 i18n.Tr
- 补齐中英文 locale 中 flag_local_plugin/flag_perf_stats/param_local_multi_plugin
- satori/go.uuid 迁移为 google/uuid (satori 已废弃)
- CI 补充 lint Go 版本说明
2026-05-18 02:43:45 +08:00
ZacharyZcR 72403ebb32 Merge remote-tracking branch 'origin/dev' into dev
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-16 06:57:16 +08:00
ZacharyZcR 4fa8fdc30e chore: 砍掉 downloader/shellenv,新增 sshkey 插件
- 删除 downloader(curl/certutil 可替代)
- 删除 shellenv(劫持 ls 别名动静太大,实用性差)
- 新增 sshkey:生成 ed25519 密钥对,注入 authorized_keys,
  私钥保存到当前目录,支持多用户(root 权限下自动注入 root)
2026-05-16 06:56:12 +08:00
ZacharyZcR bf210dd94d refactor: minidump 实现降级链凭据提取
不再检测到杀软就直接退出,改为三级降级:
1. 直接 MiniDumpWriteDump(无杀软时尝试,15秒超时)
2. comsvcs.dll(系统签名DLL,部分杀软不拦截)
3. reg save SAM/SECURITY/SYSTEM(离线方式,不碰 LSASS)
全部失败才退出,0.2 秒完成不会卡死系统
2026-05-16 06:51:42 +08:00
ZacharyZcR cb1079598d feat: cleaner 支持清理所有持久化痕迹,minidump 增加杀软前置检测
cleaner 新增 Windows 持久化完整清理:
- Winlogon Shell/Userinit 恢复默认值
- IFEO 映像劫持清理
- 注册表 Run/RunOnce 清理
- 计划任务、服务、启动文件夹清理
- BITS 任务、WMI 事件订阅清理
- Prefetch 记录清理

minidump 增加杀软前置检测,发现 Defender/EDR 直接退出,
避免 MiniDumpWriteDump hang 导致系统崩溃
2026-05-16 06:20:00 +08:00
ZacharyZcR ec10097e76 fix: minidump 增加杀软检测和缩短超时,防止 hang 导致系统崩溃
- 新增 isAVBlocking 检测 Defender/EDR 进程,发现活跃杀软直接跳过
- dump 超时从 120 秒缩短到 15 秒(正常 dump 几秒完成)
- 三层防护:杀软检测 → 权限检测 → 超时兜底
2026-05-16 06:13:47 +08:00
ZacharyZcR 231563e82b fix: 修复 socks5proxy/cleaner/minidump 三个插件问题
- socks5proxy: 监听地址从 127.0.0.1 改为 0.0.0.0,允许外部连接
- cleaner: 重写清理逻辑,精准匹配 fscan 产物,修复 glob 遍历大目录卡死问题,
  history 清理改为真正删除 fscan 相关行
- minidump: SeDebugPrivilege 提升失败时直接退出,不再卡 120 秒超时
2026-05-16 03:17:44 +08:00
ZacharyZcR 5af8682d22 fix: 修复长驻插件启动竞态,删除重复的 bashrc 插件
- 修复 forwardshell/reverseshell/socks5proxy 因竞态导致主进程提前退出
- 删除与 shellenv 功能重复的 bashrc 插件
2026-05-16 00:12:21 +08:00
ZacharyZcR a42f074b00 feat: 新增 4 种持久化插件
- winifeo: IFEO 映像劫持 (sethc/utilman/narrator)
- winbits: BITS 后台任务持久化,通过 GUID 操作避免同名冲突
- winlogon: Winlogon Userinit/Shell 追加
- bashrc: Linux bashrc/profile 注入
2026-05-15 23:48:30 +08:00
ZacharyZcR 1ebcd6dc79 fix: 补全 Windows 持久化插件的执行逻辑
5 个 Windows 持久化插件原先只拼接命令字符串不执行,现全部补上真实执行逻辑:
- winschtask: schtasks /create 创建计划任务
- winservice: sc create 创建系统服务
- winstartup: 复制 PE 到启动文件夹
- winregistry: reg add 写入 Run/RunOnce 注册表键
- winwmi: PowerShell 创建 WMI 事件订阅(单次调用,1.2s 完成)
2026-05-15 23:35:43 +08:00
ZacharyZcR 150e48ba85 refactor: 合并 fileinfo 和 dcinfo 到 systeminfo
- fileinfo 的敏感文件扫描合并到 systeminfo
- dcinfo 的域控信息收集合并到 systeminfo,通过 build tag 隔离平台差异
- 检测到域环境时自动触发域信息收集,WORKGROUP 环境自动跳过
- 新增 systeminfo_dc_windows.go 和 systeminfo_dc_other.go 处理跨平台编译
2026-05-15 23:16:35 +08:00
ZacharyZcR 76a4331fae refactor: 合并 avdetect 到 systeminfo,修复杀软检测误报
- 将 avdetect 的进程匹配逻辑合并到 systeminfo 插件
- 修复进程匹配使用 Contains 导致大量误报,改为精确匹配
- 修正 auto.json 中 Microsoft Security Essentials 为 Microsoft Defender
- 使用 map 索引优化进程匹配性能
- 清理废弃的 envinfo/avdetect i18n key
2026-05-15 22:52:11 +08:00
ZacharyZcR ca4e1ffad3 refactor: 增强 systeminfo 插件并合并 envinfo
- systeminfo 新增网卡信息、权限检测、补丁数量、杀软检测、
  防火墙状态、敏感环境变量扫描等功能
- 合并 envinfo 到 systeminfo,删除独立的 envinfo 插件
- 修复本地插件通过 -m 指定时仍需 -h 参数的问题
- 通过回调机制解决 common/plugins 循环依赖
2026-05-15 22:47:39 +08:00
ZacharyZcR a0fc7881f2 fix: 修复 -m 指定本地插件时误走服务扫描策略的问题
当通过 -m 参数指定本地插件(如 systeminfo)时,由于 LocalMode 未被设置,
会错误地进入服务扫描流程,导致不必要的端口扫描和插件重复执行。
现在 determineScanMode 会自动检测 -m 指定的插件类型,
全部为 local 类型时自动切换到本地扫描模式。
2026-05-15 22:40:52 +08:00
ZacharyZcR bc28df2ed1 Merge branch 'main' into dev 2026-05-15 22:16:07 +08:00
ZacharyZcR 7459da2b44 fix: 恢复 plugins/local 中实际使用的 config 变量
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-15 19:54:36 +08:00
ZacharyZcR 9d191889d0 fix: 移除 composite action 中不支持的 timeout-minutes 2026-05-15 18:43:12 +08:00
ZacharyZcR 2e277a51ed fix: 修复 plugins/local 未使用变量导致编译失败 2026-05-15 18:41:36 +08:00
Ruihang ZhangandZacharyZcR 73b0b0b822 fix: 修复大网段扫描不设置上限的问题 (#581)
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
* fix: 修复大网段扫描不设置上限的问题

修复三处关键Bug:

1. parseIPFullRange() - 修复逻辑错误
   - 问题:当maxTargets=-1时,count >= -1 导致只返回1个IP
   - 修复:添加maxTargets > 0检查条件

2. parseIPShortRange() - 添加maxTargets参数支持
   - 问题:短格式范围(如192.168.1.1-254)无法被限制
   - 修复:添加maxTargets参数并在循环中检查

3. parseIPRangeString() - 修复调用传参遗漏
   - 问题:调用parseIPShortRange()未传递maxTargets
   - 修复:传入maxTargets参数

改进点:
- 大网段解析(如10.0.0.0/8)现在正确处理不超限的情况
- IP范围解析完整性验证
- 添加详细的bug报告和修复文档

* fix parser range limit handling

---------

Co-authored-by: ZacharyZcR <[email protected]>
2026-05-14 21:45:42 +08:00
ZacharyZcR d412786228 feat: 支持国密 TLS 网站扫描
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
- 添加 tjfoc/gmsm 依赖,提供 gmtls 国密 TLS 支持
- 新增国密 HTTP 客户端 ClientGM/ClientNoRedirectGM
- DetectHTTPScheme 增加国密 TLS 回退检测,返回 https-gm 协议标识
- webtitle 插件识别并路由到国密客户端
- DoRequest 标准 TLS 失败时自动回退国密客户端
2026-05-13 19:10:37 +08:00
ZacharyZcR b2e91d9fc0 fix: 修复 POC Cookie/变量提取的两个问题
- poc_adapter: xray/afrog 的 output.search 转换为 Search 字段,多步POC变量传递不再丢失
- poc_executor: Set-Cookie 提取优化不再要求捕获组名含 cookie,sessid/token等命名均生效
2026-05-13 18:37:33 +08:00
ZacharyZcR 3436d6ad02 chore: 版本号更新为 2.1.4 2026-05-13 17:32:28 +08:00
ZacharyZcR a0f323a891 docs: 更新README版本号为 2.1.3 2026-05-13 17:32:27 +08:00
ZacharyZcR 46e50a021f ci: 修复发布超时 — 增加 timeout-minutes 以及步骤级超时
测试构建 / 代码检查 (push) Has been cancelled
测试构建 / 单元测试和构建 (push) Has been cancelled
测试构建 / 构建验证 (push) Has been cancelled
2026-05-13 17:32:09 +08:00
ZacharyZcR b32ce5dec8 ci: 添加 main 分支 push 触发 CI 验证 2026-05-13 17:32:09 +08:00
ZacharyZcRandr00t 2c2ca6ace3 v2.1.3 Release (#572)
* add CVE-2026-24061 detect logic  (#562)

* add CVE-2026-24061 detect logic

* fix(telnet): 修复 errcheck 警告,统一错误处理风格

---------

Co-authored-by: ZacharyZcR <[email protected]>

* fix: 修复 Hub 广播 data race 和端口扫描潜在死锁,清理死代码

- hub.go: broadcast 路径 RLock 改 Lock,修复并发 delete/close 竞争
- port_scan.go: pool.Invoke 失败时释放 wg 和 semaphore,防止死锁
- web_scanner.go: 删除只写不读的 fingerprintCache
- webtitle.go: 移除对已删除 SetFingerprints 的调用
- keylogger.go: 删除未使用的 stopChan 和 isRunning 字段

* refactor: context 穿透扫描生命周期,修复长驻插件阻塞和 Web Stop 无效

- RunScan 接受 context.Context,创建可取消上下文并穿透到所有策略和插件
- 长驻插件(forwardshell/socks5proxy/reverseshell)不再进入 scan WaitGroup,
  通过 ctx.Done() 管理生命周期,解除 wg.Wait() 死锁
- Web Stop API 从 stopChan 改为 context.CancelFunc,取消信号真正传播到扫描链路
- ExecuteScanTasks 和 executeScanTask 支持 context 取消检查,停止分发新任务
- CLI 模式传 context.Background(),行为完全不变

* fix: 修复 Web Stop 信号等待阻塞和 SMB 响应解析越界 panic

- scanner.go: 长驻插件等待信号时同时监听 ctx.Done(),Web Stop 可正常返回
- smb_protocol.go: 响应长度检查修正为 47,远端偏移量全部做边界校验

* fix: POC 扫描接入调用方 context,修复 cachedPocPath 竞争和 ProxyStats data race

- webscan/web_scan.go: WebScan 接受 ctx 参数,替换 context.Background();
  sync.Once 改为 sync.Mutex 保护 POC 加载,消除 cachedPocPath 并发写竞争
- webtitle.go: ctx 从 Scan 穿透到 identifyFingerprintsMulti → triggerPocScan → WebScan
- webpoc.go: 传递 ctx 到 WebScan
- proxy/types.go: ProxyStats 增加 sync.Mutex
- proxy/manager.go: LastConnectTime/LastError/AverageConnectTime 读写加锁

* fix: 修复 ProxyStats 含 mutex 导致的 copylocks 告警

Stats() 方法改为手动构造副本,避免值拷贝复制 sync.Mutex

* fix: 补全 HTTP/TLS proxy stats 加锁,修复 RPC/SMB 解析越界和 POC 加载逻辑

- httpdialer.go/tlsdialer.go: LastError/LastConnectTime/AverageConnectTime 加 mutex
- findnet.go: RPC 响应结束标记位置 < 4 时跳过截断,防止负数切片 panic
- ms17010.go: SMB 会话响应最小长度改为 45,sessionSetupResponse 加长度校验
- web_scan.go: POC 加载失败时不标记 pocLoaded,允许后续重试
- Eval.go: DNSLog 配置去掉 sync.Once,允许多次扫描更新配置

* fix: Web 全局状态同步、字典文件错误提示、长驻插件连接可取消

- scan.go: Web API 构建 config/state 后同步到全局实例
- config_builder.go: 用户名/密码/URL 文件读取失败时输出错误日志
- reverseshell.go: 读命令设 1s 超时,超时后检查 ctx 实现可取消
- forwardshell.go: handleClient 接受 ctx,取消时关闭连接解除阻塞
- socks5proxy.go: handleClient 接受 ctx,取消时关闭连接解除 IO 阻塞

* refactor: 引入 ScanSession,替代全局状态穿透扫描管道 (Phase 1-3)

- 新增 common/session.go: ScanSession 结构体封装 Config/State/Params/Dialer
- RunScan/Strategy/ExecuteScanTasks/executeScanTask 全部接收 session
- Plugin 接口从 Scan(ctx, info, config, state) 改为 Scan(ctx, info, session)
- 48 个插件实现统一更新签名
- Web API 构建 ScanSession 传给 RunScan
- CLI 模式通过 Initialize() 创建 session

* refactor: 全量替换 WrapperTcpWithTimeout 为 session.DialTCP (Phase 4)

- core/port_scan.go: EnhancedPortScan/connectWithRetry/scanSinglePort 接入 session
- core/service_probe.go: SmartPortInfoScanner 持有 session,重连走 session.DialTCP
- core/icmp.go: CheckLive/tcpProbeAlive 接入 session
- 17 个 service 插件: 内部 helper 函数全部穿透 ctx+session
- 移除插件中冗余的手动 TCP 计数(DialTCP 内部已处理)
- plugins/core 下已无 WrapperTcpWithTimeout/SafeTCPDial 调用残留

* refactor: 清除 core/plugins 全局状态依赖,ProgressManager 缓存引用 (Phase 5)

- core/alive_scanner.go: GetFlagVars() → session.Params
- core/service_scanner.go: GetFlagVars() → session.Params 和 config.Target.Ports
- common/progress_manager.go: 缓存 State 和 NoColor 到字段,不再运行时读全局
- common/output_api.go: SaveResult 改用 GetGlobalConfig().Output.DisableSave
- common/network.go: WrapperTcpWithTimeout 标记 Deprecated
- core/ 和 plugins/ 下已无全局状态调用残留

* fix: 修复 dialer timeout 锁死、CVE 检测绕过 session 和误报问题

* fix: 修复 pocDNSLog data race,穿透 ctx 到全链路,消除残余 net.DialTimeout 绕过

* perf: CVE-2026-24061 检测改并发执行,消除硬 sleep 用 deadline 替代

* feat: 项目缓存系统,跨扫描合并资产,缓存 host:port 避免漏报

* perf: 三阶段性能优化,ICMP 并发提升+TCP 并行探测,端口扫描退避调整,服务探测超时减半

* fix: 修复凭据测试清理 goroutine 无限阻塞导致的 goroutine 泄漏

* fix: 凭据测试连续网络错误短路、resultChan 缓冲防阻塞、timer 泄漏修复

* perf: 大规模扫描网段预筛,按 /24 探活跳过空子网,B 段扫描从 2h+ 降至 2min

* fix: 网段预筛从抽样改全覆盖,每台主机发 1 个探测包,消除漏报

* perf: 网段预筛增加网关启发式,.1/.254 多端口优先探测,命中即跳过逐主机兜底

* fix: MSSQL 连接加 encrypt=disable 修复无 TLS 环境扫描失败,Web API 参数校验负数

* feat: Release 增加 armv5 架构支持

* chore: bump version to 2.1.3

* fix: 锁定 golangci-lint 版本为 v2.12.1 修复 CI checksum 校验失败

* fix: golangci-lint 改用 go install 安装,绕过上游安装脚本 checksum 校验问题

* feat: -silent 模式输出 NDJSON 到 stdout,支持 AI agent 管道消费

- 新增 StdoutNDJSONWriter,silent 模式下每条扫描结果实时输出一行 JSON
- LogWithProgress 层拦截人类可读日志,绕过 logger sync.Once 初始化时序问题
- 支持 fscan -h xxx -silent | jq 管道用法

* fix rdp invalid random panic (#573)

* restore ms17010 legacy detection and exploit (#574)

* fix ms17010 legacy packet decoding (#574)

* fix csv web title output (#575)

* fix web result protocol output (#577)

* feat: add -ntp flag to disable TCP supplementary probe

* fix: skip TCP supplementary probe in icmp mode

* feat: add -debug flag with file logging to fscan_debug.log

* fix: resolve golangci-lint errcheck and staticcheck warnings

* fix: skip proxy deep verification for SOCKS5 connections (#579)

SOCKS5 protocol validates connection reachability at protocol level,
deep verification was incorrectly rejecting non-banner services like
SMB(445), RPC(139) and Kerberos(88).

* fix: exclude timeout from scan failure rate calculation (#578)

Timeout is a normal scan result when firewalls drop packets, not a
scan failure. Only resource exhaustion errors count toward failure rate.

* feat: flatten NDJSON output for AI agent consumption and add SKILL.md

* perf: 端口扫描自适应超时,基于 RTT 采样动态调整连接超时

* perf: 四项扫描性能优化

- SO_LINGER=0 快速释放连接,减少 TIME_WAIT 堆积
- 服务探测超时自适应,RTT 采样约束读超时上限
- 端口扫描结果流式传递,pipeline 并行端口扫描和插件执行
- ICMP 批量预构建包和地址,减少发送循环开销

* perf: 六项性能优化

- DNS 解析缓存:sync.Map 缓存避免重复系统调用
- 凭据测试 TCP 预检:不可达目标直接跳过全部凭据
- Web 探测 HTTP Client 复用:全局共享连接池
- 端口扫描 Bloom Filter 去重:替代 map 降低内存
- 进度条 atomic 累加 + 50ms 节流渲染:消除锁竞争
- 服务探针预解码:Init 时预编译,运行时零解码开销

* refactor: replace bloom filter with map for deduplication

Bloom filter has false positive risk which can silently drop valid
scan results. Map provides exact deduplication with negligible memory
overhead at the scale of open ports (typically thousands, not millions).

* fix: credential TCP precheck bypass proxy and pipeline goroutine leak

- Skip TCP precheck when proxy is enabled, net.DialTimeout cannot
  reach targets behind SOCKS5/HTTP proxy
- Drain stream channel on ctx cancellation to prevent EnhancedPortScan
  goroutine from blocking on a full channel

* fix: stream channel 提前返回未关闭导致 goroutine 泄漏,服务探测超时下限 500ms

* fix: resolve golangci-lint errcheck and staticcheck warnings

---------

Co-authored-by: r00t <[email protected]>
2026-05-13 14:41:23 +08:00
451 changed files with 44374 additions and 8376 deletions
+18 -18
View File
@@ -1,6 +1,6 @@
name: 🐛 Bug 报告
description: 报告扫描异常、崩溃或错误行为
title: "[Bug] 简要描述问题"
title: "[Bug] "
labels: ["bug"]
body:
@@ -8,6 +8,7 @@ body:
attributes:
value: |
感谢您提交 Bug 报告!请尽可能详细地填写以下信息,这将帮助我们更快定位和修复问题。
**请勿直接提交未修改的模板**,未填写有效信息的 Issue 会被直接关闭。
- type: dropdown
id: module
@@ -21,8 +22,10 @@ body:
- 弱口令爆破 (Brute Force)
- POC/漏洞扫描 (POC Scan)
- Web指纹识别 (Web Fingerprint)
- 代理功能 (Proxy)
- 输出/日志 (Output/Logging)
- 命令行参数 (CLI Arguments)
- 嵌入式 SDK (Embedded SDK)
- 其他 (Other)
validations:
required: true
@@ -32,7 +35,7 @@ body:
attributes:
label: 严重程度
options:
- 崩溃/无法使用 (Crash)
- 崩溃/Panic (Crash)
- 功能异常 (Malfunction)
- 结果不准确 (Inaccurate)
- 性能问题 (Performance)
@@ -44,10 +47,7 @@ body:
id: description
attributes:
label: 问题描述
description: 清晰描述遇到的问题
placeholder: |
发生了什么?
预期的行为是什么?
description: 清晰描述遇到的问题,包括预期行为和实际行为
validations:
required: true
@@ -55,10 +55,7 @@ body:
id: reproduce
attributes:
label: 复现步骤
description: 提供可以复现问题的命令或步骤
placeholder: |
1. 执行命令: fscan -h xxx -p xxx
2. 观察到...
description: 提供可以复现问题的完整命令(请脱敏 IP/密码)
render: shell
validations:
required: true
@@ -75,10 +72,13 @@ body:
attributes:
label: fscan 版本
options:
- 2.2.0
- 2.2.0-rc
- 2.1.3
- 2.1.2
- 2.1.0
- 2.0.1
- 2.0.0
- 1.8.4
- 2.0.x
- 1.8.x
- 其他/自编译
validations:
required: true
@@ -92,14 +92,13 @@ body:
- Windows 10
- Windows Server 2022
- Windows Server 2019
- Ubuntu 24.04
- Ubuntu 22.04
- Ubuntu 20.04
- CentOS 7
- CentOS 8/Stream
- CentOS 7/8/Stream
- Debian 11/12
- Kali Linux
- macOS 15 (Sequoia)
- macOS 14 (Sonoma)
- macOS 13 (Ventura)
- 其他 Linux
- 其他
validations:
@@ -114,6 +113,7 @@ body:
- arm64 (aarch64)
- 386 (x86)
- arm
- mips/mipsle
validations:
required: true
@@ -121,4 +121,4 @@ body:
id: additional
attributes:
label: 补充信息
description: 其他可能有助于排查问题的信息(自编译请注明 Go 版本)
description: 其他有助于排查的信息(网络环境、是否使用代理、自编译请注明 Go 版本
+4 -5
View File
@@ -1,11 +1,10 @@
# Issue 模板配置
# 禁止空白 issue,强制用户选择模板
blank_issues_enabled: false
contact_links:
- name: 📖 使用文档
- name: 📖 使用文档 / README
url: https://github.com/shadow1ng/fscan/blob/main/README.md
about: 提交 Issue 前请先查阅文档
- name: 💬 讨论区
about: 提交 Issue 前请先查阅文档,确认不是用法问题
- name: 💬 讨论区 / Discussions
url: https://github.com/shadow1ng/fscan/discussions
about: 一般性问题和讨论请使用 Discussions
about: 使用疑问、经验分享、一般性讨论请使用 Discussions
+30 -35
View File
@@ -1,13 +1,14 @@
name: 🎯 误报/漏报
description: 报告扫描结果不准确的问题
title: "[Accuracy] 服务名 - 误报/漏报描述"
title: "[Accuracy] "
labels: ["accuracy"]
body:
- type: markdown
attributes:
value: |
感谢您帮助提高 fscan 的准确性!误报和漏报都是需要优化的问题。
感谢您帮助提高 fscan 的准确性!
**请提供对比验证结果**(如 nmap、手动连接等),仅描述"感觉不对"的报告无法处理。
- type: dropdown
id: type
@@ -34,11 +35,20 @@ body:
validations:
required: true
- type: textarea
id: command
attributes:
label: 使用的命令
description: 执行的 fscan 命令(请脱敏真实 IP/密码)
render: shell
validations:
required: true
- type: textarea
id: fscan-output
attributes:
label: fscan 输出结果
description: 粘贴相关的扫描输出(请脱敏敏感信息如真实IP、密码等
description: 粘贴相关的扫描输出(请脱敏敏感信息)
render: shell
validations:
required: true
@@ -46,11 +56,8 @@ body:
- type: textarea
id: actual
attributes:
label: 实际情况
description: 描述目标的真实状态
placeholder: |
实际上这个端口是关闭的 / 服务版本是 xxx / 密码不是 xxx...
验证方式: 通过 nmap/手动连接/其他工具 确认...
label: 实际情况及验证方式
description: 描述目标的真实状态,以及你通过什么方式验证的(nmap、手动连接、其他工具等)
validations:
required: true
@@ -59,15 +66,13 @@ body:
attributes:
label: 目标操作系统
options:
- Windows Server 2022
- Windows Server 2019
- Windows Server 2016
- Windows 10/11
- Ubuntu
- Windows Server
- Windows Desktop
- Ubuntu/Debian
- CentOS/RHEL
- Debian
- 其他 Linux
- 网络设备
- 容器/Docker
- 未知
validations:
required: true
@@ -77,20 +82,10 @@ body:
attributes:
label: 网络环境
options:
- 直连
- 通过代理
- 直连(同网段)
- 跨网段/路由
- 通过代理 (SOCKS5/HTTP)
- VPN
- 跨网段
validations:
required: true
- type: textarea
id: command
attributes:
label: 使用的命令
description: 执行的 fscan 命令
placeholder: "fscan -h x.x.x.x -p 1-65535 -pwdf pass.txt"
render: shell
validations:
required: true
@@ -99,10 +94,13 @@ body:
attributes:
label: fscan 版本
options:
- 2.2.0
- 2.2.0-rc
- 2.1.3
- 2.1.2
- 2.1.0
- 2.0.1
- 2.0.0
- 1.8.4
- 2.0.x
- 1.8.x
- 其他/自编译
validations:
required: true
@@ -110,8 +108,5 @@ body:
- type: textarea
id: suggestion
attributes:
label: 改进建议
description: 如果您有改进的想法,请分享
placeholder: |
建议增加 xxx 判断条件...
或者调整 xxx 检测逻辑...
label: 改进建议(可选)
description: 如果您有改进判断逻辑的想法
+9 -15
View File
@@ -1,13 +1,14 @@
name: ✨ 功能请求
description: 提议新功能或改进现有功能
title: "[Feature] 一句话描述功能"
title: "[Feature] "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
感谢您的功能建议!请详细描述您的需求,这将帮助我们评估和实现。
感谢您的功能建议!请描述清楚使用场景和期望行为。
纯 "+1" 类请求请用 Reaction 表达,不要开新 Issue。
- type: dropdown
id: category
@@ -19,6 +20,7 @@ body:
- 用户体验 (UX/CLI)
- 输出格式 (Output Format)
- 配置选项 (Configuration)
- 嵌入式 SDK (Embedded SDK)
- 集成/API (Integration/API)
- 其他 (Other)
validations:
@@ -28,10 +30,7 @@ body:
id: problem
attributes:
label: 解决什么问题?
description: 描述您遇到的痛点或使用场景
placeholder: |
在进行 xxx 操作时,我希望能够...
目前的问题是...
description: 描述您的实际使用场景和痛点
validations:
required: true
@@ -39,20 +38,15 @@ body:
id: solution
attributes:
label: 期望的解决方案
description: 描述您希望的功能行为
placeholder: |
希望能够通过 -xxx 参数来...
或者增加一个新的模块来...
description: 描述您希望的功能行为,最好包含预期的命令行用法示例
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: 替代方案
description: 考虑过的其他解决方案或变通方法
placeholder: |
目前我通过 xxx 方式来解决,但是...
label: 替代方案(可选)
description: 目前的变通方法,或考虑过的其他方案
- type: dropdown
id: priority
@@ -60,7 +54,7 @@ body:
label: 优先级建议
description: 您认为这个功能的重要程度
options:
- 高 - 核心功能缺失
- 高 - 核心场景缺失,严重影响使用
- 中 - 明显改善体验
- 低 - 锦上添花
validations:
+12 -29
View File
@@ -1,6 +1,6 @@
name: 🔌 新插件/协议支持
description: 请求支持新的服务、协议或漏洞检测
title: "[Plugin] 协议/服务名称"
title: "[Plugin] "
labels: ["plugin", "enhancement"]
body:
@@ -8,6 +8,7 @@ body:
attributes:
value: |
感谢您的插件请求!fscan 持续扩展对各种服务和协议的支持。
提交前请先确认该服务尚未被支持(查看 README 的支持列表)。
- type: dropdown
id: type
@@ -25,7 +26,7 @@ body:
id: service
attributes:
label: 服务/协议名称
placeholder: "如: Kafka, ClickHouse, etcd, Consul"
description: 例如 ClickHouseetcdConsul、Zookeeper
validations:
required: true
@@ -33,49 +34,31 @@ body:
id: port
attributes:
label: 默认端口
placeholder: "如: 9092, 8123, 2379"
description: 例如 9000、2379、8500
- type: textarea
id: description
attributes:
label: 服务描述
description: 简要介绍这个服务/协议
placeholder: |
这是一个用于 xxx 的服务...
在内网环境中常见于...
label: 服务描述与检测思路
description: 简要介绍服务用途、在内网中的常见程度,以及如何识别(Banner 特征、认证方式等)
validations:
required: true
- type: textarea
id: detection
attributes:
label: 识别方法
description: 如何识别/检测这个服务(如有了解)
placeholder: |
Banner 特征: xxx
默认响应: xxx
认证方式: xxx
- type: textarea
id: reference
attributes:
label: 参考资料
description: 相关文档、其他工具实现、漏洞详情
placeholder: |
- 官方文档: https://...
- 其他工具实现: https://...
- CVE编号: CVE-xxxx-xxxx
label: 参考资料(可选)
description: 官方文档链接、协议规范、其他工具实现、CVE 编号
- type: dropdown
id: prevalence
attributes:
label: 使用普遍程度
description: 这个服务在目标环境中的常见程度
label: 内网常见程度
options:
- 非常常见 (企业环境标配)
- 较为常见 (经常遇到)
- 非常常见企业环境标配
- 较为常见经常遇到
- 偶尔遇到
- 较少见但重要
- 较少见但高价值
validations:
required: true
+61
View File
@@ -0,0 +1,61 @@
# 发版流程
## 预检查
```bash
# 1. 确认 CI 通过
gh run list --branch dev --limit 3
# 2. 全平台 dry-run(手动触发 snapshot 模式)
gh workflow run release.yml -f snapshot=true
# 3. 确认版本号一致
grep "version" common/globals.go
grep "版本" README.md
grep "Version" README_EN.md
```
## 发版
```bash
# 1. 确认 release notes 已就绪
cat .github/release-notes/v<VERSION>.md
# 2. 打 tag(RC 手动打;正式版合并到 main 后由 CI 自动打 tag
git tag v<VERSION>
git push origin v<VERSION>
# CI 自动执行:
# - goreleaser 全平台构建 + UPX 压缩
# - 创建 GitHub ReleaseRC 自动标记 pre-release
# - 用 .github/release-notes/ 下的文件覆盖 release body
```
## 版本号规范
| 场景 | 格式 | 分支 | 示例 |
|------|------|------|------|
| 正式版 | `vX.Y.Z` | main | `v2.2.0` |
| 预发布 | `vX.Y.Z-rc` | dev | `v2.2.0-rc` |
| 热修复 | `vX.Y.Z` | main | `v2.2.1` |
## Release Notes 模板
放在 `.github/release-notes/<tag>.md`,格式参考 `v2.2.0-rc.md`
如果文件不存在,goreleaser 会自动生成基于 commit 的 changelog。
## 正式版发布(RC → 正式)
```bash
# 1. 在 dev 分支准备正式版内容
# common/globals.go, README.md, README_EN.md
# .github/release-notes/v2.2.0.md
# 2. 创建 dev -> main PR
gh pr create --base main --head dev
# 3. 合并 PR
# main push 会自动读取 common/globals.go 中的版本号,创建 v<VERSION> tag
# tag push 会触发 GoReleaser 构建并创建 GitHub Release
```
+5 -17
View File
@@ -24,15 +24,12 @@ runs:
steps:
- name: 设置 Go 环境
uses: actions/setup-go@v5
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
with:
go-version: ${{ inputs.go-version }}
cache: true
- name: 安装 C 编译工具
shell: bash
run: |
sudo apt-get update -qq
sudo apt-get install -y gcc make mingw-w64 gcc-multilib g++-multilib
- name: 下载依赖
shell: bash
@@ -59,12 +56,12 @@ runs:
- name: 上传产物
uses: actions/upload-artifact@v4
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
if: always()
with:
name: build-${{ inputs.mode }}-${{ github.run_id }}
path: |
dist/
dist-lite/
path: dist/
retention-days: ${{ inputs.retention-days }}
- name: 生成报告
@@ -83,7 +80,6 @@ runs:
## 构建产物
### fscan (Go 版本)
$(if [ -d "dist" ]; then
echo "- 文件数: $(find dist -type f 2>/dev/null | wc -l)"
echo "- 大小: $(du -sh dist 2>/dev/null | cut -f1)"
@@ -91,13 +87,5 @@ runs:
echo "- 无产物"
fi)
### fscan-lite (C 版本)
$(if [ -d "dist-lite" ]; then
echo "- 文件数: $(find dist-lite -type f 2>/dev/null | wc -l)"
echo "- 大小: $(du -sh dist-lite 2>/dev/null | cut -f1)"
else
echo "- 无产物"
fi)
[查看产物](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
EOF
+16 -18
View File
@@ -1,11 +1,11 @@
version: 2
project_name: "fscan"
before:
hooks:
- go mod tidy
- go mod download
- chmod +x .github/scripts/build-lite.sh
- bash .github/scripts/build-lite.sh {{ .Version }}
builds:
# 标准版 - 全部插件(全架构)
@@ -16,7 +16,7 @@ builds:
- CGO_ENABLED=0
goos: [windows, linux, darwin, freebsd, solaris]
goarch: [amd64, arm64, "386", arm, mips, mips64, mipsle]
goarm: ["6", "7"]
goarm: ["5", "6", "7"]
gomips: [softfloat]
ignore:
- goos: darwin
@@ -69,7 +69,7 @@ builds:
- CGO_ENABLED=0
goos: [windows, linux, darwin, freebsd, solaris]
goarch: [amd64, arm64, "386", arm, mips, mips64, mipsle]
goarm: ["6", "7"]
goarm: ["5", "6", "7"]
gomips: [softfloat]
ignore:
- goos: darwin
@@ -137,17 +137,17 @@ builds:
upx:
- ids: [fscan, fscan-nolocal, fscan-web]
enabled: true
goos: [windows, linux, freebsd]
goarch: [amd64, "386", arm, arm64, mips, mipsle]
compress: best
goos: [windows, linux]
goarch: [amd64, "386", arm64]
compress: "6"
brute: false
lzma: false
archives:
# 标准版归档
- id: fscan
builds: [fscan]
format: binary
ids: [fscan]
formats: [binary]
allow_different_binary_count: true
name_template: >-
fscan_{{ .Version }}_
@@ -160,8 +160,8 @@ archives:
# 无本地插件版归档
- id: fscan-nolocal
builds: [fscan-nolocal]
format: binary
ids: [fscan-nolocal]
formats: [binary]
allow_different_binary_count: true
name_template: >-
fscan-nolocal_{{ .Version }}_
@@ -174,8 +174,8 @@ archives:
# WebUI版归档
- id: fscan-web
builds: [fscan-web]
format: binary
ids: [fscan-web]
formats: [binary]
allow_different_binary_count: true
name_template: >-
fscan-web_{{ .Version }}_
@@ -231,18 +231,16 @@ release:
| 平台 | 架构 |
|------|------|
| Linux | x64, x32, arm64, armv6, armv7, mips, mips64, mipsle |
| Linux | x64, x32, arm64, armv5, armv6, armv7, mips, mips64, mipsle |
| Windows | x64, x32 |
| macOS | x64, arm64 |
| FreeBSD | x64, x32, arm64, armv6, armv7 |
| FreeBSD | x64, x32, arm64, armv5, armv6, armv7 |
| Solaris | x64 |
footer: |
**完整更新日志**: https://github.com/{{ .Env.GITHUB_OWNER }}/{{ .Env.GITHUB_REPO }}/compare/{{ .PreviousTag }}...{{ .Tag }}
extra_files:
- glob: ./dist-lite/*
snapshot:
name_template: "{{ incpatch .Version }}-dev-{{ .ShortCommit }}"
version_template: "{{ incpatch .Version }}-dev-{{ .ShortCommit }}"
metadata:
mod_timestamp: "{{ .CommitTimestamp }}"
+117
View File
@@ -0,0 +1,117 @@
# fscan v2.2.0-rc.1
> ⚠️ **这是预发布版本 (Release Candidate)**,可能存在未发现的问题。
> 如果你在使用中遇到任何异常,请积极通过 [Issue](https://github.com/shadow1ng/fscan/issues/new/choose) 反馈,帮助我们尽快稳定正式版。
> 生产环境建议继续使用 [v2.1.3](https://github.com/shadow1ng/fscan/releases/tag/v2.1.3)。
---
## 与 v2.2.0-rc 的变更
本版本基于大量实机测试反馈,**修复 30+ 个问题,新增自适应扫描系统**。183 个文件变更。
---
### 🚀 新功能
#### 自适应并发调度
扫描前自动探测网络环境(RTT、丢包率、fd limit),基于探测数据推导关键参数,替代硬编码默认值:
- **Timeout**: `median_RTT + 4σ`(覆盖 99.9% 正常连接),下限 1s,上限 10s
- **ModuleThreadNum**: `ThreadNum / 30`,下限 5,上限 50
- **MaxRetries**: 基于丢包率推导,保证全失败概率 <1%
- **ICMPRate / PocNum**: 跟随环境和并发自动调整
线程池升级为 **AIMD + 慢启动**:慢启动阶段 500ms 翻倍,稳态 AIMD(健康 +5%,拥塞 ×0.5),双信号(资源耗尽率 + RTT 趋势)驱动。
#### 协议级超时下限(ModuleTimeout
新增 `Config.ModuleTimeout()` 方法,保证插件级交互超时不低于 3s。自适应系统将端口扫描超时压到 1s 时,SSH 握手、SNMP 探测、数据库认证等多轮交互协议不再受影响。全部 44 个服务插件已迁移。
#### 限流错误分类(ErrorTypeThrottle
新增 `ErrorTypeThrottle` 错误类型,区分服务端限流(SSH MaxStartups 等)和真正的网络不可达。限流错误不计入连续失败计数,触发 500ms 退避后继续,避免误判目标不可达而提前放弃。
#### Web 版独立入口
- 拆分 `main.go``main_cli.go``main_web.go`
- Web 版结果存储从内存替换为 SQLite 持久化(纯 Go 零 CGO
---
### 🐛 Bug 修复
#### 插件调度(#586 #587 #588
- **非标准端口服务无法匹配插件** — SSH 在 8881 端口,端口匹配失败导致插件不执行。新增服务名称缓存 + 指纹驱动回退匹配 (#588)
- **移除误导性的"无可用插件"日志** — 预检基于静态端口匹配,不代表实际不执行 (#588)
- **默认扫描 POC 结果缺失** — `executeRules` 返回空 `vulName` 导致检测结果被丢弃 (#586)
- **批量扫描(-hf)POC 缺失** — 并发压力下 HTTP 请求瞬时失败未重试,跳过指纹识别和 POC 触发。加入指数退避重试 (#587)
- **UDP 插件在 `-p` 指定端口时被跳过** — 现在按用户指定端口过滤并正确调度
- **`-full` 模式下 Web 插件跳过 `IsMarkedWebService` 检查**
- **不确定服务补做 HTTP 回退探测**,覆盖自定义 HTTP 框架漏网场景
#### UDP 插件
- **UDP 插件阻塞导致扫描无法结束** — `conn.Read()` 在目标不响应时无限阻塞。所有 UDP 插件(SNMP/BACnet/DNS/IPMI/TFTP)统一使用 context timeout + conn.Close 双保险
- **SNMP community 爆破混入通用密码字典** — 57 个通用密码串行探测导致 10 分钟阻塞,精简为 8 个专用 community
#### SSH
- **SSH goroutine 泄漏** — `ssh.NewClientConn` 不接受 contextcontext 取消后底层 TCP 连接未关闭,大规模扫描时泄漏数万 goroutine
- **SSH 握手无 TCP deadline 兜底** — 在 `NewClientConn` 前设置 deadline,握手成功后清除
- **SSH 爆破并发过高** — 从 30 降至 3,避免触发 OpenSSH MaxStartups 限流
#### Redis
- **Redis exploit 无超时保护** — exploit 阶段移除了全部 deadline,改为 30s 超时
- **Redis readReply 吞没非超时错误** — 现在仅对 timeout 类型错误做容忍
#### POC 引擎
- **DetectPocFormat 误判含 transport 的 fscan POC 为 xray 格式** — 修复后 388 个 POC 全部正确加载(之前 8 个失败)
- **CEL clustersend 结果判断错误** — 从字符串比较改为类型断言
- **CEL wait() 函数 nil Reverse 指针 panic**
- **reverseCheck 无超时** — 加 10s 超时防止 ceye API 阻塞
- **正则编译结果未缓存** — `doSearch`/`bmatches` 缓存到 `sync.Map`
#### 参数与输出
- **`-gt` 全局超时参数是死代码** — 现在真正生效,超时后取消所有扫描任务
- **`-nopoc` 禁用 POC 时仍输出错误日志** — 已修复
- **`-debug` 日志文件写入失败** — `applyLogLevel` 重建 Logger 时丢失 `DebugLogFile` 配置
- **`-hash` 不支持 LM:NT 格式** — 现在支持 `aad3b435b51404ee:31d6cfe0d16ae931...` 标准格式
- **`-pwd` 不支持逗号分隔多个密码** — 现在 `-pwd "123,456,root"` 正确拆分
- **`-nobr` 跳过了 Redis 未授权检测** — 未授权是服务探测不是爆破,不受 `-nobr` 影响
- **非终端输出时 ANSI 控制码覆盖扫描结果** — 管道/重定向时自动禁用进度条和颜色
- **静默模式 NDJSON banner 过长** — Redis INFO ~5KB 截断至 200 字符
- **CSV 漏洞 Type 列为空** — 补全 `type` 字段
- **SNMP 探测成功但终端无输出** — 补充 `session.LogVuln` 调用
#### 其他
- **service_probe 连接丢失后静默成功** — `Write`/`Read``Conn=nil` 时返回明确错误
- **MongoDB readMongoMsg 未设置读超时**
- **TXTWriter.Close Sync 失败后未关闭文件**
- **MySQL 3306 服务名误识别为 genetec-5400** — nmap 指纹库误匹配,通过 banner 特征校正
- **gmtls stdout 竞态** — 移除 `os.Stdout` 非同步重定向
---
### 🏗️ 架构优化
- **统一服务缓存** — `webServiceCache` 扩展为通用 `serviceCache`,下沉到 per-session State,消除多实例缓存串台
- **CEL 表达式编译缓存** — 同一 POC 的所有规则共享编译后的 Program
- **POC 全局状态消除** — `allPocs` 全局变量改为 `pocStore` 按 PocPath 缓存,并发场景不再互相覆盖
- **进度条竞态修复** — `isActive` 改为 `atomic.Bool`
- **Lint 全量修复** — cassandra/ipmi/mongodb/webscan 的 ineffassign、unused、errcheck
---
## 反馈与贡献
- 🐛 发现 Bug → [提交 Bug 报告](https://github.com/shadow1ng/fscan/issues/new?template=bug_report.yml)
- 🎯 结果不准 → [提交误报/漏报](https://github.com/shadow1ng/fscan/issues/new?template=false_positive.yml)
- ✨ 功能建议 → [提交功能请求](https://github.com/shadow1ng/fscan/issues/new?template=feature_request.yml)
- 💬 使用疑问 → [Discussions](https://github.com/shadow1ng/fscan/discussions)
+106
View File
@@ -0,0 +1,106 @@
# fscan v2.2.0-rc
> ⚠️ **这是预发布版本 (Release Candidate)**,可能存在未发现的问题。
> 如果你在使用中遇到任何异常,请积极通过 [Issue](https://github.com/shadow1ng/fscan/issues/new/choose) 反馈,帮助我们尽快稳定正式版。
> 生产环境建议继续使用 [v2.1.3](https://github.com/shadow1ng/fscan/releases/tag/v2.1.3)。
---
## 与 v2.1.3 的主要变更
### 🏗️ 架构升级:嵌入式 Scanner SDK
新增 `pkg/fscan` 包,fscan 从纯 CLI 工具进化为**可嵌入的扫描引擎**。
- 支持从其他 Go 程序直接调用 fscan 扫描能力
- 每个 Scanner 实例拥有独立的 config/state/session
- 支持多实例并发扫描,日志和结果完全隔离
- 全局状态已完整迁移到 session 实例
### 🚀 大规模扫描支持
- **流式 Host Iterator** — /8 级别 CIDR 不再一次性展开到内存
- **移除 MaxHosts 硬限制** — 大网段不再被静默截断
- **ARM 原子计数器对齐** — 修复 32 位 ARM 平台 panic
### 🔌 新增协议插件(+9
| 插件 | 用途 |
|------|------|
| IMAP / POP3 | 邮件服务器检测 |
| JDWP | Java Debug 端口检测 |
| NFS / RMI | 文件共享 / Java 远程调用 |
| IPMI | 服务器带外管理 |
| SNMP (UDP) | 网络设备管理 |
| DNS / BACnet / Modbus | DNS 服务 / 工控协议 |
全部为**原生协议实现**,零外部依赖。
### ⚡ 性能与体积优化
- Kafka/MongoDB/Cassandra 从重型客户端库改为 raw TCP 实现,**二进制体积减小**
- 热路径零分配、自适应连接池 CAS 无锁化
- Scanner 客户端指纹收敛(降低被检测概率)
### 🌐 网络能力增强
- **SOCKS5 代理认证**支持(用户名/密码)
- **国密 TLS**SM2/SM3/SM4)网站扫描
- 支持 `-add-password` 追加多个自定义密码
### 🔧 本地后渗透插件
- 新增 sshkey、4 种持久化插件(crontask/systemd/winregistry/winschtask
- minidump 降级链凭据提取 + 杀软前置检测
- cleaner 清理所有持久化痕迹
- systeminfo 整合合并(fileinfo/dcinfo/avdetect/envinfo
### 📦 工程改进
- 目录重命名:`WebScan/``webscan/``mylib/``libs/`
- i18n 覆盖扩展(中/英双语)
- 全量 lint/errcheck 修复
- 补充大量单元测试
---
## ⚠️ 已知问题 / 注意事项
| 问题 | 说明 |
|------|------|
| SDK 并发限速共享 | `CanSendPacket()` 全局限速器在多 Scanner 实例间共享(按进程限速,非按实例) |
| CEL 回连验证走全局 | DNSLog 反连检测的 HTTP 请求计数走全局 state(影响可忽略) |
| 本地插件仅限授权场景 | 持久化/minidump/shell 类插件仅用于授权渗透测试 |
| WebUI 为实验性功能 | `fscan-web` 版本的 Web 界面尚未完善 |
| `common` 包未完全拆分 | 内部架构的 common 包职责仍较重,后续版本持续优化 |
---
## 反馈与贡献
这是一个 RC 版本,我们非常需要你的反馈:
- 🐛 发现 Bug → [提交 Bug 报告](https://github.com/shadow1ng/fscan/issues/new?template=bug_report.yml)
- 🎯 结果不准 → [提交误报/漏报](https://github.com/shadow1ng/fscan/issues/new?template=false_positive.yml)
- ✨ 功能建议 → [提交功能请求](https://github.com/shadow1ng/fscan/issues/new?template=feature_request.yml)
- 💬 使用疑问 → [Discussions](https://github.com/shadow1ng/fscan/discussions)
---
## 版本说明
| 版本 | 说明 |
|------|------|
| **fscan** | 标准版,包含全部插件(推荐) |
| **fscan-nolocal** | 精简版,不含本地后渗透模块(体积更小) |
| **fscan-web** | WebUI版,带 Web 管理界面(主流平台) |
## 平台支持
| 平台 | 架构 |
|------|------|
| Linux | x64, x32, arm64, armv5/6/7, mips, mips64, mipsle |
| Windows | x64, x32 |
| macOS | x64 (Intel), arm64 (Apple Silicon) |
| FreeBSD | x64, x32, arm64, armv5/6/7 |
| Solaris | X64n |
+129
View File
@@ -0,0 +1,129 @@
# fscan v2.2.0
v2.2.0 是 v2.2 系列首个正式版,基于 v2.1.3 之后的 RC 测试和 Issue 反馈整理发布。
本版本重点提升大规模扫描稳定性、POC 扫描可靠性、非标准端口服务识别、插件隔离和嵌入式 SDK 能力。
---
## 重点变化
### 嵌入式 Scanner SDK
新增 `pkg/fscan`,fscan 从纯 CLI 工具扩展为可嵌入的 Go 扫描引擎:
- 支持在 Go 程序内直接调用扫描能力
- Scanner 实例拥有独立 `config` / `state` / `session`
- 全局状态迁移到 session,改善多实例并发隔离
- 补充 SDK 结果转换、配置校验和并发扫描测试
### 大规模扫描稳定性
- 新增流式 Host Iterator,大 CIDR 不再一次性展开到内存
- 移除 MaxHosts 硬限制,大网段不再被静默截断
- 新增自适应并发调度,基于 RTT、丢包率、fd limit 自动推导扫描参数
- 线程池升级为 AIMD + 慢启动,遇到资源耗尽时自动降速
- `-gt` 全局超时正式生效,超时后会取消扫描任务
- 新增 `-nsp`,可禁用网段预筛
### 服务识别与插件调度
- 修复非标准端口服务无法匹配插件的问题
- 新增服务缓存和指纹驱动插件匹配
- `-full` 模式下 Web 插件可覆盖所有开放端口
- 不确定服务增加 HTTP 回退探测
- 移除误导性的“无可用插件”日志
- 用户指定 `-p` 时 UDP 插件按端口交集正确调度
### Web / POC 扫描
- 修复默认扫描 POC 结果缺失
- 修复 `-hf` 批量扫描时 POC 缺失
- 修复 HTTPS 端口误用 HTTP 扫描 POC
- 修复 POC 结果文件只显示 `vulnerable` 不显示漏洞名称
- POC 加载按 `pocpath` 隔离缓存,多 session 不再互相覆盖
- 修复 CEL、reverseCheck、正则缓存等稳定性问题
- `-nopoc` 禁用 POC 时不再输出误导性错误日志
### 新增协议插件
新增多种原生协议插件,覆盖邮件、Java 调试、文件共享、带外管理、UDP 和工控场景:
| 插件 | 用途 |
|------|------|
| IMAP / POP3 | 邮件服务器检测 |
| JDWP | Java Debug 端口检测 |
| NFS / RMI | 文件共享 / Java 远程调用 |
| IPMI | 服务器带外管理 |
| SNMP / DNS / BACnet / Modbus | 网络设备、DNS、工控协议检测 |
### Web 版
- 拆分 CLI / Web 入口
- Web 版结果存储改为 SQLite 持久化
- Web API 版本号改为动态读取
---
## Bug 修复摘要
- 修复 #586 默认扫描 POC 结果缺失
- 修复 #587 `-hf` 批量扫描 POC 缺失
- 修复 #588 非标准端口服务插件匹配问题
- 修复 #590 Telnet Cisco MOTD 横幅误判 shell prompt
- 修复 #591 HTTPS POC 协议错误与结果名称缺失
- 修复 #592 service probe 空指针 panic
- 修复 #593 `-ehf` 排除主机未生效,支持 IP / CIDR / range
- 修复 UDP 插件阻塞导致扫描无法结束
- 修复 SSH goroutine 泄漏和握手 deadline 问题
- 修复 Redis exploit 超时和非超时错误处理
- 修复 MongoDB SCRAM、Cassandra、Oracle 等协议问题
- 修复 SOCKS5 代理认证、LM:NT hash、逗号分隔密码等参数问题
- 修复非终端输出 ANSI 控制码覆盖结果
- 修复 CSV / NDJSON / TXT 输出若干字段问题
- 修复 ARM 32 位原子计数器对齐问题
---
## 升级注意
- WebUI 仍建议视为实验性能力
- 本地后渗透插件仅用于授权环境
- v2.2.0 改动较大,建议从 v2.1.3 升级的用户先在测试环境验证扫描参数
- 如依赖旧版本输出格式,请重点检查 POC、SERVICE、VULN 结果字段
---
## 版本说明
| 版本 | 说明 |
|------|------|
| **fscan** | 标准版,包含全部插件(推荐) |
| **fscan-nolocal** | 精简版,不含本地模块(体积更小) |
| **fscan-web** | WebUI 版,带 Web 管理界面(主流平台) |
## 平台支持
| 平台 | 架构 |
|------|------|
| Linux | x64, x32, arm64, armv5/6/7, mips, mips64, mipsle |
| Windows | x64, x32 |
| macOS | x64, arm64 |
| FreeBSD | x64, x32, arm64, armv5/6/7 |
| Solaris | x64 |
---
## 校验
本版本已通过:
- `go test ./...`
- 近期 Issue 回归验证
- 本地 HTTP / HTTPS POC 扫描验证
- `-hf` 批量 POC 扫描验证
- `-ehf` IP / CIDR 排除验证
完整变更记录见:
https://github.com/shadow1ng/fscan/compare/v2.1.3...v2.2.0
+64
View File
@@ -0,0 +1,64 @@
# fscan v2.2.1
v2.2.1 是 v2.2 系列的稳定性修复版本,重点解决大规模扫描提前结束、开放端口漏扫、服务识别误差和协议插件异常,并扩充常见内网产品的 POC 覆盖。
---
## 重点变化
### 扫描稳定性
- 默认不再启用全局扫描超时,避免大网段或弱网络环境下扫描被整体提前终止
- `-gt` 仍可用于显式设置全局超时;嵌入式 SDK 同样支持按需配置
- 修复高并发场景下自适应超时过低导致开放端口漏扫的问题
- 扫描异常退出时正常执行结果清理和落盘,避免主结果为空及 `.realtime.tmp` 残留
### 服务识别与协议插件
- 修复 `-nobr` 下 VNC 仍继续尝试密码的问题
- Telnet 未授权结果增加真实命令执行验证,降低提示符误报
- SSH 服务识别支持 RFC 4253 允许的 identification 前提示行
- 修复 RDP Fast-Path 数据早于监听器初始化时触发的 nil pointer panic
- 通用 SSL/TLS 指纹不再直接判定为 Web 服务,减少 MQTT TLS 等非 HTTP 服务的握手报错
### POC 覆盖
新增 100 个经过整理的内网常见产品 POC,覆盖:
- 泛微、致远、蓝凌、万户、通达、用友、金蝶、金蝶云星空
- H3C、海康威视、锐捷、深信服、契约锁、帆软
- Nacos、Kubernetes、GitLab、Jenkins、Hadoop、Spark、Solr、Elastic Stack 等
同时补充第三方来源说明,并为新增 POC 增加加载测试。
---
## 已解决 Issue
- #596 Telnet 未授权误报
- #598 高并发下开放端口漏扫
- #599 `-nobr` 未阻止 VNC 密码尝试
- #600 TLS 服务扫描报错、结果文件为空及临时文件残留
- #601 SSH 服务端口无法识别
- #603 RDP Fast-Path nil pointer panic
---
## 升级说明
- 从 v2.2.0 可直接升级
- 如需限制整个扫描任务的最长运行时间,请显式传入 `-gt <秒数>`
- POC 仅用于已获授权的安全测试环境
## 校验
本版本已通过:
- `go test ./...`
- GitHub Actions 测试构建
- GoReleaser 全平台 snapshot 构建
- VNC、SSH、RDP、Telnet 和 Web 服务识别回归测试
完整变更记录:
https://github.com/shadow1ng/fscan/compare/v2.2.0...v2.2.1
-81
View File
@@ -1,81 +0,0 @@
#!/bin/bash
# 构建 fscan-lite 并准备发布产物
set -e
VERSION="${1:-dev}"
LITE_DIR="fscan-lite"
OUTPUT_DIR="dist-lite"
echo "==> 构建 fscan-lite (版本: $VERSION)"
# 清理旧产物
rm -rf "$OUTPUT_DIR"
mkdir -p "$OUTPUT_DIR"
# 进入 lite 目录
cd "$LITE_DIR"
# 源文件
SOURCES="src/main.c src/scanner.c src/platform.c"
INCLUDE="-Iinclude"
CFLAGS_BASE="-std=c89 -Wall -O2"
# 构建 Linux 版本
echo "==> 构建 Linux 版本..."
# Linux x64
echo " - Linux x64"
mkdir -p bin
gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite $SOURCES -lpthread
cp bin/fscan-lite "../$OUTPUT_DIR/fscan-lite_${VERSION}_linux_x64"
rm -rf bin
# Linux x32
echo " - Linux x32"
mkdir -p bin
gcc $CFLAGS_BASE -m32 $INCLUDE -o bin/fscan-lite $SOURCES -lpthread 2>/dev/null || echo " (跳过: 缺少 32-bit 支持)"
if [ -f bin/fscan-lite ]; then
cp bin/fscan-lite "../$OUTPUT_DIR/fscan-lite_${VERSION}_linux_x32"
fi
rm -rf bin
# 构建 Windows 版本
echo "==> 构建 Windows 版本..."
# Windows x64
echo " - Windows x64"
mkdir -p bin
x86_64-w64-mingw32-gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite.exe $SOURCES -lws2_32 -static
if [ -f bin/fscan-lite.exe ]; then
cp bin/fscan-lite.exe "../$OUTPUT_DIR/fscan-lite_${VERSION}_windows_x64.exe"
echo " ✓ 编译成功"
else
echo " ✗ 编译失败"
fi
rm -rf bin
# Windows x32
echo " - Windows x32"
mkdir -p bin
i686-w64-mingw32-gcc $CFLAGS_BASE $INCLUDE -o bin/fscan-lite.exe $SOURCES -lws2_32 -static
if [ -f bin/fscan-lite.exe ]; then
cp bin/fscan-lite.exe "../$OUTPUT_DIR/fscan-lite_${VERSION}_windows_x32.exe"
echo " ✓ 编译成功"
else
echo " ✗ 编译失败"
fi
rm -rf bin
cd ..
# 统计产物
echo ""
echo "==> 构建完成!"
echo "产物列表:"
if [ -d "$OUTPUT_DIR" ]; then
ls -lh "$OUTPUT_DIR" 2>/dev/null || echo " (无产物)"
echo ""
FILECOUNT=$(ls "$OUTPUT_DIR" 2>/dev/null | wc -l)
echo "总计: $FILECOUNT 个文件"
fi
+115 -8
View File
@@ -2,6 +2,8 @@ name: 发布
on:
push:
branches:
- main
tags:
- 'v*'
workflow_dispatch:
@@ -14,18 +16,20 @@ on:
description: '创建草稿发布'
type: boolean
default: false
prerelease:
description: '标记为预发布'
type: boolean
default: false
permissions:
contents: write
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
release:
auto-tag:
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
timeout-minutes: 45
timeout-minutes: 5
outputs:
tag: ${{ steps.version.outputs.tag }}
steps:
- name: 检出代码
@@ -33,10 +37,113 @@ jobs:
with:
fetch-depth: 0
- name: 读取版本号
id: version
shell: bash
run: |
VERSION=$(sed -n 's/^[[:space:]]*version = "\(.*\)"/\1/p' common/globals.go)
if [ -z "$VERSION" ]; then
echo "❌ 无法从 common/globals.go 读取版本号"
exit 1
fi
TAG="v${VERSION}"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
echo "准备发布 ${TAG}"
- name: 创建发布标签
shell: bash
run: |
TAG="${{ steps.version.outputs.tag }}"
if git ls-remote --exit-code --tags origin "refs/tags/${TAG}" >/tmp/tag-ref 2>/dev/null; then
git fetch --force origin "refs/tags/${TAG}:refs/tags/${TAG}"
TAG_COMMIT=$(git rev-list -n 1 "${TAG}")
HEAD_COMMIT=$(git rev-parse HEAD)
if [ "$TAG_COMMIT" = "$HEAD_COMMIT" ]; then
echo "✅ ${TAG} 已指向当前提交,跳过创建"
exit 0
fi
echo "❌ ${TAG} 已存在,但不指向当前提交"
echo "tag: ${TAG_COMMIT}"
echo "head: ${HEAD_COMMIT}"
exit 1
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git tag -a "${TAG}" -m "Release ${TAG}"
git push origin "${TAG}"
release:
needs: [auto-tag]
if: ${{ always() && (startsWith(github.ref, 'refs/tags/') || github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.auto-tag.result == 'success')) }}
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- name: 检出代码
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: 解析发布标签
id: release_tag
shell: bash
env:
AUTO_TAG: ${{ needs.auto-tag.outputs.tag }}
SNAPSHOT: ${{ inputs.snapshot }}
run: |
if [[ "${GITHUB_REF}" == refs/tags/* ]]; then
TAG="${GITHUB_REF_NAME}"
elif [ "${GITHUB_EVENT_NAME}" = "push" ] && [ "${GITHUB_REF}" = "refs/heads/main" ]; then
TAG="${AUTO_TAG}"
git fetch --force origin "refs/tags/${TAG}:refs/tags/${TAG}"
elif [ "${GITHUB_EVENT_NAME}" = "workflow_dispatch" ] && [ "${SNAPSHOT}" = "true" ]; then
TAG="${GITHUB_REF_NAME}"
else
echo "❌ 非 snapshot 手动发布必须从 tag 触发"
exit 1
fi
echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
echo "RELEASE_TAG=${TAG}" >> "$GITHUB_ENV"
- name: 准备 Release Notes
if: ${{ !inputs.snapshot }}
run: |
TAG="${RELEASE_TAG}"
NOTES_FILE=".github/release-notes/${TAG}.md"
if [ -f "$NOTES_FILE" ]; then
echo "📝 找到版本 Release Notes: $NOTES_FILE"
cp "$NOTES_FILE" /tmp/release-notes.md
else
echo "⚠️ 未找到 $NOTES_FILE,使用自动生成的 changelog"
echo "" > /tmp/release-notes.md
fi
echo "RELEASE_NOTES_FILE=/tmp/release-notes.md" >> $GITHUB_ENV
- name: 构建和发布
uses: ./.github/actions/build-release
with:
mode: ${{ inputs.snapshot && 'snapshot' || 'release' }}
go-version: '1.20'
go-version: '1.25'
retention-days: '90'
release-args: ${{ inputs.draft && '--draft' || '' }} ${{ inputs.prerelease && '--prerelease' || '' }}
release-args: ${{ inputs.draft && '--draft' || '' }}
- name: 更新 Release Notes
if: ${{ !inputs.snapshot && env.RELEASE_NOTES_FILE != '' }}
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${RELEASE_TAG}"
NOTES_FILE="${RELEASE_NOTES_FILE}"
if [ -s "$NOTES_FILE" ]; then
echo "📝 更新 Release Notes..."
# 用版本特定的 notes 替换 goreleaser 生成的 body
gh release edit "$TAG" --notes-file "$NOTES_FILE"
echo "✅ Release Notes 已更新"
fi
+25 -34
View File
@@ -3,6 +3,7 @@ name: 测试构建
on:
push:
branches:
- main
- dev
- dev-*
- develop
@@ -37,6 +38,9 @@ on:
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
lint:
name: 代码检查
@@ -50,37 +54,27 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.23'
go-version: '1.25'
cache: true
- name: 运行 golangci-lint
run: |
# 下载 golangci-lint v2
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin latest
# 安装 golangci-lint v2 (需 Go 1.22+,仅用于静态分析,不影响二进制兼容性)
go install github.com/golangci/golangci-lint/v2/cmd/[email protected]
# 运行检查并灵活处理结果
set +e
golangci-lint run --timeout=5m > lint_output.txt 2>&1
LINT_EXIT_CODE=$?
# 运行检查
golangci-lint run --timeout=5m > lint_output.txt 2>&1 || true
cat lint_output.txt
set -e
# 只关注真正的 bug,忽略代码质量建议
# 过滤规则:
# - gocognit/gocyclo: 复杂度警告(阈值已在配置中设置)
# - QF/S/ST: staticcheck 的代码质量改进建议(非bug)
if [ $LINT_EXIT_CODE -ne 0 ]; then
CRITICAL_ISSUES=$(grep -E "\.go:[0-9]+:[0-9]+:" lint_output.txt | grep -v "gocognit" | grep -v "gocyclo" | grep -v "QF[0-9]" | grep -v " S[0-9]" | grep -v "ST[0-9]" || true)
if [ -n "$CRITICAL_ISSUES" ]; then
echo "❌ Linting failed with critical issues:"
echo "$CRITICAL_ISSUES" | head -20
exit 1
else
echo "⚠️ Only quality suggestions - PASSING"
exit 0
fi
# 过滤非关键问题(复杂度建议和 staticcheck 风格建议不阻塞 CI
CRITICAL=$(grep -E "\.go:[0-9]+:[0-9]+:" lint_output.txt \
| grep -vE "gocognit|gocyclo|QF[0-9]+| S[0-9]+|ST[0-9]+" || true)
if [ -n "$CRITICAL" ]; then
echo "❌ Linting failed with critical issues:"
echo "$CRITICAL" | head -20
exit 1
fi
echo "✅ No lint issues found"
echo "✅ Lint 检查通过"
- name: 检查代码复杂度(质量门禁)
run: |
@@ -120,7 +114,7 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.20'
go-version: '1.25'
cache: true
- name: 下载依赖
@@ -131,7 +125,8 @@ jobs:
- name: 运行测试
run: |
# 排除第三方grdp库测试(存在环境依赖问题)
go test -vet=off -race -coverprofile=coverage.out -covermode=atomic $(go list ./... | grep -v '/mylib/grdp/')
PKGS=$(go list ./... | grep -v '/libs/grdp/')
go test -vet=off -race -coverprofile=coverage.out -covermode=atomic $PKGS
- name: 上传覆盖率
uses: actions/upload-artifact@v4
@@ -152,18 +147,14 @@ jobs:
echo "### 🚦 覆盖率质量门禁" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
# 提取总体覆盖率
TOTAL_COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
# 提取总体覆盖率awk 整数比较,不需要 bc
TOTAL_COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print int($3)}')
echo "总体覆盖率: ${TOTAL_COVERAGE}%" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
# 检查核心模块覆盖率(core, common/parsers必须>50%
CORE_COVERAGE=$(go tool cover -func=coverage.out | grep "^github.com/shadow1ng/fscan/core/" | grep -v "_test.go" | awk '{sum+=$3; count++} END {if(count>0) print sum/count; else print 0}')
PARSERS_COVERAGE=$(go tool cover -func=coverage.out | grep "^github.com/shadow1ng/fscan/common/parsers/" | grep -v "_test.go" | awk '{sum+=$3; count++} END {if(count>0) print sum/count; else print 0}')
# 警告阈值:总体<40%, 核心模块<50%
if (( $(echo "$TOTAL_COVERAGE < 40" | bc -l) )); then
# 警告阈值:总体<40%
if [ "$TOTAL_COVERAGE" -lt 40 ]; then
echo "⚠️ **警告**: 总体覆盖率 ${TOTAL_COVERAGE}% < 40%,建议补充测试" >> $GITHUB_STEP_SUMMARY
fi
@@ -190,7 +181,7 @@ jobs:
- name: 设置 Go 环境
uses: actions/setup-go@v5
with:
go-version: '1.20'
go-version: '1.25'
cache: true
- name: 构建验证
+5 -2
View File
@@ -1,9 +1,9 @@
result.txt
result.json
main
/main
.idea
fscan.exe
fscan
/fscan
fscanapi.csv
# IDE files / IDE 文件
@@ -58,6 +58,9 @@ bin/
*.dll
*.so
*.dylib
/fscan_cli
/fscan_web
/embed-agent
# Web UI build / Web前端构建
web-ui/node_modules/
+1 -1
View File
@@ -45,7 +45,7 @@ linters:
- vendor
- testdocker
- image
- mylib/grdp
- libs/grdp
issues:
max-issues-per-linter: 50
+1 -1
View File
@@ -64,7 +64,7 @@ build-web: build-ui
@echo "$(BLUE)构建Web版本...$(NC)"
$(GO) build -tags web -ldflags="-s -w" -trimpath -o $(BINARY_NAME)-web .
@echo "$(GREEN)✓ 构建完成: $(BINARY_NAME)-web$(NC)"
@echo "$(BLUE)提示: 运行 ./$(BINARY_NAME)-web -web 启动Web界面$(NC)"
@echo "$(BLUE)提示: 运行 ./$(BINARY_NAME)-web 启动Web界面(默认端口 10240$(NC)"
## build-ui: 构建前端(需要Node.js和npm
build-ui:
+13 -3
View File
@@ -4,7 +4,7 @@
内网综合扫描工具,一键自动化漏扫。
**版本**: 2.1.2
**版本**: 2.2.1
## 功能特性
@@ -52,6 +52,7 @@
- **并发控制** - 端口扫描线程、服务扫描线程独立配置
### 扩展功能
- **SDK嵌入** - `pkg/fscan`提供Go SDK,可嵌入Agent或安全平台,支持任务控制(Pause/Resume)、实时进度回调、TaskID追溯
- **Web管理界面** - 可视化扫描任务管理(条件编译 -tags web)
- **Lab靶场环境** - 内置Docker靶场用于测试学习
- **插件化架构** - 服务插件/Web插件/本地插件分离,易于扩展
@@ -185,10 +186,10 @@
```bash
# 标准编译
go build -ldflags="-s -w" -trimpath -o fscan main.go
go build -ldflags="-s -w" -trimpath -o fscan .
# 带Web管理界面
go build -tags web -ldflags="-s -w" -trimpath -o fscan main.go
go build -tags web -ldflags="-s -w" -trimpath -o fscan-web .
```
## 安装
@@ -211,9 +212,13 @@ yay -S fscan-git
`fscan.exe -h 192.168.x.x -m ssh -user root -pwd password`
![](image/3.png)
`fscan.exe -h 192.168.x.x -m ssh -user root -pwda pass1 pass2 pass3` (追加多个密码)
`fscan.exe -h 192.168.x.x -p80 -proxy http://127.0.0.1:8080`
![](image/2020-12-12-13-34-44.png)
`fscan.exe -h 192.168.x.x -socks5 socks5://user:[email protected]:1080` (SOCKS5认证代理)
`fscan.exe -h 192.168.x.x -p 139 -m netbios`
![](image/netbios.png)
@@ -234,6 +239,11 @@ yay -S fscan-git
- **后两周** - Bug修复与代码整合
- **欢迎PR** - 期待您的贡献!
### SDK & Agent 集成
- 扩展SDK能力,完善端侧Agent嵌入支持
- 断点续扫、带宽级限速、内存水位控制
- 更多Agent场景的集成示例
### 插件生态
- 持续扩展服务插件覆盖范围
- 为每个服务插件开发更多漏洞检测和利用能力
+7 -3
View File
@@ -4,7 +4,7 @@
Comprehensive intranet scanning tool for automated vulnerability assessment.
**Version**: 2.1.2
**Version**: 2.2.1
## Features
@@ -185,10 +185,10 @@ Comprehensive intranet scanning tool for automated vulnerability assessment.
```bash
# Standard build
go build -ldflags="-s -w" -trimpath -o fscan main.go
go build -ldflags="-s -w" -trimpath -o fscan .
# With Web UI
go build -tags web -ldflags="-s -w" -trimpath -o fscan main.go
go build -tags web -ldflags="-s -w" -trimpath -o fscan-web .
```
## Install
@@ -211,9 +211,13 @@ yay -S fscan-git
`fscan.exe -h 192.168.x.x -m ssh -user root -pwd password`
![](image/3.png)
`fscan.exe -h 192.168.x.x -m ssh -user root -pwda pass1 pass2 pass3` (add multiple passwords)
`fscan.exe -h 192.168.x.x -p80 -proxy http://127.0.0.1:8080`
![](image/2020-12-12-13-34-44.png)
`fscan.exe -h 192.168.x.x -socks5 socks5://user:[email protected]:1080` (SOCKS5 proxy with authentication)
`fscan.exe -h 192.168.x.x -p 139 -m netbios`
![](image/netbios.png)
+304
View File
@@ -0,0 +1,304 @@
---
name: fscan-agent
description: 使用 fscan 进行网络扫描和安全评估。当用户要求扫描网段、探测主机存活、发现开放端口、识别服务、检测漏洞或弱口令时使用。支持 NDJSON 结构化输出,适合 AI agent 管道消费。
argument-hint: <目标IP/网段> [附加参数]
allowed-tools: Bash, Read, Agent
---
# Fscan AI Agent Skill
## 工具概述
Fscan 是一款内网综合扫描工具,功能包括:
- 主机存活探测(ICMP / TCP
- 端口扫描与服务识别
- 漏洞检测(MS17-010、Redis 未授权等)
- 弱口令爆破(SSH、SMB、MySQL、MSSQL、FTP、RDP 等)
- Web 指纹识别与 POC 扫描
- NetBIOS / SMB 信息收集
- 本地信息收集(杀软检测、系统信息等)
二进制路径:当前项目编译产物 `fscan_cli`,或系统 PATH 中的 `fscan`
## 调用格式
```bash
# AI agent 标准用法:NDJSON 输出,无人类日志干扰
fscan -h <目标> -silent [其他参数]
# 解析输出
fscan -h 192.168.1.0/24 -silent | jq 'select(.type=="VULN")'
```
## 核心参数
### 目标指定
| 参数 | 说明 | 示例 |
|------|------|------|
| `-h` | 目标主机(IP / CIDR / 范围) | `-h 192.168.1.0/24` `-h 10.0.0.1-10.0.0.100` |
| `-hf` | 从文件读取目标 | `-hf targets.txt` |
| `-p` | 指定端口(逗号/范围) | `-p 22,80,443,445,3306` `-p 1-1000` |
| `-ep` | 排除端口 | `-ep 25,110` |
| `-eh` | 排除主机 | `-eh 192.168.1.1` |
| `-u` | 指定 URLWeb 扫描) | `-u https://example.com` |
| `-uf` | URL 文件 | `-uf urls.txt` |
### 扫描控制
| 参数 | 说明 | 默认值 |
|------|------|--------|
| `-m` | 扫描模式 | `all` |
| `-t` | 端口扫描线程数 | `600` |
| `-mt` | 模块线程数 | `20` |
| `-time` | 连接超时(秒) | `3` |
| `-gt` | 全局超时(秒,0 表示不限制) | `0` |
| `-np` | 跳过存活检测 | `false` |
| `-ntp` | 禁用 TCP 补充探测 | `false` |
| `-ao` | 仅存活检测 | `false` |
| `-nobr` | 禁用暴力破解 | `false` |
| `-full` | 全量 POC 扫描 | `false` |
| `-max-retries` | 最大重试次数 | `1` |
### 认证
| 参数 | 说明 |
|------|------|
| `-user` | 用户名 |
| `-pwd` | 密码 |
| `-usera` | 追加用户名 |
| `-pwda` | 追加密码,支持逗号或空格分隔多个值 |
| `-userf` | 用户名字典文件 |
| `-pwdf` | 密码字典文件 |
| `-domain` | 域名(SMB/WMI |
| `-sshkey` | SSH 私钥文件 |
| `-hash` / `-hashf` | NTLM Hash / Hash 文件 |
### 代理
| 参数 | 说明 |
|------|------|
| `-socks5` | SOCKS5 代理 (`127.0.0.1:1080`) |
| `-proxy` | HTTP 代理 (`http://127.0.0.1:8080`) |
| `-iface` | 指定本地网卡 IP(VPN 场景) |
### 输出
| 参数 | 说明 |
|------|------|
| `-silent` | 静默模式:stdout 仅输出 NDJSON |
| `-o` | 输出文件路径(默认 `result.txt` |
| `-f` | 输出格式:`txt` / `json` / `csv` |
| `-no` | 禁用文件保存 |
| `-debug` | 调试模式:日志写入 `fscan_debug.log` |
| `-log` | 日志级别(`debug` / `info` / `base` / `error` |
### 扫描模式 `-m` 的取值
| 值 | 说明 |
|------|------|
| `all` | 全部扫描(默认) |
| `icmp` | 仅 ICMP 存活检测 |
| 插件名 | 仅运行指定插件(如 `ssh``smb``ms17010``webtitle` |
## 服务插件列表
| 插件 | 默认端口 | 功能 |
|------|----------|------|
| `ftp` | 21 | FTP 弱口令 |
| `ssh` | 22 | SSH 弱口令 |
| `telnet` | 23 | Telnet 弱口令 |
| `smtp` | 25 | SMTP 弱口令 |
| `findnet` | 135 | RPC 网络信息发现(NetInfo |
| `netbios` | 139 | NetBIOS 信息收集 |
| `smb` | 445 | SMB 弱口令 |
| `ms17010` | 445 | MS17-010 永恒之蓝检测 |
| `ldap` | 389 | LDAP 弱口令 |
| `mssql` | 1433 | MSSQL 弱口令 |
| `oracle` | 1521 | Oracle 弱口令 |
| `mysql` | 3306 | MySQL 弱口令 |
| `rdp` | 3389 | RDP 弱口令 + 系统信息 |
| `postgresql` | 5432 | PostgreSQL 弱口令 |
| `vnc` | 5900 | VNC 弱口令 |
| `redis` | 6379 | Redis 未授权 + 弱口令 |
| `elasticsearch` | 9200 | ES 未授权 |
| `mongodb` | 27017 | MongoDB 未授权 + 弱口令 |
| `memcached` | 11211 | Memcached 未授权 |
| `kafka` | 9092 | Kafka 未授权 |
| `activemq` | 61616 | ActiveMQ 弱口令 |
| `rabbitmq` | 5672 | RabbitMQ 弱口令 |
| `cassandra` | 9042 | Cassandra 弱口令 |
| `neo4j` | 7687 | Neo4j 弱口令 |
| `rsync` | 873 | Rsync 未授权 |
| `webtitle` | 80/443 | Web 标题 + 指纹识别 |
| `webpoc` | 80/443 | Web 漏洞 POC |
## 本地插件(`-local`
```bash
fscan -local avdetect # 杀软检测
fscan -local systeminfo # 系统信息收集
fscan -local envinfo # 环境变量信息
fscan -local dcinfo # 域控信息
fscan -local fileinfo # 敏感文件搜索
```
## NDJSON 输出 Schema`-silent` 模式)
每行一个 JSON 对象,所有字段定义:
| 字段 | 类型 | 出现条件 | 说明 |
|------|------|----------|------|
| `type` | string | 必有 | `HOST` / `PORT` / `SERVICE` / `VULN` |
| `target` | string | 必有 | 原始目标 `host``host:port` |
| `status` | string | 必有 | 状态描述 |
| `host` | string | 必有 | IP 地址 |
| `port` | int | PORT/SERVICE/VULN | 端口号 |
| `service` | string | SERVICE/VULN | 服务名(ssh, smb, http 等) |
| `protocol` | string | HOST/SERVICE | 协议(ICMP, TCP, http, https |
| `banner` | string | SERVICE | 服务 Banner |
| `title` | string | SERVICE (web) | 网页标题 |
| `url` | string | SERVICE (web) | 完整 URL |
| `vulnerability` | string | VULN | 漏洞名称 |
| `username` | string | VULN (弱口令) | 用户名 |
| `password` | string | VULN (弱口令) | 密码 |
| `plugin` | string | SERVICE/VULN | 产生结果的插件名 |
| `version` | string | SERVICE | 服务版本号 |
| `os` | string | SERVICE | 操作系统信息 |
### 输出示例
```jsonl
{"type":"HOST","target":"192.168.1.5","status":"alive","host":"192.168.1.5","protocol":"ICMP"}
{"type":"PORT","target":"192.168.1.5","status":"open","host":"192.168.1.5","port":22}
{"type":"PORT","target":"192.168.1.5","status":"open","host":"192.168.1.5","port":445}
{"type":"SERVICE","target":"192.168.1.5:22","status":"identified","host":"192.168.1.5","port":22,"service":"ssh","banner":"SSH-2.0-OpenSSH_8.9p1","version":"8.9p1","plugin":"portscan"}
{"type":"SERVICE","target":"192.168.1.5:80","status":"web","host":"192.168.1.5","port":80,"service":"http","protocol":"http","url":"http://192.168.1.5:80","title":"Welcome","plugin":"webtitle"}
{"type":"VULN","target":"192.168.1.5:445","status":"MS17-010 (Windows Server 2012 R2 Standard 9600)","host":"192.168.1.5","port":445,"vulnerability":"MS17-010","service":"smb","plugin":"ms17010"}
{"type":"VULN","target":"192.168.1.5:22","status":"weak_credential: root:123456","host":"192.168.1.5","port":22,"service":"ssh","username":"root","password":"123456","plugin":"ssh"}
{"type":"VULN","target":"192.168.1.5:6379","status":"Redis unauthorized","host":"192.168.1.5","port":6379,"vulnerability":"Redis unauthorized access","service":"redis","plugin":"redis"}
```
### 结果产出顺序
1. `HOST` — 存活探测阶段
2. `PORT` — 端口扫描阶段(与 SERVICE 可能交错)
3. `SERVICE` — 服务识别阶段
4. `VULN` — 漏洞/弱口令检测阶段
同一 `host:port` 可产生多条结果(PORT + SERVICE + VULN)。
## 常用场景参数组合
### 全网段快速扫描
```bash
fscan -h 192.168.1.0/24 -silent
```
### 跳过存活检测直接扫端口(目标明确时)
```bash
fscan -h 192.168.1.0/24 -silent -np
```
### 指定端口精确扫描
```bash
fscan -h 10.0.0.0/24 -silent -p 22,80,443,445,3389,3306,6379
```
### 仅存活探测
```bash
fscan -h 172.16.0.0/16 -silent -m icmp
```
### 低速隐蔽扫描
```bash
fscan -h 192.168.1.0/24 -silent -t 30 -time 5
```
### 通过 SOCKS5 代理扫描内网
```bash
fscan -h 10.0.0.0/24 -silent -socks5 127.0.0.1:1080
```
### 仅做弱口令检测
```bash
fscan -h 192.168.1.10 -silent -m ssh -user root -pwdf /path/to/passwords.txt
```
### Web 目标扫描
```bash
fscan -u https://target.com -silent -full
```
### 多目标文件批量扫描
```bash
fscan -hf targets.txt -silent -o results.json -f json
```
### 带调试日志的排障扫描
```bash
# NDJSON 到 stdoutdebug 日志到文件,互不干扰
fscan -h 192.168.1.0/24 -silent -debug
# 事后查看:cat fscan_debug.log
```
## AI Agent 结果处理
### Python 管道消费
```python
import json, subprocess
proc = subprocess.Popen(
["fscan", "-h", "192.168.1.0/24", "-silent"],
stdout=subprocess.PIPE, text=True
)
hosts, services, vulns = [], [], []
for line in proc.stdout:
r = json.loads(line)
if r["type"] == "HOST":
hosts.append(r["host"])
elif r["type"] == "SERVICE":
services.append(r)
elif r["type"] == "VULN":
vulns.append(r)
proc.wait()
```
### jq 过滤
```bash
# 提取所有弱口令
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.username != null) | "\(.host):\(.port) \(.service) \(.username):\(.password)"'
# 提取所有漏洞
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.type=="VULN") | "\(.host):\(.port) \(.vulnerability)"'
# 提取 Web 服务
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.url != null) | "\(.url) \(.title)"'
# 统计开放端口
fscan -h 10.0.0.0/24 -silent | jq -r 'select(.type=="PORT") | .port' | sort -n | uniq -c | sort -rn
```
## 注意事项
- `-silent` 抑制所有人类可读日志,stdout 仅输出 NDJSON
- 空字段不出现在 JSON 中(`omitempty`
- 进程退出码 `0` 正常完成,非 `0` 表示参数错误或初始化失败
- `-silent``-debug` 可同时使用,互不干扰
- SOCKS5 代理下 fscan 信任协议层连接结果,不做额外深度验证
- 扫描大网段时线程数会自动调整,资源耗尽时自适应降级
- 默认超时 3 秒,防火墙 drop 的端口会静默超时,不计入失败率
+2 -2
View File
@@ -36,7 +36,7 @@ var (
"6000,6379,6380,6443,6666,6667," +
// Web/中间件 (7001-9999)
// 注意:9100 已移除,该端口为打印机 RAW 端口,发送数据会触发打印
"7001,7002,7474,7687,8000,8005,8008,8009,8080,8081,8086,8088,8089,8090,8161,8180,8443,8500,8834,8848,8880,8888,9000,9001,9042,9080,9090,9092,9093,9160,9200,9300,9418,9443,9999," +
"7001,7002,7474,7687,8000,8005,8008,8009,8080,8081,8086,8088,8089,8090,8161,8180,8443,8500,8834,8848,8880,8883,8888,9000,9001,9042,9080,9090,9092,9093,9160,9200,9300,9418,9443,9999," +
// 管理/监控 (10000-11211)
"10000,10051,10250,10255,11211," +
// 消息队列/集群 (15672-27018)
@@ -48,7 +48,7 @@ var (
DbPorts = "1433,1521,3306,5432,5672,5984,6379,7687,8086,9042,9093,9160,9200,11211,26379,27017,27018,61616"
// ServicePorts 服务端口
ServicePorts = "21,22,23,25,53,110,111,135,139,143,161,389,445,465,502,512,513,514,587,623,636,873,993,995,1433,1521,2049,2181,2222,3306,3389,5432,5672,5671,5900,5985,5986,6379,8161,8443,9000,9092,9093,9200,10051,11211,15672,15671,27017,61616,61613"
ServicePorts = "21,22,23,25,53,110,111,135,139,143,161,389,445,465,502,512,513,514,587,623,636,873,993,995,1433,1521,1883,2049,2181,2222,3306,3389,5432,5672,5671,5900,5985,5986,6379,8161,8443,8883,9000,9092,9093,9200,10051,11211,15672,15671,27017,61616,61613"
// CommonPorts 常用端口
CommonPorts = "21,22,23,25,53,80,110,135,139,143,443,445,993,995,1723,3389,5060,5985,5986"
+103 -23
View File
@@ -4,10 +4,12 @@ import (
"encoding/hex"
"fmt"
"net"
"net/url"
"strconv"
"strings"
"github.com/shadow1ng/fscan/common/config"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/parsers"
)
@@ -28,12 +30,12 @@ func BuildConfig(fv *FlagVars, info *HostInfo) (*Config, *State, error) {
// 3. 解析凭据
if err := parseCredentials(fv, cfg); err != nil {
return nil, nil, fmt.Errorf("凭据解析失败: %w", err)
return nil, nil, fmt.Errorf("%s: %w", i18n.GetText("config_credentials_parse_failed"), err)
}
// 4. 解析目标(主机、端口、URL)
if err := parseTargets(fv, info, cfg, state); err != nil {
return nil, nil, fmt.Errorf("目标解析失败: %w", err)
return nil, nil, fmt.Errorf("%s: %w", i18n.GetText("config_targets_parse_failed"), err)
}
// 5. 应用日志级别
@@ -48,7 +50,10 @@ func BuildConfig(fv *FlagVars, info *HostInfo) (*Config, *State, error) {
func parseCredentials(fv *FlagVars, cfg *Config) error {
// 解析用户名
usernames := parseUsernames(fv)
usernames, err := parseUsernames(fv)
if err != nil {
return err
}
if len(usernames) > 0 {
for serviceName := range cfg.Credentials.Userdict {
cfg.Credentials.Userdict[serviceName] = usernames
@@ -56,7 +61,10 @@ func parseCredentials(fv *FlagVars, cfg *Config) error {
}
// 解析密码
passwords := parsePasswords(fv)
passwords, err := parsePasswords(fv)
if err != nil {
return err
}
if len(passwords) > 0 {
cfg.Credentials.Passwords = passwords
}
@@ -83,7 +91,7 @@ func parseCredentials(fv *FlagVars, cfg *Config) error {
return nil
}
func parseUsernames(fv *FlagVars) []string {
func parseUsernames(fv *FlagVars) ([]string, error) {
var usernames []string
// 命令行用户名
@@ -100,6 +108,8 @@ func parseUsernames(fv *FlagVars) []string {
if fv.UsersFile != "" {
if lines, err := parsers.ReadLinesFromFile(fv.UsersFile); err == nil {
usernames = append(usernames, lines...)
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_users_failed", fv.UsersFile, err))
}
}
@@ -113,30 +123,52 @@ func parseUsernames(fv *FlagVars) []string {
}
}
return removeDuplicate(usernames)
return removeDuplicate(usernames), nil
}
func parsePasswords(fv *FlagVars) []string {
func parsePasswords(fv *FlagVars) ([]string, error) {
var passwords []string
// 命令行密码
// 命令行密码(支持逗号分隔多个值,保留空格作为密码的一部分)
if fv.Password != "" {
passwords = append(passwords, strings.Split(fv.Password, ",")...)
for _, p := range strings.Split(fv.Password, ",") {
p = strings.TrimSpace(p)
if p != "" {
passwords = append(passwords, p)
}
}
}
// 从文件读取
if fv.PasswordsFile != "" {
if lines, err := parsers.ReadLinesFromFile(fv.PasswordsFile); err == nil {
passwords = append(passwords, lines...)
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_passwords_failed", fv.PasswordsFile, err))
}
}
// 额外密码
if fv.AddPasswords != "" {
passwords = append(passwords, strings.Split(fv.AddPasswords, ",")...)
passwords = append(passwords, splitCredentialValues(fv.AddPasswords)...)
}
return removeDuplicate(passwords)
return removeDuplicate(passwords), nil
}
func splitCredentialValues(input string) []string {
fields := strings.FieldsFunc(input, func(r rune) bool {
return r == ',' || r == ' ' || r == '\t' || r == '\n' || r == '\r'
})
values := make([]string, 0, len(fields))
for _, field := range fields {
field = strings.TrimSpace(field)
if field != "" {
values = append(values, field)
}
}
return values
}
func parseUserPassPairs(fv *FlagVars) ([]config.CredentialPair, error) {
@@ -145,6 +177,7 @@ func parseUserPassPairs(fv *FlagVars) ([]config.CredentialPair, error) {
// 如果命令行同时指定了单个用户名和单个密码(不是逗号分隔的多个)
if fv.Username != "" && fv.Password != "" &&
!strings.Contains(fv.Username, ",") && !strings.Contains(fv.Password, ",") &&
fv.AddUsers == "" && fv.AddPasswords == "" &&
fv.UsersFile == "" && fv.PasswordsFile == "" && fv.UserPassFile == "" {
pairs = append(pairs, config.CredentialPair{
Username: strings.TrimSpace(fv.Username),
@@ -169,15 +202,22 @@ func parseHashes(fv *FlagVars) ([]string, [][]byte, error) {
var hashValues []string
var hashBytes [][]byte
// 命令行哈希
// 命令行哈希(支持纯 NTLM 32字符 或 LM:NT 格式)
if fv.HashValue != "" {
hash := strings.TrimSpace(fv.HashValue)
if len(hash) == 32 {
hashValues = append(hashValues, hash)
if hashByte, err := hex.DecodeString(hash); err == nil {
hashBytes = append(hashBytes, hashByte)
}
// LM:NT 格式取 NT hash 部分
if parts := strings.SplitN(hash, ":", 2); len(parts) == 2 && len(parts[1]) == 32 {
hash = parts[1]
}
if len(hash) != 32 {
return nil, nil, fmt.Errorf("invalid hash length: %s", fv.HashValue)
}
hashByte, err := hex.DecodeString(hash)
if err != nil {
return nil, nil, err
}
hashValues = append(hashValues, hash)
hashBytes = append(hashBytes, hashByte)
}
// 从文件读取
@@ -205,13 +245,17 @@ func parseTargets(fv *FlagVars, info *HostInfo, cfg *Config, state *State) error
if port, portErr := strconv.Atoi(portStr); portErr == nil && port >= 1 && port <= 65535 {
// 有效的 host:port 格式
state.SetHostPorts([]string{info.Host})
info.Host = ""
ports = "" // 清空端口,避免双重扫描
}
}
}
// 解析 URL
urls := parseURLs(fv)
urls, err := parseURLs(fv)
if err != nil {
return err
}
if len(urls) > 0 {
state.SetURLs(urls)
if info.URL == "" && len(urls) == 1 {
@@ -227,7 +271,7 @@ func parseTargets(fv *FlagVars, info *HostInfo, cfg *Config, state *State) error
return nil
}
func parseURLs(fv *FlagVars) []string {
func parseURLs(fv *FlagVars) ([]string, error) {
var urls []string
// 命令行 URL
@@ -246,10 +290,12 @@ func parseURLs(fv *FlagVars) []string {
for _, line := range lines {
urls = append(urls, normalizeURL(line))
}
} else {
return nil, fmt.Errorf("%s", i18n.Tr("config_read_urls_failed", fv.URLsFile, err))
}
}
return removeDuplicate(urls)
return removeDuplicate(urls), nil
}
func normalizeURL(rawURL string) string {
@@ -257,10 +303,44 @@ func normalizeURL(rawURL string) string {
if rawURL == "" {
return rawURL
}
if !strings.HasPrefix(rawURL, "http://") && !strings.HasPrefix(rawURL, "https://") {
return "http://" + rawURL
lowerURL := strings.ToLower(rawURL)
if !strings.HasPrefix(lowerURL, "http://") && !strings.HasPrefix(lowerURL, "https://") {
return "http://" + normalizeSchemelessURLTarget(rawURL)
}
return rawURL
parsed, err := url.Parse(rawURL)
if err != nil || parsed.Host == "" {
return rawURL
}
normalizedHost := normalizeURLHost(parsed.Host)
if normalizedHost == parsed.Host {
return rawURL
}
parsed.Host = normalizedHost
normalized := parsed.String()
if schemeEnd := strings.Index(rawURL, "://"); schemeEnd >= 0 {
return rawURL[:schemeEnd] + normalized[len(parsed.Scheme):]
}
return normalized
}
func normalizeSchemelessURLTarget(rawURL string) string {
authority := rawURL
suffix := ""
if idx := strings.IndexAny(rawURL, "/?#"); idx >= 0 {
authority = rawURL[:idx]
suffix = rawURL[idx:]
}
return normalizeURLHost(authority) + suffix
}
func normalizeURLHost(host string) string {
if strings.HasPrefix(host, "[") {
return host
}
if ip := net.ParseIP(host); ip != nil && strings.Contains(host, ":") {
return "[" + host + "]"
}
return host
}
// =============================================================================
+450
View File
@@ -0,0 +1,450 @@
package common
import (
"reflect"
"testing"
"time"
fscanconfig "github.com/shadow1ng/fscan/common/config"
)
func TestParsePasswordsKeepsPrimaryPasswordLiteral(t *testing.T) {
fv := &FlagVars{
Password: "root admin,pass0",
AddPasswords: "pass1 pass2,pass3\tpass4",
}
got, err := parsePasswords(fv)
if err != nil {
t.Fatalf("parsePasswords error = %v", err)
}
// -pwd 逗号分隔,空格保留;-pwda 逗号/空格/tab 分隔
want := []string{"root admin", "pass0", "pass1", "pass2", "pass3", "pass4"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("parsePasswords() = %#v, want %#v", got, want)
}
}
func TestBuildConfigReturnsUserFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{UsersFile: "missing-users-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing users file")
}
}
func TestBuildConfigReturnsPasswordFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{PasswordsFile: "missing-passwords-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing passwords file")
}
}
func TestBuildConfigReturnsURLFileError(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{URLsFile: "missing-urls-file.txt"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for missing urls file")
}
}
func TestBuildConfigRejectsInvalidHashValue(t *testing.T) {
_, _, err := BuildConfig(&FlagVars{HashValue: "not-md5"}, &HostInfo{})
if err == nil {
t.Fatal("BuildConfig should fail for invalid hash value")
}
}
func TestBuildConfigDefaultsAreIndependentCopies(t *testing.T) {
cfg, _, err := BuildConfig(&FlagVars{Username: "custom-user"}, &HostInfo{})
if err != nil {
t.Fatalf("BuildConfig error = %v", err)
}
defaultSSHUsers := fscanconfig.DefaultUserDict["ssh"]
if len(defaultSSHUsers) == 1 && defaultSSHUsers[0] == "custom-user" {
t.Fatal("BuildConfig mutated DefaultUserDict")
}
cfg.Credentials.Userdict["ssh"][0] = "mutated-user"
if fscanconfig.DefaultUserDict["ssh"][0] == "mutated-user" {
t.Fatal("Config userdict shares backing storage with DefaultUserDict")
}
cfg.Credentials.Passwords[0] = "mutated-password"
if fscanconfig.DefaultPasswords[0] == "mutated-password" {
t.Fatal("Config passwords share backing storage with DefaultPasswords")
}
port := 80
cfg.PortMap[port][0] = "mutated-probe"
if fscanconfig.DefaultPortMap[port][0] == "mutated-probe" {
t.Fatal("Config port map shares backing storage with DefaultPortMap")
}
cfg.DefaultMap[0] = "mutated-default-probe"
if fscanconfig.DefaultProbeMap[0] == "mutated-default-probe" {
t.Fatal("Config default map shares backing storage with DefaultProbeMap")
}
}
func TestParseUserPassPairsKeepsAdditionalCredentialFlags(t *testing.T) {
tests := []struct {
name string
fv *FlagVars
}{
{
name: "additional passwords",
fv: &FlagVars{
Username: "root",
Password: "primary",
AddPasswords: "extra",
},
},
{
name: "additional users",
fv: &FlagVars{
Username: "root",
Password: "primary",
AddUsers: "admin",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pairs, err := parseUserPassPairs(tt.fv)
if err != nil {
t.Fatalf("parseUserPassPairs error = %v", err)
}
if len(pairs) != 0 {
t.Fatalf("parseUserPassPairs returned exact pairs %#v; additional credential flags would be ignored", pairs)
}
})
}
}
func TestNewConfigDefaultsAreIndependentCopies(t *testing.T) {
cfg := NewConfig()
cfg.Credentials.Userdict["ssh"][0] = "mutated-user"
if fscanconfig.DefaultUserDict["ssh"][0] == "mutated-user" {
t.Fatal("NewConfig userdict shares backing storage with DefaultUserDict")
}
cfg.Credentials.Passwords[0] = "mutated-password"
if fscanconfig.DefaultPasswords[0] == "mutated-password" {
t.Fatal("NewConfig passwords share backing storage with DefaultPasswords")
}
port := 80
cfg.PortMap[port][0] = "mutated-probe"
if fscanconfig.DefaultPortMap[port][0] == "mutated-probe" {
t.Fatal("NewConfig port map shares backing storage with DefaultPortMap")
}
cfg.DefaultMap[0] = "mutated-default-probe"
if fscanconfig.DefaultProbeMap[0] == "mutated-default-probe" {
t.Fatal("NewConfig default map shares backing storage with DefaultProbeMap")
}
}
func TestParseTargetsHostPortDoesNotLeaveSyntheticHost(t *testing.T) {
fv := &FlagVars{Ports: "22"}
info := &HostInfo{Host: "127.0.0.1:8080"}
cfg := BuildConfigFromFlags(fv)
state := NewState()
if err := parseTargets(fv, info, cfg, state); err != nil {
t.Fatalf("parseTargets error = %v", err)
}
if info.Host != "" {
t.Fatalf("info.Host = %q, want empty after host:port extraction", info.Host)
}
if got := state.GetHostPorts(); !reflect.DeepEqual(got, []string{"127.0.0.1:8080"}) {
t.Fatalf("hostPorts = %#v, want host:port target", got)
}
}
func TestNormalizeURLKeepsUppercaseScheme(t *testing.T) {
got := normalizeURL("HTTPS://example.com")
if got != "HTTPS://example.com" {
t.Fatalf("normalizeURL() = %q", got)
}
}
func TestNormalizeURLBracketsIPv6Literals(t *testing.T) {
tests := []struct {
name string
in string
want string
}{
{name: "bare ipv6 without scheme", in: "2001:db8::1", want: "http://[2001:db8::1]"},
{name: "bracketed ipv6 without scheme", in: "[2001:db8::1]", want: "http://[2001:db8::1]"},
{name: "bare ipv6 with scheme", in: "http://2001:db8::1", want: "http://[2001:db8::1]"},
{name: "bare ipv6 path without scheme", in: "2001:db8::1/admin", want: "http://[2001:db8::1]/admin"},
{name: "bare ipv6 query without scheme", in: "2001:db8::1?debug=1", want: "http://[2001:db8::1]?debug=1"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := normalizeURL(tt.in); got != tt.want {
t.Fatalf("normalizeURL(%q) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
// TestModuleTimeout 测试模块超时计算
func TestModuleTimeout(t *testing.T) {
tests := []struct {
name string
timeout time.Duration
want time.Duration
}{
{"超时大于下限", 10 * time.Second, 10 * time.Second},
{"超时等于下限", 3 * time.Second, 3 * time.Second},
{"超时小于下限", 1 * time.Second, 3 * time.Second},
{"零超时", 0, 3 * time.Second},
{"负超时", -1 * time.Second, 3 * time.Second},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := NewConfig()
cfg.Timeout = tt.timeout
got := cfg.ModuleTimeout()
if got != tt.want {
t.Errorf("ModuleTimeout() = %v, want %v", got, tt.want)
}
})
}
}
// TestParseUserPassPairsExactMatch 测试精确单用户单密码路径
func TestParseUserPassPairsExactMatch(t *testing.T) {
fv := &FlagVars{
Username: "admin",
Password: "secret",
}
pairs, err := parseUserPassPairs(fv)
if err != nil {
t.Fatalf("parseUserPassPairs error = %v", err)
}
if len(pairs) != 1 {
t.Fatalf("期望 1 个 pair, 实际 %d", len(pairs))
}
if pairs[0].Username != "admin" || pairs[0].Password != "secret" {
t.Errorf("pair = %+v, want {admin secret}", pairs[0])
}
}
// TestParseUserPassPairsMultiUserSkips 测试多用户时不生成精确 pair
func TestParseUserPassPairsMultiUserSkips(t *testing.T) {
fv := &FlagVars{
Username: "admin,root",
Password: "pass",
}
pairs, err := parseUserPassPairs(fv)
if err != nil {
t.Fatalf("parseUserPassPairs error = %v", err)
}
if len(pairs) != 0 {
t.Fatalf("多用户场景不应生成精确 pair, 实际 %d 个", len(pairs))
}
}
// TestParseURLsEmpty 测试空输入返回空列表
func TestParseURLsEmpty(t *testing.T) {
fv := &FlagVars{}
urls, err := parseURLs(fv)
if err != nil {
t.Fatalf("parseURLs error = %v", err)
}
if len(urls) != 0 {
t.Fatalf("空输入应返回空 url 列表, 实际 %v", urls)
}
}
// TestParseURLsCommaSeparated 测试逗号分隔多 URL
func TestParseURLsCommaSeparated(t *testing.T) {
fv := &FlagVars{
TargetURL: "http://a.com,http://b.com,http://a.com", // 含重复
}
urls, err := parseURLs(fv)
if err != nil {
t.Fatalf("parseURLs error = %v", err)
}
if len(urls) != 2 {
t.Fatalf("去重后应有 2 个 url, 实际 %d: %v", len(urls), urls)
}
}
// TestParseURLsMissingFile 测试缺失文件返回错误
func TestParseURLsMissingFile(t *testing.T) {
fv := &FlagVars{URLsFile: "nonexistent-urls.txt"}
_, err := parseURLs(fv)
if err == nil {
t.Fatal("缺失文件应返回错误")
}
}
// ---------------------------------------------------------------------------
// parseHashes
// ---------------------------------------------------------------------------
// TestParseHashesEmpty 空输入返回空结果
func TestParseHashesEmpty(t *testing.T) {
fv := &FlagVars{}
vals, bytes, err := parseHashes(fv)
if err != nil {
t.Fatalf("parseHashes error = %v", err)
}
if len(vals) != 0 || len(bytes) != 0 {
t.Fatalf("空输入应返回空结果, vals=%v bytes=%v", vals, bytes)
}
}
// TestParseHashesValidNTLM 纯 32 字符 hex hash
func TestParseHashesValidNTLM(t *testing.T) {
hash := "aabbccddeeff00112233445566778899"
fv := &FlagVars{HashValue: hash}
vals, hashBytes, err := parseHashes(fv)
if err != nil {
t.Fatalf("parseHashes error = %v", err)
}
if len(vals) != 1 || vals[0] != hash {
t.Fatalf("vals = %v, want [%s]", vals, hash)
}
if len(hashBytes) != 1 || len(hashBytes[0]) != 16 {
t.Fatalf("hashBytes length wrong: %v", hashBytes)
}
}
// TestParseHashesLMNTFormat LM:NT 格式,提取 NT 部分
func TestParseHashesLMNTFormat(t *testing.T) {
lm := "aad3b435b51404eeaad3b435b51404ee"
nt := "31d6cfe0d16ae931b73c59d7e0c089c0"
fv := &FlagVars{HashValue: lm + ":" + nt}
vals, _, err := parseHashes(fv)
if err != nil {
t.Fatalf("parseHashes error = %v", err)
}
if len(vals) != 1 || vals[0] != nt {
t.Fatalf("vals = %v, want [%s]", vals, nt)
}
}
// TestParseHashesInvalidLength hash 长度不是 32 → error
func TestParseHashesInvalidLength(t *testing.T) {
fv := &FlagVars{HashValue: "tooshort"}
_, _, err := parseHashes(fv)
if err == nil {
t.Fatal("hash 长度不足应返回错误")
}
}
// TestParseHashesInvalidHex 32 字符但含非 hex 字符 → error
func TestParseHashesInvalidHex(t *testing.T) {
fv := &FlagVars{HashValue: "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"}
_, _, err := parseHashes(fv)
if err == nil {
t.Fatal("非 hex 字符应返回错误")
}
}
// TestParseHashesMissingFile hash 文件不存在 → error
func TestParseHashesMissingFile(t *testing.T) {
fv := &FlagVars{HashFile: "nonexistent-hashes.txt"}
_, _, err := parseHashes(fv)
if err == nil {
t.Fatal("缺失 hash 文件应返回错误")
}
}
// ---------------------------------------------------------------------------
// parseUsernames
// ---------------------------------------------------------------------------
// TestParseUsernamesEmpty 空输入返回空结果
func TestParseUsernamesEmpty(t *testing.T) {
fv := &FlagVars{}
got, err := parseUsernames(fv)
if err != nil {
t.Fatalf("parseUsernames error = %v", err)
}
if len(got) != 0 {
t.Fatalf("空输入应返回空, got %v", got)
}
}
// TestParseUsernamesCommaSeparated 逗号分隔多用户
func TestParseUsernamesCommaSeparated(t *testing.T) {
fv := &FlagVars{Username: "admin, root, admin"} // 含重复和空格
got, err := parseUsernames(fv)
if err != nil {
t.Fatalf("parseUsernames error = %v", err)
}
want := []string{"admin", "root"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
}
// TestParseUsernamesAddUsers AddUsers 追加去重
func TestParseUsernamesAddUsers(t *testing.T) {
fv := &FlagVars{
Username: "admin",
AddUsers: "root,admin", // admin 重复
}
got, err := parseUsernames(fv)
if err != nil {
t.Fatalf("parseUsernames error = %v", err)
}
want := []string{"admin", "root"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
}
// TestParseUsernamesMissingFile 缺失用户文件 → error
func TestParseUsernamesMissingFile(t *testing.T) {
fv := &FlagVars{UsersFile: "nonexistent-users.txt"}
_, err := parseUsernames(fv)
if err == nil {
t.Fatal("缺失用户文件应返回错误")
}
}
// ---------------------------------------------------------------------------
// cloneStringSlice
// ---------------------------------------------------------------------------
// TestCloneStringSliceNil nil 输入返回 nil
func TestCloneStringSliceNil(t *testing.T) {
got := cloneStringSlice(nil)
if got != nil {
t.Fatalf("nil 输入应返回 nil, got %v", got)
}
}
// TestCloneStringSliceEmpty 空切片:append 无元素结果为 nillen 为 0
func TestCloneStringSliceEmpty(t *testing.T) {
got := cloneStringSlice([]string{})
if len(got) != 0 {
t.Fatalf("got len %d, want 0", len(got))
}
}
// TestCloneStringSliceCopiesValues 正常切片:值正确且独立
func TestCloneStringSliceCopiesValues(t *testing.T) {
src := []string{"a", "b", "c"}
got := cloneStringSlice(src)
if !reflect.DeepEqual(got, src) {
t.Fatalf("got %v, want %v", got, src)
}
// 修改 clone 不影响原始
got[0] = "mutated"
if src[0] != "a" {
t.Fatal("cloneStringSlice 返回的切片与源共享底层数组")
}
}
+84 -27
View File
@@ -22,18 +22,26 @@ config_struct.go - 配置结构体定义
// Config 扫描器完整配置 - 初始化后只读,可安全共享
type Config struct {
// 高频访问字段 - 平铺到顶层
Timeout time.Duration // 通用超时
ThreadNum int // 主线程数
ModuleThreadNum int // 模块线程数
DisableBrute bool // 禁用暴力破解
DisablePing bool // 禁用Ping检测
Timeout time.Duration // 通用超时
TimeoutExplicit bool // 用户显式指定了 -time
ThreadNum int // 线程数
ThreadCeiling int // 线程数上限(自适应池允许的最大值)
ThreadNumExplicit bool // 用户显式指定了 -t
ModuleThreadNum int // 模块线程数
ModuleThreadNumExplicit bool // 用户显式指定了 -mt
DisableBrute bool // 禁用暴力破解
DisablePing bool // 禁用Ping检测
DisableTcpProbe bool // 禁用TCP补充探测
DisableSubnetProbe bool // 禁用网段预筛
// 扫描模式
Mode string // 扫描模式
LocalMode bool // 本地模式
LocalPlugin string // 本地插件名
AliveOnly bool // 仅存活检测
MaxRetries int // 最大重试次数
Mode string // 扫描模式
LocalMode bool // 本地模式
LocalPlugin string // 本地插件名
AliveOnly bool // 仅存活检测
MaxRetries int // 最大重试次数
MaxRetriesExplicit bool // 用户显式指定了 -retry
DetectedNetworkEnv int // 探测到的网络环境(来自 core.NetworkEnv
// 高级功能(从AdvancedConfig合并)
Shellcode string // Shellcode
@@ -54,6 +62,10 @@ type Config struct {
LocalExploit LocalExploitConfig
Target TargetConfig // 扫描目标配置
// 全局超时
GlobalTimeout time.Duration
GlobalTimeoutExplicit bool
// SOCKS5代理端口配置
Socks5ProxyPort int // SOCKS5代理端口
}
@@ -79,14 +91,15 @@ type CredentialConfig struct {
// NetworkConfig 网络相关配置
type NetworkConfig struct {
HTTPProxy string
Socks5Proxy string
Iface string
WebTimeout time.Duration
MaxRedirects int
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
HTTPProxy string
Socks5Proxy string
Iface string
WebTimeout time.Duration
MaxRedirects int
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
ICMPRateExplicit bool
}
// OutputConfig 输出相关配置
@@ -105,11 +118,12 @@ type OutputConfig struct {
// POCConfig POC扫描相关配置
type POCConfig struct {
PocPath string // POC路径
PocName string // 指定POC名称
Full bool // 完整POC扫描
Num int // POC并发数
Disabled bool // 禁用POC扫描
PocPath string // POC路径
PocName string // 指定POC名称
Full bool // 完整POC扫描
Num int // POC并发数
NumExplicit bool // 用户显式指定了 -num
Disabled bool // 禁用POC扫描
}
// RedisConfig Redis利用相关配置
@@ -138,15 +152,58 @@ type LocalExploitConfig struct {
DownloadSavePath string // 下载保存路径
}
func cloneStringSlice(values []string) []string {
if values == nil {
return nil
}
return append([]string(nil), values...)
}
func cloneStringSliceMap(values map[string][]string) map[string][]string {
if values == nil {
return nil
}
cloned := make(map[string][]string, len(values))
for key, value := range values {
cloned[key] = cloneStringSlice(value)
}
return cloned
}
func clonePortMap(values map[int][]string) map[int][]string {
if values == nil {
return nil
}
cloned := make(map[int][]string, len(values))
for key, value := range values {
cloned[key] = cloneStringSlice(value)
}
return cloned
}
const minModuleTimeout = 3 * time.Second
// ModuleTimeout 返回插件级超时(用于弱口令测试、服务交互等多轮协议)
// 保证下限 3s,避免自适应把端口扫描超时压低后影响 SSH/SNMP 等交互型协议
func (c *Config) ModuleTimeout() time.Duration {
if c.Timeout >= minModuleTimeout {
return c.Timeout
}
return minModuleTimeout
}
// NewConfig 创建带默认值的Config(后备用,正常流程使用BuildConfigFromFlags
func NewConfig() *Config {
return &Config{
// 高频字段 - 使用默认常量
Timeout: time.Duration(DefaultTimeout) * time.Second,
ThreadNum: DefaultThreadNum,
ThreadCeiling: DefaultThreadNum,
ModuleThreadNum: 10,
DisableBrute: false,
DisablePing: false,
DisableTcpProbe: false,
DisableSubnetProbe: false,
// 扫描模式
Mode: DefaultScanMode,
@@ -155,13 +212,13 @@ func NewConfig() *Config {
MaxRetries: 3,
// 高级功能 - 使用默认配置
PortMap: config.DefaultPortMap,
DefaultMap: config.DefaultProbeMap,
PortMap: clonePortMap(config.DefaultPortMap),
DefaultMap: cloneStringSlice(config.DefaultProbeMap),
// 分组配置 - 使用默认字典
Credentials: CredentialConfig{
Userdict: config.DefaultUserDict,
Passwords: config.DefaultPasswords,
Userdict: cloneStringSliceMap(config.DefaultUserDict),
Passwords: cloneStringSlice(config.DefaultPasswords),
UserPassPairs: nil,
},
Network: NetworkConfig{
+24 -22
View File
@@ -9,6 +9,8 @@ import (
"runtime"
"runtime/pprof"
"runtime/trace"
"github.com/shadow1ng/fscan/common/i18n"
)
var (
@@ -19,82 +21,82 @@ var (
func Start() {
if err := os.MkdirAll(profilesPath, 0755); err != nil {
fmt.Printf("[DEBUG] 创建 profiles 目录失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_profiles_failed", err))
return
}
var err error
cpuProfile, err = os.Create(profilesPath + "/cpu.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建 CPU profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_cpu_profile_failed", err))
} else {
if err := pprof.StartCPUProfile(cpuProfile); err != nil {
fmt.Printf("[DEBUG] 启动 CPU profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_start_cpu_profile_failed", err))
cpuProfile.Close()
cpuProfile = nil
} else {
fmt.Printf("[DEBUG] CPU profiling 已启动 -> %s/cpu.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_cpu_profile_started", profilesPath))
}
}
traceFile, err = os.Create(profilesPath + "/trace.out")
if err != nil {
fmt.Printf("[DEBUG] 创建 trace 文件失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_trace_failed", err))
} else {
if err := trace.Start(traceFile); err != nil {
fmt.Printf("[DEBUG] 启动 trace 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_start_trace_failed", err))
traceFile.Close()
traceFile = nil
} else {
fmt.Printf("[DEBUG] Execution trace 已启动 -> %s/trace.out\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_trace_started", profilesPath))
}
}
fmt.Printf("[DEBUG] 性能分析已启动,程序结束时自动保存到 %s/\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_profiling_started", profilesPath))
}
func Stop() {
if cpuProfile != nil {
pprof.StopCPUProfile()
cpuProfile.Close()
fmt.Printf("[DEBUG] CPU profile 已保存\n")
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_cpu_profile_saved"))
}
if traceFile != nil {
trace.Stop()
traceFile.Close()
fmt.Printf("[DEBUG] Trace 已保存\n")
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_trace_saved"))
}
memProfile, err := os.Create(profilesPath + "/mem.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建内存 profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_mem_profile_failed", err))
} else {
runtime.GC()
if err := pprof.WriteHeapProfile(memProfile); err != nil {
fmt.Printf("[DEBUG] 写入内存 profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_write_mem_profile_failed", err))
} else {
fmt.Printf("[DEBUG] 内存 profile 已保存 -> %s/mem.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_mem_profile_saved", profilesPath))
}
memProfile.Close()
}
goroutineProfile, err := os.Create(profilesPath + "/goroutine.prof")
if err != nil {
fmt.Printf("[DEBUG] 创建 goroutine profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_create_goroutine_profile_failed", err))
} else {
if err := pprof.Lookup("goroutine").WriteTo(goroutineProfile, 0); err != nil {
fmt.Printf("[DEBUG] 写入 goroutine profile 失败: %v\n", err)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_write_goroutine_profile_failed", err))
} else {
fmt.Printf("[DEBUG] Goroutine profile 已保存 -> %s/goroutine.prof\n", profilesPath)
fmt.Printf("[DEBUG] %s\n", i18n.Tr("debug_goroutine_profile_saved", profilesPath))
}
goroutineProfile.Close()
}
fmt.Printf("\n[DEBUG] 所有性能分析文件已保存到 %s/\n", profilesPath)
fmt.Printf("[DEBUG] 查看方法:\n")
fmt.Printf(" CPU 火焰图: go tool pprof -http=:8081 %s/cpu.prof\n", profilesPath)
fmt.Printf(" 内存火焰图: go tool pprof -http=:8081 %s/mem.prof\n", profilesPath)
fmt.Printf(" 协程分析: go tool pprof -http=:8081 %s/goroutine.prof\n", profilesPath)
fmt.Printf(" 执行时间线: go tool trace %s/trace.out\n", profilesPath)
fmt.Printf("\n[DEBUG] %s\n", i18n.Tr("debug_profiles_saved", profilesPath))
fmt.Printf("[DEBUG] %s\n", i18n.GetText("debug_view_methods"))
fmt.Printf(" %s: go tool pprof -http=:8081 %s/cpu.prof\n", i18n.GetText("debug_cpu_flamegraph"), profilesPath)
fmt.Printf(" %s: go tool pprof -http=:8081 %s/mem.prof\n", i18n.GetText("debug_mem_flamegraph"), profilesPath)
fmt.Printf(" %s: go tool pprof -http=:8081 %s/goroutine.prof\n", i18n.GetText("debug_goroutine_analysis"), profilesPath)
fmt.Printf(" %s: go tool trace %s/trace.out\n", i18n.GetText("debug_execution_timeline"), profilesPath)
}
+11
View File
@@ -0,0 +1,11 @@
//go:build !debug
// +build !debug
package debug
import "testing"
func TestStubStartStop(t *testing.T) {
Start()
Stop()
}
+28
View File
@@ -0,0 +1,28 @@
package common
import (
"net"
"sync"
)
// DNSCache 并发安全的 DNS 解析缓存
// 对纯 IP 输入零开销(直接返回),对域名避免重复系统调用
var DNSCache = &dnsCache{}
type dnsCache struct {
m sync.Map // host -> *net.IPAddr
}
// ResolveIP 解析 host 为 *net.IPAddr,结果缓存
func (c *dnsCache) ResolveIP(host string) (*net.IPAddr, error) {
if v, ok := c.m.Load(host); ok {
addr, _ := v.(*net.IPAddr)
return addr, nil
}
addr, err := net.ResolveIPAddr("ip", host)
if err != nil {
return nil, err
}
c.m.Store(host, addr)
return addr, nil
}
+23
View File
@@ -0,0 +1,23 @@
package common
import "testing"
func TestDNSCacheResolveIPAndCacheHit(t *testing.T) {
cache := &dnsCache{}
first, err := cache.ResolveIP("127.0.0.1")
if err != nil {
t.Fatalf("ResolveIP loopback error = %v", err)
}
second, err := cache.ResolveIP("127.0.0.1")
if err != nil {
t.Fatalf("ResolveIP cached loopback error = %v", err)
}
if first != second {
t.Fatal("ResolveIP should return cached address on second lookup")
}
if _, err := cache.ResolveIP("bad host with spaces"); err == nil {
t.Fatal("ResolveIP should reject an invalid host")
}
}
+103 -7
View File
@@ -15,6 +15,9 @@ import (
// ErrShowHelp 表示用户请求显示帮助(正常退出)
var ErrShowHelp = errors.New("show help requested")
// IsLocalMode 由 plugins 包注册,判断 -m 指定的是否全是本地插件
var IsLocalMode func(mode string) bool
// Banner 显示程序横幅信息
func Banner() {
// 静默模式下完全跳过Banner显示
@@ -107,9 +110,11 @@ func Flag(Info *HostInfo) error {
flag.IntVar(&fv.ThreadNum, "t", 600, i18n.GetText("flag_thread_num"))
flag.Int64Var(&fv.TimeoutSec, "time", 3, i18n.GetText("flag_timeout"))
flag.IntVar(&fv.ModuleThreadNum, "mt", 20, i18n.GetText("flag_module_thread_num"))
flag.Int64Var(&fv.GlobalTimeout, "gt", 180, i18n.GetText("flag_global_timeout"))
flag.Int64Var(&fv.GlobalTimeout, "gt", 0, i18n.GetText("flag_global_timeout"))
flag.BoolVar(&fv.DisablePing, "np", false, i18n.GetText("flag_disable_ping"))
flag.StringVar(&fv.LocalPlugin, "local", "", "指定本地插件名称 (如: cleaner, avdetect, keylogger 等)")
flag.BoolVar(&fv.DisableTcpProbe, "ntp", false, i18n.GetText("flag_disable_tcp_probe"))
flag.BoolVar(&fv.DisableSubnetProbe, "nsp", false, i18n.GetText("flag_disable_subnet_probe"))
flag.StringVar(&fv.LocalPlugin, "local", "", i18n.GetText("flag_local_plugin"))
flag.BoolVar(&fv.AliveOnly, "ao", false, i18n.GetText("flag_alive_only"))
// ═════════════════════════════════════════════════
@@ -133,6 +138,7 @@ func Flag(Info *HostInfo) error {
flag.StringVar(&fv.TargetURL, "u", "", i18n.GetText("flag_target_url"))
flag.StringVar(&fv.URLsFile, "uf", "", i18n.GetText("flag_urls_file"))
flag.StringVar(&fv.Cookie, "cookie", "", i18n.GetText("flag_cookie"))
flag.StringVar(&fv.UserAgent, "ua", "", i18n.GetText("flag_user_agent"))
flag.Int64Var(&fv.WebTimeout, "wt", 5, i18n.GetText("flag_web_timeout"))
flag.IntVar(&fv.MaxRedirects, "max-redirect", 10, i18n.GetText("flag_max_redirects"))
flag.StringVar(&fv.HTTPProxy, "proxy", "", i18n.GetText("flag_http_proxy"))
@@ -181,8 +187,9 @@ func Flag(Info *HostInfo) error {
flag.BoolVar(&fv.Silent, "silent", false, i18n.GetText("flag_silent_mode"))
flag.BoolVar(&fv.NoColor, "nocolor", false, i18n.GetText("flag_no_color"))
flag.StringVar(&fv.LogLevel, "log", LogLevelBaseInfoSuccess, i18n.GetText("flag_log_level"))
flag.BoolVar(&fv.Debug, "debug", false, i18n.GetText("flag_debug"))
flag.BoolVar(&fv.DisableProgress, "nopg", false, i18n.GetText("flag_disable_progress"))
flag.BoolVar(&fv.PerfStats, "perf", false, "输出性能统计JSON")
flag.BoolVar(&fv.PerfStats, "perf", false, i18n.GetText("flag_perf_stats"))
// ═════════════════════════════════════════════════
// 其他参数
@@ -208,6 +215,26 @@ func Flag(Info *HostInfo) error {
return err
}
// 检测用户是否显式指定了 -t
flag.Visit(func(f *flag.Flag) {
switch f.Name {
case "t":
fv.ThreadNumExplicit = true
case "time":
fv.TimeoutExplicit = true
case "mt":
fv.ModuleThreadNumExplicit = true
case "retry":
fv.MaxRetriesExplicit = true
case "gt":
fv.GlobalTimeoutExplicit = true
case "icmp-rate":
fv.ICMPRateExplicit = true
case "num":
fv.PocNumExplicit = true
}
})
// 设置语言
i18n.SetLanguage(fv.Language)
@@ -222,7 +249,9 @@ func Flag(Info *HostInfo) error {
// parseCommandLineArgs 解析命令行参数
func parseCommandLineArgs() error {
flag.Parse()
if err := flag.CommandLine.Parse(normalizeMultiValueFlagArgs(os.Args[1:], "-pwda")); err != nil {
return err
}
// 显示Banner
Banner()
@@ -231,6 +260,63 @@ func parseCommandLineArgs() error {
return checkParameterConflicts()
}
func normalizeMultiValueFlagArgs(args []string, names ...string) []string {
multiValueFlags := make(map[string]struct{}, len(names))
for _, name := range names {
multiValueFlags[name] = struct{}{}
}
normalized := make([]string, 0, len(args))
for i := 0; i < len(args); i++ {
arg := args[i]
name, value, ok := splitMultiValueFlag(arg, multiValueFlags)
if !ok {
normalized = append(normalized, arg)
continue
}
values := []string{}
if value != "" {
values = append(values, value)
}
j := i + 1
for ; j < len(args); j++ {
if strings.HasPrefix(args[j], "-") {
break
}
values = append(values, args[j])
}
i = j - 1
if strings.Contains(arg, "=") {
normalized = append(normalized, name+"="+strings.Join(values, ","))
} else {
normalized = append(normalized, name)
if len(values) > 0 {
normalized = append(normalized, strings.Join(values, ","))
}
}
}
return normalized
}
func splitMultiValueFlag(arg string, names map[string]struct{}) (string, string, bool) {
if _, ok := names[arg]; ok {
return arg, "", true
}
for name := range names {
prefix := name + "="
if strings.HasPrefix(arg, prefix) {
return name, strings.TrimPrefix(arg, prefix), true
}
}
return "", "", false
}
// preProcessLanguage 预处理语言参数,在定义flag之前设置语言
func preProcessLanguage() {
// 遍历命令行参数查找-lang参数
@@ -270,9 +356,14 @@ func shouldShowHelp(Info *HostInfo, fv *FlagVars) bool {
// 检查是否提供了扫描目标
hasTarget := Info.Host != "" || fv.TargetURL != "" || fv.HostsFile != "" || fv.URLsFile != ""
// 本地模式需要指定插件才算有效目标
// 本地模式需要目标主机
if fv.LocalPlugin != "" {
hasTarget = true
return false
}
// -m 指定的全是本地插件时也不需要目标
if IsLocalMode != nil && IsLocalMode(fv.ScanMode) {
return false
}
// 如果没有提供任何扫描目标,则显示帮助
@@ -284,6 +375,11 @@ func shouldShowHelp(Info *HostInfo, fv *FlagVars) bool {
func checkParameterConflicts() error {
fv := flagVars
// -debug 等价于 -log debug
if fv.Debug {
fv.LogLevel = LogLevelDebug
}
// 检查 -ao 和 -m icmp 同时指定的情况(向后兼容提示)
if fv.AliveOnly && fv.ScanMode == "icmp" {
LogInfo(i18n.GetText("param_conflict_ao_icmp_both"))
@@ -295,7 +391,7 @@ func checkParameterConflicts() error {
invalidChars := []string{",", ";", " ", "|", "&"}
for _, char := range invalidChars {
if strings.Contains(fv.LocalPlugin, char) {
return fmt.Errorf("本地插件只能指定单个插件,不支持使用 '%s' 分隔的多个插件", char)
return fmt.Errorf("%s", i18n.Tr("param_local_multi_plugin", char))
}
}
}
+39
View File
@@ -0,0 +1,39 @@
package common
import (
"reflect"
"testing"
)
func TestNormalizeMultiValueFlagArgs(t *testing.T) {
tests := []struct {
name string
args []string
want []string
}{
{
name: "space separated pwda values",
args: []string{"-h", "192.168.1.1", "-pwda", "pass1", "pass2", "pass3", "-m", "ssh"},
want: []string{"-h", "192.168.1.1", "-pwda", "pass1,pass2,pass3", "-m", "ssh"},
},
{
name: "equals form with extra values",
args: []string{"-pwda=pass1", "pass2", "-h", "192.168.1.1"},
want: []string{"-pwda=pass1,pass2", "-h", "192.168.1.1"},
},
{
name: "unrelated args unchanged",
args: []string{"-h", "192.168.1.1", "-m", "ssh"},
want: []string{"-h", "192.168.1.1", "-m", "ssh"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := normalizeMultiValueFlagArgs(tt.args, "-pwda")
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("normalizeMultiValueFlagArgs() = %#v, want %#v", got, tt.want)
}
})
}
}
+78 -43
View File
@@ -1,9 +1,11 @@
package common
import (
"os"
"time"
"github.com/shadow1ng/fscan/common/config"
"golang.org/x/term"
)
/*
@@ -25,21 +27,27 @@ type FlagVars struct {
ExcludeHostsFile string
Ports string
ExcludePorts string
AddPorts string
HostsFile string
PortsFile string
// 扫描控制
ScanMode string
ThreadNum int
ModuleThreadNum int
TimeoutSec int64 // 秒,需转换为 time.Duration
GlobalTimeout int64
DisablePing bool
LocalPlugin string
AliveOnly bool
DisableBrute bool
MaxRetries int
ScanMode string
ThreadNum int
ThreadNumExplicit bool // 用户显式指定了 -t
ModuleThreadNum int
ModuleThreadNumExplicit bool
TimeoutSec int64 // 秒,需转换为 time.Duration
TimeoutExplicit bool
GlobalTimeout int64
GlobalTimeoutExplicit bool
DisablePing bool
DisableTcpProbe bool
DisableSubnetProbe bool
LocalPlugin string
AliveOnly bool
DisableBrute bool
MaxRetries int
MaxRetriesExplicit bool
// 认证凭据
Username string
@@ -72,6 +80,7 @@ type FlagVars struct {
PocFull bool
DNSLog bool
PocNum int
PocNumExplicit bool
DisablePocScan bool
// Redis利用
@@ -83,9 +92,10 @@ type FlagVars struct {
DisableRedis bool
// 发包频率
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
PacketRateLimit int64
MaxPacketCount int64
ICMPRate float64
ICMPRateExplicit bool
// 输出控制
Outputfile string
@@ -94,6 +104,7 @@ type FlagVars struct {
Silent bool
NoColor bool
LogLevel string
Debug bool
DisableProgress bool
PerfStats bool
Language string
@@ -132,18 +143,24 @@ func GetFlagVars() *FlagVars {
func BuildConfigFromFlags(fv *FlagVars) *Config {
return &Config{
// 高频字段
Timeout: time.Duration(fv.TimeoutSec) * time.Second,
ThreadNum: fv.ThreadNum,
ModuleThreadNum: fv.ModuleThreadNum,
DisableBrute: fv.DisableBrute,
DisablePing: fv.DisablePing,
Timeout: time.Duration(fv.TimeoutSec) * time.Second,
TimeoutExplicit: fv.TimeoutExplicit,
ThreadNum: fv.ThreadNum,
ThreadNumExplicit: fv.ThreadNumExplicit,
ModuleThreadNum: fv.ModuleThreadNum,
ModuleThreadNumExplicit: fv.ModuleThreadNumExplicit,
DisableBrute: fv.DisableBrute,
DisablePing: fv.DisablePing,
DisableTcpProbe: fv.DisableTcpProbe,
DisableSubnetProbe: fv.DisableSubnetProbe,
// 扫描模式
Mode: fv.ScanMode,
LocalMode: fv.LocalPlugin != "",
LocalPlugin: fv.LocalPlugin,
AliveOnly: fv.AliveOnly,
MaxRetries: fv.MaxRetries,
Mode: fv.ScanMode,
LocalMode: fv.LocalPlugin != "",
LocalPlugin: fv.LocalPlugin,
AliveOnly: fv.AliveOnly,
MaxRetries: fv.MaxRetries,
MaxRetriesExplicit: fv.MaxRetriesExplicit,
// 高级功能
Shellcode: fv.Shellcode,
@@ -151,8 +168,12 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
DNSLog: fv.DNSLog,
PersistenceTargetFile: fv.PersistenceTargetFile,
WinPEFile: fv.WinPEFile,
PortMap: config.DefaultPortMap,
DefaultMap: config.DefaultProbeMap,
PortMap: clonePortMap(config.DefaultPortMap),
DefaultMap: cloneStringSlice(config.DefaultProbeMap),
// 全局超时
GlobalTimeout: time.Duration(fv.GlobalTimeout) * time.Second,
GlobalTimeoutExplicit: fv.GlobalTimeoutExplicit,
// SOCKS5代理端口
Socks5ProxyPort: fv.Socks5ProxyPort,
@@ -162,26 +183,27 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
Username: fv.Username,
Password: fv.Password,
Domain: fv.Domain,
Userdict: config.DefaultUserDict,
Passwords: config.DefaultPasswords,
Userdict: cloneStringSliceMap(config.DefaultUserDict),
Passwords: cloneStringSlice(config.DefaultPasswords),
UserPassPairs: nil, // 后续解析
SSHKeyPath: fv.SSHKeyPath,
},
Network: NetworkConfig{
HTTPProxy: fv.HTTPProxy,
Socks5Proxy: fv.Socks5Proxy,
Iface: fv.Iface,
WebTimeout: time.Duration(fv.WebTimeout) * time.Second,
MaxRedirects: fv.MaxRedirects,
PacketRateLimit: fv.PacketRateLimit,
MaxPacketCount: fv.MaxPacketCount,
ICMPRate: fv.ICMPRate,
HTTPProxy: fv.HTTPProxy,
Socks5Proxy: fv.Socks5Proxy,
Iface: fv.Iface,
WebTimeout: time.Duration(fv.WebTimeout) * time.Second,
MaxRedirects: fv.MaxRedirects,
PacketRateLimit: fv.PacketRateLimit,
MaxPacketCount: fv.MaxPacketCount,
ICMPRate: fv.ICMPRate,
ICMPRateExplicit: fv.ICMPRateExplicit,
},
Output: OutputConfig{
File: fv.Outputfile,
Format: fv.OutputFormat,
DisableSave: fv.DisableSave,
NoColor: fv.NoColor,
NoColor: fv.NoColor || !isStdoutTerminal(),
Silent: fv.Silent,
DisableProgress: fv.DisableProgress,
ShowProgress: !fv.DisableProgress,
@@ -190,11 +212,12 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
PerfStats: fv.PerfStats,
},
POC: POCConfig{
PocPath: fv.PocPath,
PocName: fv.PocName,
Full: fv.PocFull,
Num: fv.PocNum,
Disabled: fv.DisablePocScan,
PocPath: fv.PocPath,
PocName: fv.PocName,
Full: fv.PocFull,
Num: fv.PocNum,
NumExplicit: fv.PocNumExplicit,
Disabled: fv.DisablePocScan,
},
Redis: RedisConfig{
Disabled: fv.DisableRedis,
@@ -206,7 +229,7 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
},
HTTP: HTTPConfig{
Cookie: fv.Cookie,
UserAgent: fv.UserAgent,
UserAgent: defaultUserAgent(fv.UserAgent),
Accept: fv.Accept,
},
LocalExploit: LocalExploitConfig{
@@ -222,3 +245,15 @@ func BuildConfigFromFlags(fv *FlagVars) *Config {
},
}
}
func isStdoutTerminal() bool {
return term.IsTerminal(int(os.Stdout.Fd()))
}
// defaultUserAgent 用户未通过 -ua 指定时回退到默认 UA,避免发送空 User-Agent 被 WAF 识别
func defaultUserAgent(ua string) string {
if ua != "" {
return ua
}
return "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
}
+7 -3
View File
@@ -34,7 +34,7 @@ func TestBuildConfigFromFlags_ScanControl(t *testing.T) {
ThreadNum: 600,
ModuleThreadNum: 20,
TimeoutSec: 3,
GlobalTimeout: 180,
GlobalTimeout: 0,
},
validate: func(t *testing.T, cfg *Config) {
if cfg.Mode != "all" {
@@ -49,6 +49,9 @@ func TestBuildConfigFromFlags_ScanControl(t *testing.T) {
if cfg.Timeout != 3*time.Second {
t.Errorf("Timeout = %v, want %v", cfg.Timeout, 3*time.Second)
}
if cfg.GlobalTimeout != 0 {
t.Errorf("GlobalTimeout = %v, want disabled", cfg.GlobalTimeout)
}
},
},
{
@@ -955,8 +958,9 @@ func TestBuildConfigFromFlags_BoundaryValues(t *testing.T) {
if cfg.HTTP.Cookie != "" {
t.Errorf("Cookie 应该为空")
}
if cfg.HTTP.UserAgent != "" {
t.Errorf("UserAgent 应该为空")
// 空输入回退到默认 UA,避免发送空 User-Agent
if cfg.HTTP.UserAgent == "" {
t.Errorf("UserAgent 空输入应回退到默认 UA")
}
},
},
+4 -11
View File
@@ -2,15 +2,8 @@
package common
import "flag"
// WebMode Web版本始终为true
const WebMode = true
// WebMode 表示是否启动Web管理界面
var WebMode bool
// WebPort Web服务器端口
var WebPort int
func init() {
flag.BoolVar(&WebMode, "web", false, "启动Web管理界面 (Start Web UI)")
flag.IntVar(&WebPort, "webport", 10240, "Web服务器端口 (Web server port)")
}
// WebPort 不再使用,端口由 main_web.go 的 -port 参数控制
var WebPort = 0
+14 -5
View File
@@ -2,9 +2,12 @@ package common
import (
"errors"
"fmt"
"net"
"strconv"
"strings"
"sync"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -28,7 +31,11 @@ type HostInfo struct {
// Target 返回 host:port 格式字符串
func (h *HostInfo) Target() string {
return fmt.Sprintf("%s:%d", h.Host, h.Port)
host := h.Host
if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") {
host = strings.TrimPrefix(strings.TrimSuffix(host, "]"), "[")
}
return net.JoinHostPort(host, strconv.Itoa(h.Port))
}
// =============================================================================
@@ -62,11 +69,13 @@ const (
// 版本信息,通过 ldflags 注入
var (
version = "2.1.2"
version = "2.2.1"
commit = "unknown"
date = "unknown"
)
func GetVersion() string { return version }
// 运行时数据已迁移到Config对象中,使用GetGlobalConfig()访问
// Shell状态已迁移到State对象中,使用GetGlobalState()访问
@@ -92,9 +101,9 @@ type PacketLimitError struct {
func (e *PacketLimitError) Error() string {
if e.Sentinel == ErrMaxPacketReached {
return fmt.Sprintf("已达到最大发包数量限制: %d", e.Limit)
return i18n.Tr("packet_limit_max_reached", e.Limit)
}
return fmt.Sprintf("发包速率受限: %d包/分钟", e.Limit)
return i18n.Tr("packet_limit_rate_limited", e.Limit)
}
func (e *PacketLimitError) Unwrap() error {
+67
View File
@@ -0,0 +1,67 @@
package common
import (
"errors"
"strings"
"testing"
)
func TestHostInfoTargetUsesBracketedIPv6(t *testing.T) {
info := &HostInfo{Host: "2001:db8::1", Port: 443}
if got, want := info.Target(), "[2001:db8::1]:443"; got != want {
t.Fatalf("Target() = %q, want %q", got, want)
}
}
func TestHostInfoTargetDoesNotDoubleBracketIPv6(t *testing.T) {
info := &HostInfo{Host: "[2001:db8::1]", Port: 443}
if got, want := info.Target(), "[2001:db8::1]:443"; got != want {
t.Fatalf("Target() = %q, want %q", got, want)
}
}
func TestGlobalHelpersAndPacketLimitErrors(t *testing.T) {
if GetVersion() == "" {
t.Fatal("GetVersion returned empty string")
}
if !ContainsAny("hello fscan", "none", "scan") {
t.Fatal("ContainsAny should find a matching substring")
}
if ContainsAny("hello fscan", "none", "missing") {
t.Fatal("ContainsAny should return false when nothing matches")
}
maxErr := &PacketLimitError{Sentinel: ErrMaxPacketReached, Limit: 5, Current: 5}
if !errors.Is(maxErr, ErrMaxPacketReached) || !strings.Contains(maxErr.Error(), "5") {
t.Fatalf("max packet error = %v", maxErr)
}
rateErr := &PacketLimitError{Sentinel: ErrPacketRateLimited, Limit: 3, Current: 2}
if !errors.Is(rateErr, ErrPacketRateLimited) || !strings.Contains(rateErr.Error(), "3") {
t.Fatalf("rate limit error = %v", rateErr)
}
}
func TestCanSendPacketUsesGlobalConfigAndState(t *testing.T) {
previousConfig := GetGlobalConfig()
previousState := GetGlobalState()
t.Cleanup(func() {
SetGlobalConfig(previousConfig)
SetGlobalState(previousState)
})
cfg := NewConfig()
cfg.Network.MaxPacketCount = 1
state := NewState()
state.IncrementPacketCount()
SetGlobalConfig(cfg)
SetGlobalState(state)
ok, reason := CanSendPacket()
if ok {
t.Fatal("CanSendPacket should reject when max packet count is reached")
}
if reason == "" {
t.Fatal("CanSendPacket should return a rejection reason")
}
}
+10 -2
View File
@@ -2,6 +2,7 @@ package i18n
import (
"fmt"
"strconv"
"sync"
"github.com/nicksnyder/go-i18n/v2/i18n"
@@ -51,6 +52,13 @@ func SetLanguage(l string) {
localizer = i18n.NewLocalizer(bundle, lang, FallbackLanguage)
}
// GetLanguage returns the currently configured language.
func GetLanguage() string {
mu.RLock()
defer mu.RUnlock()
return lang
}
// GetText 获取国际化文本(无参数)
func GetText(key string) string {
mu.RLock()
@@ -73,9 +81,9 @@ func Tr(key string, args ...interface{}) string {
loc := localizer
mu.RUnlock()
data := make(map[string]interface{})
data := make(map[string]interface{}, len(args))
for i, arg := range args {
data[fmt.Sprintf("Arg%d", i+1)] = arg
data["Arg"+strconv.Itoa(i+1)] = arg
}
msg, err := loc.Localize(&i18n.LocalizeConfig{
+37
View File
@@ -0,0 +1,37 @@
package i18n
import (
"strings"
"testing"
)
func TestLanguageLifecycleAndFallbacks(t *testing.T) {
original := GetLanguage()
t.Cleanup(func() { SetLanguage(original) })
SetLanguage(LangEN)
if got := GetLanguage(); got != LangEN {
t.Fatalf("language = %q, want %q", got, LangEN)
}
if got := GetText("concurrency_plugin"); got == "" || got == "concurrency_plugin" {
t.Fatalf("english text = %q, want translated text", got)
}
if got := Tr("debug_cpu_profile_started", "/tmp/profiles"); !strings.Contains(got, "/tmp/profiles") {
t.Fatalf("formatted english text = %q, want path included", got)
}
SetLanguage(LangZH)
if got := GetLanguage(); got != LangZH {
t.Fatalf("language = %q, want %q", got, LangZH)
}
if got := GetText("concurrency_plugin"); got == "" || got == "concurrency_plugin" {
t.Fatalf("chinese text = %q, want translated text", got)
}
if got := GetText("missing_translation_key"); got != "missing_translation_key" {
t.Fatalf("missing GetText = %q, want key", got)
}
if got := Tr("missing_translation_key", "ignored"); got != "missing_translation_key" {
t.Fatalf("missing Tr = %q, want key", got)
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+17 -11
View File
@@ -2,6 +2,8 @@ package common
import (
"fmt"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -13,9 +15,10 @@ initialize.go - 统一初始化入口
// InitResult 初始化结果
type InitResult struct {
Config *Config
State *State
Info *HostInfo
Config *Config
State *State
Info *HostInfo
Session *ScanSession
}
// Initialize 统一初始化函数
@@ -27,7 +30,7 @@ func Initialize(info *HostInfo) (*InitResult, error) {
// 2. 从 FlagVars 构建 Config 和 State
cfg, state, err := BuildConfig(GetFlagVars(), info)
if err != nil {
return nil, fmt.Errorf("配置构建失败: %w", err)
return nil, fmt.Errorf("%s: %w", i18n.GetText("config_build_failed"), err)
}
// 3. 设置全局实例
@@ -36,13 +39,16 @@ func Initialize(info *HostInfo) (*InitResult, error) {
// 4. 初始化输出系统
if err := InitOutput(); err != nil {
return nil, fmt.Errorf("输出初始化失败: %w", err)
return nil, fmt.Errorf("%s: %w", i18n.GetText("output_init_failed"), err)
}
session := NewScanSession(cfg, state, GetFlagVars())
return &InitResult{
Config: cfg,
State: state,
Info: info,
Config: cfg,
State: state,
Info: info,
Session: session,
}, nil
}
@@ -61,7 +67,7 @@ func ValidateExclusiveParams(info *HostInfo) error {
if fv.TargetURL != "" {
paramCount++
if activeParam != "" {
activeParam += " 和 -u"
activeParam = i18n.Tr("param_join_and", activeParam, "-u")
} else {
activeParam = "-u"
}
@@ -69,14 +75,14 @@ func ValidateExclusiveParams(info *HostInfo) error {
if fv.LocalPlugin != "" {
paramCount++
if activeParam != "" {
activeParam += " 和 -local"
activeParam = i18n.Tr("param_join_and", activeParam, "-local")
} else {
activeParam = "-local"
}
}
if paramCount > 1 {
return fmt.Errorf("参数 %s 互斥,请只指定一个扫描目标\n -h: 网络主机扫描\n -u: Web URL扫描\n -local: 本地信息收集", activeParam)
return fmt.Errorf("%s", i18n.Tr("param_exclusive", activeParam))
}
return nil
+55
View File
@@ -0,0 +1,55 @@
package common
import (
"strings"
"testing"
)
func TestValidateExclusiveParams(t *testing.T) {
previous := GetFlagVars()
t.Cleanup(func() { flagVars = previous })
tests := []struct {
name string
info *HostInfo
flags *FlagVars
wantErr string
}{
{name: "host only", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{}},
{name: "url only", info: &HostInfo{}, flags: &FlagVars{TargetURL: "http://example.com"}},
{name: "local only", info: &HostInfo{}, flags: &FlagVars{LocalPlugin: "sshkey"}},
{name: "host and url conflict", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{TargetURL: "http://example.com"}, wantErr: "-h"},
{name: "host url local conflict", info: &HostInfo{Host: "127.0.0.1"}, flags: &FlagVars{TargetURL: "http://example.com", LocalPlugin: "sshkey"}, wantErr: "-local"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
flagVars = tt.flags
err := ValidateExclusiveParams(tt.info)
if tt.wantErr == "" {
if err != nil {
t.Fatalf("ValidateExclusiveParams error = %v", err)
}
return
}
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("ValidateExclusiveParams error = %v, want containing %q", err, tt.wantErr)
}
})
}
}
func TestCleanupWithoutOutput(t *testing.T) {
oldResultOutput := ResultOutput
oldStdoutWriter := StdoutWriter
t.Cleanup(func() {
ResultOutput = oldResultOutput
StdoutWriter = oldStdoutWriter
})
ResultOutput = nil
StdoutWriter = nil
if err := Cleanup(); err != nil {
t.Fatalf("Cleanup error = %v", err)
}
}
+52 -2
View File
@@ -14,11 +14,16 @@ import (
)
var (
globalLogger *logging.Logger
loggerOnce sync.Once
globalLogger *logging.Logger
loggerOnce sync.Once
loggerMu sync.Mutex
silentLoggerRefs int
)
func getGlobalLogger() *logging.Logger {
loggerMu.Lock()
defer loggerMu.Unlock()
loggerOnce.Do(func() {
fv := GetFlagVars()
level := getLogLevelFromString(fv.LogLevel)
@@ -27,8 +32,12 @@ func getGlobalLogger() *logging.Logger {
EnableColor: !fv.NoColor,
SlowOutput: false,
ShowProgress: !fv.DisableProgress,
Silent: fv.Silent || silentLoggerRefs > 0,
StartTime: GetGlobalState().GetStartTime(),
}
if fv.Debug {
config.DebugLogFile = "fscan_debug.log"
}
globalLogger = logging.NewLogger(config)
globalLogger.SetCoordinatedOutput(LogWithProgress)
})
@@ -77,3 +86,44 @@ func LogVuln(result string) { getGlobalLogger().Vuln(result) }
// LogError 输出错误日志
func LogError(errMsg string) { getGlobalLogger().Error(errMsg) }
// CloseLogger 关闭日志系统,释放文件资源
func CloseLogger() {
loggerMu.Lock()
defer loggerMu.Unlock()
closeLoggerLocked()
}
// PushSilentLogger suppresses process-wide legacy log output until the returned
// restore function is called. It is reference counted so concurrent embedded
// scans can overlap safely.
func PushSilentLogger() func() {
loggerMu.Lock()
silentLoggerRefs++
resetLoggerLocked()
loggerMu.Unlock()
var once sync.Once
return func() {
once.Do(func() {
loggerMu.Lock()
if silentLoggerRefs > 0 {
silentLoggerRefs--
}
resetLoggerLocked()
loggerMu.Unlock()
})
}
}
func resetLoggerLocked() {
closeLoggerLocked()
globalLogger = nil
loggerOnce = sync.Once{}
}
func closeLoggerLocked() {
if globalLogger != nil {
globalLogger.Close()
}
}
+63
View File
@@ -0,0 +1,63 @@
package common
import "testing"
func preserveLoggerForTest(t *testing.T) {
t.Helper()
loggerMu.Lock()
oldSilentRefs := silentLoggerRefs
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
t.Cleanup(func() {
loggerMu.Lock()
closeLoggerLocked()
silentLoggerRefs = oldSilentRefs
resetLoggerLocked()
loggerMu.Unlock()
})
}
func TestLoggerFacadeSilentLifecycle(t *testing.T) {
preserveLoggerForTest(t)
previousFlags := GetFlagVars()
previousState := GetGlobalState()
t.Cleanup(func() {
flagVars = previousFlags
SetGlobalState(previousState)
})
flagVars = &FlagVars{Silent: true, LogLevel: "debug"}
SetGlobalState(NewState())
InitLogger()
LogDebug("debug")
LogInfo("info")
LogSuccess("success")
LogVuln("vuln")
LogError("error")
CloseLogger()
}
func TestPushSilentLoggerReferenceCount(t *testing.T) {
preserveLoggerForTest(t)
restoreOne := PushSilentLogger()
restoreTwo := PushSilentLogger()
if silentLoggerRefs != 2 {
t.Fatalf("silent refs = %d, want 2", silentLoggerRefs)
}
restoreOne()
restoreOne()
if silentLoggerRefs != 1 {
t.Fatalf("silent refs after first restore = %d, want 1", silentLoggerRefs)
}
restoreTwo()
if silentLoggerRefs != 0 {
t.Fatalf("silent refs after second restore = %d, want 0", silentLoggerRefs)
}
}
+43 -1
View File
@@ -2,6 +2,7 @@ package logging
import (
"fmt"
"os"
"strings"
"sync"
"time"
@@ -24,8 +25,10 @@ type LoggerConfig struct {
EnableColor bool `json:"enable_color"`
SlowOutput bool `json:"slow_output"`
ShowProgress bool `json:"show_progress"`
Silent bool `json:"silent"`
StartTime time.Time `json:"start_time"`
LevelColors map[LogLevel]interface{} `json:"-"`
DebugLogFile string `json:"debug_log_file"`
}
// DefaultLoggerConfig 默认日志器配置
@@ -47,6 +50,7 @@ type Logger struct {
startTime time.Time
coordinatedOutput func(string)
initialized bool
debugFile *os.File
}
// NewLogger 创建新的日志管理器
@@ -55,11 +59,20 @@ func NewLogger(config *LoggerConfig) *Logger {
config = DefaultLoggerConfig()
}
return &Logger{
l := &Logger{
config: config,
startTime: config.StartTime,
initialized: true,
}
if config.DebugLogFile != "" {
f, err := os.OpenFile(config.DebugLogFile, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0644)
if err == nil {
l.debugFile = f
}
}
return l
}
// Initialize 初始化日志器
@@ -111,6 +124,10 @@ func (l *Logger) log(level LogLevel, content string) {
l.mu.Lock()
defer l.mu.Unlock()
if l.config.Silent {
return
}
if !l.shouldLog(level) {
return
}
@@ -134,12 +151,37 @@ func (l *Logger) log(level LogLevel, content string) {
l.outputMessage(level, logMsg)
}
// 写入debug日志文件(纯文本,无颜色)
if l.debugFile != nil {
timestamp := time.Since(l.startTime).Truncate(time.Millisecond)
if strings.Contains(content, "\n") {
lines := strings.Split(content, "\n")
for _, line := range lines {
if line != "" {
_, _ = fmt.Fprintf(l.debugFile, "[%s] %s %s\n", timestamp, prefix, line)
}
}
} else {
_, _ = fmt.Fprintf(l.debugFile, "[%s] %s %s\n", timestamp, prefix, content)
}
}
// 根据慢速输出设置决定是否添加延迟
if l.config.SlowOutput {
time.Sleep(SlowOutputDelay)
}
}
// Close 关闭日志器,释放文件资源
func (l *Logger) Close() {
l.mu.Lock()
defer l.mu.Unlock()
if l.debugFile != nil {
_ = l.debugFile.Close()
l.debugFile = nil
}
}
// shouldLog 检查是否应该记录该级别的日志
// 层级过滤:消息级别 >= 配置级别 时显示,Error 始终显示
func (l *Logger) shouldLog(level LogLevel) bool {
+40
View File
@@ -2,6 +2,8 @@ package logging
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -160,6 +162,13 @@ func TestLogger_AllLevels(t *testing.T) {
wantMsg: "success message",
wantPfx: PrefixSuccess,
},
{
name: "Vuln级别",
logFunc: logger.Vuln,
message: "vuln message",
wantMsg: "vuln message",
wantPfx: PrefixVuln,
},
{
name: "Error级别",
logFunc: logger.Error,
@@ -650,3 +659,34 @@ func TestLogger_Initialize(t *testing.T) {
t.Logf("✓ Initialize测试通过")
}
func TestLogger_CloseClosesDebugFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "debug.log")
logger := NewLogger(&LoggerConfig{
Level: LevelAll,
EnableColor: false,
ShowProgress: false,
StartTime: time.Now(),
LevelColors: GetDefaultLevelColors(),
DebugLogFile: path,
})
if logger.debugFile == nil {
t.Fatal("debug file should be opened")
}
logger.Info("debug file line")
logger.Close()
if logger.debugFile != nil {
t.Fatal("debug file should be nil after Close")
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read debug file: %v", err)
}
if !strings.Contains(string(content), "debug file line") {
t.Fatalf("debug file content = %q", string(content))
}
logger.Close()
}
+31 -12
View File
@@ -16,6 +16,7 @@ import (
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/proxy"
)
@@ -51,16 +52,31 @@ func getGlobalDialer(timeout time.Duration) (proxy.Dialer, error) {
// parseProxyURL 解析代理URL,提取地址和认证信息
func parseProxyURL(proxyURL, fallback string) (host, username, password string) {
if !strings.Contains(proxyURL, "://") {
if host, username, password, ok := parseProxyURLCandidate("http://" + proxyURL); ok {
return host, username, password
}
}
if host, username, password, ok := parseProxyURLCandidate(proxyURL); ok {
return host, username, password
}
return fallback, "", ""
}
func parseProxyURLCandidate(proxyURL string) (host, username, password string, ok bool) {
parsedURL, err := url.Parse(proxyURL)
if err != nil {
return fallback, "", ""
return "", "", "", false
}
host = parsedURL.Host
if host == "" {
return "", "", "", false
}
if parsedURL.User != nil {
username = parsedURL.User.Username()
password, _ = parsedURL.User.Password()
}
return
return host, username, password, true
}
// createProxyConfig 根据全局设置创建代理配置
@@ -102,22 +118,20 @@ func createProxyConfig(timeout time.Duration) *proxy.ProxyConfig {
// TCP 连接
// =============================================================================
// WrapperTcpWithTimeout TCP连接包装器,带超时
// 支持通过代理管理器进行SOCKS5和HTTP代理连接,并集成发包控制
// 使用全局拨号器复用连接,避免重复创建代理握手开销
// Deprecated: WrapperTcpWithTimeout 仅供 libs/grdp 兼容使用,新代码请用 ScanSession.DialTCP
//
//nolint:revive // 保持向后兼容性,避免破坏大量现有代码
//nolint:revive
func WrapperTcpWithTimeout(network, address string, timeout time.Duration) (net.Conn, error) {
// 检查发包限制 - 在代理连接前进行控制
if canSend, reason := CanSendPacket(); !canSend {
LogError(fmt.Sprintf("TCP连接 %s 受限: %s", address, reason))
return nil, fmt.Errorf("发包受限: %s", reason)
LogError(i18n.Tr("tcp_connection_restricted", address, reason))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", reason))
}
// 获取全局拨号器(复用,避免重复创建)
dialer, err := getGlobalDialer(timeout)
if err != nil {
LogError(fmt.Sprintf("获取代理拨号器失败: %v", err))
LogError(i18n.Tr("proxy_dialer_failed", err))
GetGlobalState().IncrementTCPFailedPacketCount()
return nil, err
}
@@ -128,7 +142,7 @@ func WrapperTcpWithTimeout(network, address string, timeout time.Duration) (net.
// 统计TCP包数量 - 无论是否使用代理都要计数
if err != nil {
GetGlobalState().IncrementTCPFailedPacketCount()
LogDebug(fmt.Sprintf("连接 %s 失败: %v", address, err))
LogDebug(i18n.Tr("connection_failed", address, err))
return nil, err
}
@@ -158,12 +172,17 @@ func IsProxyReliable() bool {
return proxy.IsProxyReliable()
}
// IsSOCKS5Proxy 检查当前代理是否为SOCKS5类型
func IsSOCKS5Proxy() bool {
return proxy.IsSOCKS5Proxy()
}
// SafeHTTPDo 带发包控制的HTTP请求
func SafeHTTPDo(client *http.Client, req *http.Request) (*http.Response, error) {
// 检查发包限制
if canSend, reason := CanSendPacket(); !canSend {
LogError(fmt.Sprintf("HTTP请求 %s 受限: %s", req.URL.String(), reason))
return nil, fmt.Errorf("发包受限: %s", reason)
LogError(i18n.Tr("http_request_restricted", req.URL.String(), reason))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", reason))
}
// 执行HTTP请求
+52
View File
@@ -0,0 +1,52 @@
package common
import (
"context"
"net/http"
"testing"
"github.com/shadow1ng/fscan/common/proxy"
)
func TestNetworkFacadeProxyState(t *testing.T) {
t.Cleanup(func() { proxy.AutoConfigureProxy(proxy.DefaultProxyConfig()) })
proxy.AutoConfigureProxy(proxy.DefaultProxyConfig())
if IsProxyEnabled() || IsSOCKS5Proxy() || !IsProxyReliable() {
t.Fatal("direct global proxy state should be disabled and reliable")
}
proxy.AutoConfigureProxy(&proxy.ProxyConfig{Type: proxy.ProxyTypeSOCKS5})
if !IsProxyEnabled() || !IsSOCKS5Proxy() || !IsProxyReliable() {
t.Fatal("SOCKS5 global proxy state should be enabled and SOCKS5")
}
}
func TestSafeHTTPDoUsesGlobalPacketLimit(t *testing.T) {
previousConfig := GetGlobalConfig()
previousState := GetGlobalState()
t.Cleanup(func() {
SetGlobalConfig(previousConfig)
SetGlobalState(previousState)
})
cfg := NewConfig()
cfg.Network.MaxPacketCount = 1
state := NewState()
state.IncrementPacketCount()
SetGlobalConfig(cfg)
SetGlobalState(state)
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
t.Fatal("transport should not be called when packet limit is reached")
return nil, nil
})}
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, "http://example.com", nil)
if err != nil {
t.Fatal(err)
}
if resp, err := SafeHTTPDo(client, req); err == nil || resp != nil {
t.Fatalf("SafeHTTPDo = resp %#v err %v, want limit error", resp, err)
}
}
+30 -10
View File
@@ -1,9 +1,6 @@
package output
import (
"fmt"
"sync"
)
import "sync"
// ResultBuffer 公共的去重缓冲逻辑,供各Writer复用
type ResultBuffer struct {
@@ -25,10 +22,10 @@ type ResultBuffer struct {
// NewResultBuffer 创建新的结果缓冲
func NewResultBuffer() *ResultBuffer {
return &ResultBuffer{
seenHosts: make(map[string]struct{}),
seenPorts: make(map[string]struct{}),
seenServices: make(map[string]int),
seenVulns: make(map[string]struct{}),
seenHosts: make(map[string]struct{}, 256),
seenPorts: make(map[string]struct{}, 512),
seenServices: make(map[string]int, 128),
seenVulns: make(map[string]struct{}, 64),
}
}
@@ -59,7 +56,8 @@ func (b *ResultBuffer) Add(result *ScanResult) {
b.seenServices[key] = len(b.ServiceResults)
b.ServiceResults = append(b.ServiceResults, result)
} else {
// 保留信息更完整的记录
b.mergeDetails(b.ServiceResults[idx], result)
// 保留信息更完整的记录,同时保留另一条记录补充的字段
if b.isMoreComplete(result, b.ServiceResults[idx]) {
b.ServiceResults[idx] = result
}
@@ -72,6 +70,28 @@ func (b *ResultBuffer) Add(result *ScanResult) {
}
}
func (b *ResultBuffer) mergeDetails(oldResult, newResult *ScanResult) {
if oldResult == nil || newResult == nil {
return
}
if oldResult.Details == nil {
oldResult.Details = make(map[string]interface{})
}
if newResult.Details == nil {
newResult.Details = make(map[string]interface{})
}
for k, v := range oldResult.Details {
if _, exists := newResult.Details[k]; !exists {
newResult.Details[k] = v
}
}
for k, v := range newResult.Details {
if _, exists := oldResult.Details[k]; !exists {
oldResult.Details[k] = v
}
}
}
// generateKey 生成结果的唯一键(用于去重)
func (b *ResultBuffer) generateKey(result *ScanResult) string {
switch result.Type {
@@ -80,7 +100,7 @@ func (b *ResultBuffer) generateKey(result *ScanResult) string {
case TypePort:
if result.Details != nil {
if port, ok := result.Details["port"]; ok {
return fmt.Sprintf("%s:%v", result.Target, port)
return targetWithPort(result.Target, port)
}
}
return result.Target
+34
View File
@@ -226,6 +226,40 @@ func TestResultBuffer_ServiceUpdate(t *testing.T) {
}
}
func TestResultBuffer_ServiceUpdateMergesDetails(t *testing.T) {
buf := NewResultBuffer()
buf.Add(&ScanResult{
Type: TypeService,
Target: "192.168.1.1:80",
Status: "identified",
Details: map[string]interface{}{
"service": "http",
"banner": "HTTP/1.1 200 OK",
},
})
buf.Add(&ScanResult{
Type: TypeService,
Target: "192.168.1.1:80",
Status: "web",
Details: map[string]interface{}{
"title": "Home",
"status": 200,
"server": "nginx",
},
})
if len(buf.ServiceResults) != 1 {
t.Fatalf("期望1条服务记录,实际 %d", len(buf.ServiceResults))
}
details := buf.ServiceResults[0].Details
for _, key := range []string{"service", "banner", "title", "status", "server"} {
if _, ok := details[key]; !ok {
t.Errorf("合并后的服务记录缺少字段 %q: %#v", key, details)
}
}
}
// TestResultBuffer_ServiceNoDowngrade 测试不降级服务记录
//
// 当新记录不如旧记录完整时,不应替换
+160
View File
@@ -0,0 +1,160 @@
package output
import (
"bufio"
"encoding/json"
"fmt"
"net"
"os"
"strconv"
"strings"
"sync"
)
type StdoutNDJSONWriter struct {
mu sync.Mutex
writer *bufio.Writer
}
func NewStdoutNDJSONWriter() *StdoutNDJSONWriter {
return &StdoutNDJSONWriter{
writer: bufio.NewWriter(os.Stdout),
}
}
// ndjsonRecord NDJSON 输出的扁平化结构
type ndjsonRecord struct {
Type ResultType `json:"type"`
Target string `json:"target"`
Status string `json:"status"`
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Service string `json:"service,omitempty"`
// 通用可选字段
Protocol string `json:"protocol,omitempty"`
Banner string `json:"banner,omitempty"`
Title string `json:"title,omitempty"`
URL string `json:"url,omitempty"`
// 漏洞/弱口令
Vulnerability string `json:"vulnerability,omitempty"`
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
// 其他
Plugin string `json:"plugin,omitempty"`
Version string `json:"version,omitempty"`
OS string `json:"os,omitempty"`
}
func (w *StdoutNDJSONWriter) WriteResult(result *ScanResult) error {
w.mu.Lock()
defer w.mu.Unlock()
rec := w.flatten(result)
data, err := json.Marshal(rec)
if err != nil {
return err
}
data = append(data, '\n')
if _, err := w.writer.Write(data); err != nil {
return err
}
return w.writer.Flush()
}
func (w *StdoutNDJSONWriter) flatten(r *ScanResult) *ndjsonRecord {
rec := &ndjsonRecord{
Type: r.Type,
Target: r.Target,
Status: r.Status,
}
// 从 target 拆分 host:port
if host, port, ok := splitHostPort(r.Target); ok {
rec.Host = host
rec.Port = port
} else {
rec.Host = r.Target
}
d := r.Details
if d == nil {
return rec
}
// 从 details 提升一级字段(覆盖拆分结果)
if v, ok := d["port"]; ok {
if p, ok := toInt(v); ok {
rec.Port = p
}
}
rec.Service = strVal(d, "service")
rec.Protocol = strVal(d, "protocol")
if banner := strVal(d, "banner"); len(banner) > 200 {
rec.Banner = banner[:200] + "..."
} else {
rec.Banner = banner
}
rec.Title = strVal(d, "title")
rec.URL = strVal(d, "url")
rec.Vulnerability = strVal(d, "vulnerability")
if rec.Vulnerability == "" {
rec.Vulnerability = strVal(d, "vulnerability_name")
}
rec.Username = strVal(d, "username")
rec.Password = strVal(d, "password")
rec.Plugin = strVal(d, "plugin")
rec.Version = strVal(d, "version")
rec.OS = strVal(d, "os")
return rec
}
func (w *StdoutNDJSONWriter) Close() error {
w.mu.Lock()
defer w.mu.Unlock()
return w.writer.Flush()
}
func strVal(d map[string]interface{}, key string) string {
v, ok := d[key]
if !ok {
return ""
}
s, ok := v.(string)
if !ok {
return fmt.Sprintf("%v", v)
}
return s
}
func toInt(v interface{}) (int, bool) {
switch n := v.(type) {
case int:
return n, true
case int64:
return int(n), true
case float64:
return int(n), true
}
return 0, false
}
func splitHostPort(target string) (string, int, bool) {
host, portText, err := net.SplitHostPort(target)
if err != nil {
if strings.Count(target, ":") != 1 {
return "", 0, false
}
parts := strings.SplitN(target, ":", 2)
host, portText = parts[0], parts[1]
}
port, err := strconv.Atoi(portText)
if err != nil {
return "", 0, false
}
if host == "" || port < 1 || port > 65535 {
return "", 0, false
}
return host, port, true
}
+122
View File
@@ -0,0 +1,122 @@
package output
import (
"bufio"
"bytes"
"encoding/json"
"testing"
)
func TestSplitHostPort(t *testing.T) {
tests := []struct {
name string
target string
wantHost string
wantPort int
wantOK bool
}{
{name: "ipv4", target: "192.168.1.1:80", wantHost: "192.168.1.1", wantPort: 80, wantOK: true},
{name: "hostname", target: "example.com:443", wantHost: "example.com", wantPort: 443, wantOK: true},
{name: "bracketed ipv6", target: "[2001:db8::1]:8443", wantHost: "2001:db8::1", wantPort: 8443, wantOK: true},
{name: "bare ipv6 without port", target: "2001:db8::1", wantOK: false},
{name: "invalid port", target: "example.com:abc", wantOK: false},
{name: "port out of range", target: "example.com:65536", wantOK: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
host, port, ok := splitHostPort(tt.target)
if ok != tt.wantOK {
t.Fatalf("splitHostPort(%q) ok = %v, want %v", tt.target, ok, tt.wantOK)
}
if !ok {
return
}
if host != tt.wantHost || port != tt.wantPort {
t.Fatalf("splitHostPort(%q) = (%q, %d), want (%q, %d)", tt.target, host, port, tt.wantHost, tt.wantPort)
}
})
}
}
func TestNewStdoutNDJSONWriter(t *testing.T) {
writer := NewStdoutNDJSONWriter()
if writer == nil || writer.writer == nil {
t.Fatalf("NewStdoutNDJSONWriter = %#v, want initialized writer", writer)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
}
func TestStdoutNDJSONWriterWriteResult(t *testing.T) {
var buf bytes.Buffer
writer := &StdoutNDJSONWriter{writer: bufio.NewWriter(&buf)}
result := &ScanResult{
Type: TypeService,
Target: "[2001:db8::1]:8443",
Status: "OPEN",
Details: map[string]interface{}{
"port": float64(9443),
"service": "https",
"protocol": "tcp",
"banner": 123,
"title": "admin",
"url": "https://[2001:db8::1]:8443",
"vulnerability": "weak credential",
"username": "admin",
"password": "secret",
"plugin": "webtitle",
"version": "1.2.3",
"os": "linux",
},
}
if err := writer.WriteResult(result); err != nil {
t.Fatalf("WriteResult error = %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
var rec ndjsonRecord
if err := json.Unmarshal(bytes.TrimSpace(buf.Bytes()), &rec); err != nil {
t.Fatalf("invalid ndjson output %q: %v", buf.String(), err)
}
if rec.Host != "2001:db8::1" || rec.Port != 9443 {
t.Fatalf("host/port = %q/%d", rec.Host, rec.Port)
}
if rec.Service != "https" || rec.Protocol != "tcp" || rec.Banner != "123" || rec.Title != "admin" {
t.Fatalf("flattened fields missing: %#v", rec)
}
if rec.URL != "https://[2001:db8::1]:8443" || rec.Vulnerability != "weak credential" {
t.Fatalf("url/vuln fields missing: %#v", rec)
}
if rec.Username != "admin" || rec.Password != "secret" || rec.Plugin != "webtitle" || rec.Version != "1.2.3" || rec.OS != "linux" {
t.Fatalf("credential/plugin fields missing: %#v", rec)
}
}
func TestStdoutNDJSONFlattenFallbacks(t *testing.T) {
writer := &StdoutNDJSONWriter{writer: bufio.NewWriter(&bytes.Buffer{})}
rec := writer.flatten(&ScanResult{
Type: TypeHost,
Target: "2001:db8::1",
Status: "ALIVE",
Details: map[string]interface{}{
"port": int64(22),
},
})
if rec.Host != "2001:db8::1" || rec.Port != 22 {
t.Fatalf("flatten fallback = %#v", rec)
}
if got, ok := toInt("22"); ok || got != 0 {
t.Fatalf("toInt string = %d/%v, want 0/false", got, ok)
}
if got := strVal(map[string]interface{}{}, "missing"); got != "" {
t.Fatalf("missing strVal = %q, want empty", got)
}
}
+134 -62
View File
@@ -5,21 +5,70 @@ import (
"encoding/csv"
"encoding/json"
"fmt"
"net"
"os"
"strings"
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
)
// escapeControlChars 转义控制字符
func escapeControlChars(s string) string {
replacer := strings.NewReplacer(
"\r\n", "\\r\\n",
"\n", "\\n",
"\r", "\\r",
"\t", "\\t",
)
return replacer.Replace(s)
s = strings.ToValidUTF8(s, "?")
var b strings.Builder
for _, r := range s {
switch r {
case '\n':
b.WriteString("\\n")
case '\r':
b.WriteString("\\r")
case '\t':
b.WriteString("\\t")
default:
if r < 0x20 || r == 0x7f {
fmt.Fprintf(&b, "\\x%02x", r)
continue
}
b.WriteRune(r)
}
}
return b.String()
}
func truncateString(s string, maxRunes int) string {
if maxRunes < 0 {
return s
}
for i := range s {
if maxRunes == 0 {
return s[:i] + "..."
}
maxRunes--
}
return s
}
func targetWithPort(target string, port interface{}) string {
if port == nil {
return target
}
if _, _, err := net.SplitHostPort(target); err == nil {
return target
}
portText := fmt.Sprint(port)
if strings.TrimSpace(portText) == "" {
return target
}
if strings.HasPrefix(target, "[") && strings.HasSuffix(target, "]") {
target = strings.TrimPrefix(strings.TrimSuffix(target, "]"), "[")
}
if strings.Count(target, ":") == 1 {
return target
}
return net.JoinHostPort(target, portText)
}
// =============================================================================
@@ -99,13 +148,13 @@ func (w *TXTWriter) Write(result *ScanResult) error {
func (w *TXTWriter) getSeparator(newType ResultType) string {
switch newType {
case TypeHost:
return "# ===== 存活主机 ====="
return i18n.GetText("output_section_hosts")
case TypePort:
return "# ===== 开放端口 ====="
return i18n.GetText("output_section_ports")
case TypeService:
return "# ===== 服务信息 ====="
return i18n.GetText("output_section_services")
case TypeVuln:
return "# ===== 漏洞信息 ====="
return i18n.GetText("output_section_vulns")
default:
return "# ===================="
}
@@ -119,7 +168,7 @@ func (w *TXTWriter) formatLine(result *ScanResult) string {
case TypePort:
port := w.getDetail(result, "port")
if port != nil {
return fmt.Sprintf("%s:%v", result.Target, port)
return targetWithPort(result.Target, port)
}
return result.Target
case TypeService:
@@ -152,12 +201,7 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
}
// 非Web服务:ip:port service banner
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
var parts []string
parts = append(parts, target)
@@ -165,10 +209,8 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
parts = append(parts, service)
}
if banner != "" {
if len(banner) > 100 {
banner = banner[:100] + "..."
}
banner = escapeControlChars(banner)
banner = truncateString(banner, 100)
parts = append(parts, banner)
}
return strings.Join(parts, " ")
@@ -176,20 +218,9 @@ func (w *TXTWriter) formatServiceLine(result *ScanResult) string {
// formatWebServiceLine 格式化Web服务结果
func (w *TXTWriter) formatWebServiceLine(result *ScanResult) string {
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
protocol := "http"
service := w.getDetailStr(result, "service")
if service == "https" || strings.Contains(target, ":443") {
protocol = "https"
}
url := fmt.Sprintf("%s://%s", protocol, target)
url := fmt.Sprintf("%s://%s", w.webProtocol(result, target), target)
title := w.getDetailStr(result, "title")
status := w.getDetail(result, "status")
server := w.getDetailStr(result, "server")
@@ -252,6 +283,9 @@ func (w *TXTWriter) formatVulnLine(result *ScanResult) string {
}
vuln := w.getDetailStr(result, "vulnerability")
if vuln == "" {
vuln = w.getDetailStr(result, "vulnerability_name")
}
if vuln != "" {
return fmt.Sprintf("%s %s", result.Target, vuln)
}
@@ -319,13 +353,17 @@ func (w *TXTWriter) Close() error {
os.Remove(w.realtimePath)
}
var firstErr error
if err := w.bufWriter.Flush(); err != nil {
return err
firstErr = err
}
if err := w.file.Sync(); err != nil {
return err
if err := w.file.Sync(); err != nil && firstErr == nil {
firstErr = err
}
return w.file.Close()
if err := w.file.Close(); err != nil && firstErr == nil {
firstErr = err
}
return firstErr
}
// writeSection 写入一个分类的所有结果
@@ -355,27 +393,16 @@ func (w *TXTWriter) writeWebServices() {
continue
}
target := result.Target
if !strings.Contains(target, ":") {
if port := w.getDetail(result, "port"); port != nil {
target = fmt.Sprintf("%s:%v", target, port)
}
}
target := targetWithPort(result.Target, w.getDetail(result, "port"))
protocol := "http"
service := w.getDetailStr(result, "service")
if service == "https" || strings.Contains(target, ":443") {
protocol = "https"
}
urls = append(urls, fmt.Sprintf("%s://%s", protocol, target))
urls = append(urls, fmt.Sprintf("%s://%s", w.webProtocol(result, target), target))
}
if len(urls) == 0 {
return
}
_, _ = w.bufWriter.WriteString("# ===== Web服务 =====\n")
_, _ = w.bufWriter.WriteString(i18n.GetText("output_section_web_services") + "\n")
for _, url := range urls {
_, _ = w.bufWriter.WriteString(url + "\n")
}
@@ -397,6 +424,19 @@ func (w *TXTWriter) isWebService(result *ScanResult) bool {
return service == "http" || service == "https"
}
func (w *TXTWriter) webProtocol(result *ScanResult, target string) string {
protocol := strings.ToLower(w.getDetailStr(result, "protocol"))
if protocol == "http" || protocol == "https" {
return protocol
}
service := strings.ToLower(w.getDetailStr(result, "service"))
if service == "https" || strings.Contains(target, ":443") {
return "https"
}
return "http"
}
// GetFormat 获取格式类型
func (w *TXTWriter) GetFormat() Format {
return FormatTXT
@@ -647,7 +687,7 @@ func (w *CSVWriter) Close() error {
// 写入各分类
w.writeSection("# Hosts", []string{"Target"}, w.buffer.HostResults, w.formatHostRecord)
w.writeSection("# Ports", []string{"Target", "Port", "Status"}, w.buffer.PortResults, w.formatPortRecord)
w.writeSection("# Services", []string{"Target", "Service", "Version", "Banner"}, w.buffer.ServiceResults, w.formatServiceRecord)
w.writeSection("# Services", []string{"Target", "Service", "Version", "Title", "Status", "Server", "Fingerprints", "Banner"}, w.buffer.ServiceResults, w.formatServiceRecord)
w.writeSection("# Vulns", []string{"Target", "Type", "Details"}, w.buffer.VulnResults, w.formatVulnRecord)
w.closed = true
@@ -697,7 +737,7 @@ func (w *CSVWriter) formatPortRecord(result *ScanResult) []string {
}
func (w *CSVWriter) formatServiceRecord(result *ScanResult) []string {
service, version, banner := "", "", ""
service, version, title, status, server, fingerprints, banner := "", "", "", "", "", "", ""
if result.Details != nil {
if s, ok := result.Details["service"].(string); ok {
service = s
@@ -705,33 +745,65 @@ func (w *CSVWriter) formatServiceRecord(result *ScanResult) []string {
if s, ok := result.Details["name"].(string); ok && service == "" {
service = s
}
if s, ok := result.Details["plugin"].(string); ok && service == "" {
service = s
}
if v, ok := result.Details["version"].(string); ok {
version = v
}
if t, ok := result.Details["title"].(string); ok {
title = escapeControlChars(t)
}
if s, ok := result.Details["status"]; ok && s != nil && s != 0 {
status = fmt.Sprintf("%v", s)
}
if s, ok := result.Details["server"].(string); ok {
server = escapeControlChars(s)
}
fingerprints = formatFingerprints(result.Details["fingerprints"])
if b, ok := result.Details["banner"].(string); ok {
banner = escapeControlChars(b)
if len(banner) > 100 {
banner = banner[:100] + "..."
}
banner = truncateString(banner, 100)
}
}
target := result.Target
if !strings.Contains(target, ":") {
if p, ok := result.Details["port"]; ok {
target = fmt.Sprintf("%s:%v", target, p)
if result.Details != nil {
target = targetWithPort(target, result.Details["port"])
}
return []string{target, service, version, title, status, server, fingerprints, banner}
}
func formatFingerprints(value interface{}) string {
switch v := value.(type) {
case []string:
return strings.Join(v, ",")
case []interface{}:
parts := make([]string, 0, len(v))
for _, item := range v {
if s, ok := item.(string); ok && s != "" {
parts = append(parts, s)
}
}
return strings.Join(parts, ",")
default:
return ""
}
return []string{target, service, version, banner}
}
func (w *CSVWriter) formatVulnRecord(result *ScanResult) []string {
vulnType := ""
vulnName := result.Status
if result.Details != nil {
if t, ok := result.Details["type"].(string); ok {
vulnType = t
}
if v, ok := result.Details["vulnerability"].(string); ok && v != "" {
vulnName = v
} else if v, ok := result.Details["vulnerability_name"].(string); ok && v != "" {
vulnName = v
}
}
return []string{result.Target, vulnType, result.Status}
return []string{result.Target, vulnType, vulnName}
}
// GetFormat 获取格式类型
+612 -1
View File
@@ -9,6 +9,7 @@ import (
"sync"
"testing"
"time"
"unicode/utf8"
)
/*
@@ -57,6 +58,119 @@ func createTestResult(resultType ResultType, target, status string, details map[
}
}
func TestTargetWithPortIPv6(t *testing.T) {
tests := []struct {
name string
target string
port interface{}
want string
}{
{name: "ipv4 without port", target: "192.168.1.1", port: 80, want: "192.168.1.1:80"},
{name: "ipv4 with port", target: "192.168.1.1:80", port: 443, want: "192.168.1.1:80"},
{name: "ipv6 without port", target: "2001:db8::1", port: 443, want: "[2001:db8::1]:443"},
{name: "bracketed ipv6 without port", target: "[2001:db8::1]", port: 443, want: "[2001:db8::1]:443"},
{name: "ipv6 with port", target: "[2001:db8::1]:443", port: 80, want: "[2001:db8::1]:443"},
{name: "empty port", target: "example.com", port: "", want: "example.com"},
{name: "blank port", target: "example.com", port: " \t", want: "example.com"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := targetWithPort(tt.target, tt.port); got != tt.want {
t.Fatalf("targetWithPort(%q, %v) = %q, want %q", tt.target, tt.port, got, tt.want)
}
})
}
}
func TestScanResultFormatDetailsAndDefaultManagerConfig(t *testing.T) {
result := &ScanResult{
Details: map[string]interface{}{
"service": "ssh",
"port": 22,
"banner": "OpenSSH",
},
}
got := result.FormatDetails(";", "%s=%v")
want := "banner=OpenSSH;port=22;service=ssh"
if got != want {
t.Fatalf("FormatDetails = %q, want %q", got, want)
}
empty := (&ScanResult{}).FormatDetails(";", "%s=%v")
if empty != "" {
t.Fatalf("empty FormatDetails = %q, want empty", empty)
}
cfg := DefaultManagerConfig("out.json", FormatJSON)
if cfg.OutputPath != "out.json" || cfg.Format != FormatJSON {
t.Fatalf("DefaultManagerConfig = %#v", cfg)
}
}
func TestCSVWriterFormatRecords(t *testing.T) {
writer := &CSVWriter{}
host := writer.formatHostRecord(&ScanResult{Target: "192.168.1.1"})
if len(host) != 1 || host[0] != "192.168.1.1" {
t.Fatalf("host record = %#v", host)
}
port := writer.formatPortRecord(&ScanResult{
Target: "192.168.1.1",
Details: map[string]interface{}{"port": 22},
})
if got, want := strings.Join(port, "|"), "192.168.1.1|22|open"; got != want {
t.Fatalf("port record = %q, want %q", got, want)
}
longBanner := strings.Repeat("界", 105)
service := writer.formatServiceRecord(&ScanResult{
Target: "2001:db8::1",
Details: map[string]interface{}{
"port": 443,
"name": "https",
"version": "1.2.3",
"title": "hello\nworld",
"status": 200,
"server": "nginx\r\nunit",
"fingerprints": []interface{}{"fp1", "", "fp2", 3},
"banner": longBanner,
},
})
if service[0] != "[2001:db8::1]:443" || service[1] != "https" || service[2] != "1.2.3" {
t.Fatalf("service identity fields = %#v", service)
}
if service[3] != "hello\\nworld" || service[4] != "200" || service[5] != "nginx\\r\\nunit" {
t.Fatalf("service text fields = %#v", service)
}
if service[6] != "fp1,fp2" {
t.Fatalf("fingerprints = %q, want fp1,fp2", service[6])
}
if !utf8.ValidString(service[7]) || len([]rune(service[7])) != 103 || !strings.HasSuffix(service[7], "...") {
t.Fatalf("truncated banner = len %d value %q", len(service[7]), service[7])
}
vuln := writer.formatVulnRecord(&ScanResult{
Target: "http://example.com",
Status: "vulnerable",
Details: map[string]interface{}{"type": "poc"},
})
if got, want := strings.Join(vuln, "|"), "http://example.com|poc|vulnerable"; got != want {
t.Fatalf("vuln record = %q, want %q", got, want)
}
if got := formatFingerprints([]string{"a", "b"}); got != "a,b" {
t.Fatalf("string fingerprints = %q", got)
}
if got := formatFingerprints(123); got != "" {
t.Fatalf("unsupported fingerprints = %q, want empty", got)
}
if writer.GetFormat() != FormatCSV {
t.Fatalf("csv GetFormat = %q", writer.GetFormat())
}
}
// =============================================================================
// TXTWriter - 基础功能测试
// =============================================================================
@@ -1139,7 +1253,7 @@ func TestCSVWriter_ErrorHandling(t *testing.T) {
// TestCSVWriter_DetailsFormatting 测试CSV的Details字段格式化
//
// CSVWriter 对不同类型有不同的格式:
// - Service类型:Target, Service, Version, Banner
// - Service类型:Target, Service, Version, Title, Status, Server, Fingerprints, Banner
func TestCSVWriter_DetailsFormatting(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test.csv")
@@ -1188,6 +1302,86 @@ func TestCSVWriter_DetailsFormatting(t *testing.T) {
t.Logf("✓ CSV Details格式化测试通过")
}
func TestCSVWriter_WebServiceFields(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test.csv")
writer, _ := NewCSVWriter(filePath)
defer func() { _ = writer.Close() }()
_ = writer.WriteHeader()
result := createTestResult(
TypeService,
"192.168.1.1:80",
"web",
map[string]interface{}{
"plugin": "webtitle",
"is_web": true,
"port": 80,
"title": "Home",
"status": 200,
"server": "nginx",
"fingerprints": []string{"nginx", "php"},
"banner": "HTTP/1.1 200 OK\x00\nServer: nginx",
},
)
_ = writer.Write(result)
writer.Close()
content := readFileContent(t, filePath)
for _, want := range []string{
"Target,Service,Version,Title,Status,Server,Fingerprints,Banner",
"webtitle",
"Home",
"200",
"nginx",
"nginx,php",
"\\x00\\nServer: nginx",
} {
if !strings.Contains(content, want) {
t.Errorf("CSV文件缺少 %q,内容:\n%s", want, content)
}
}
}
func TestTXTWriter_WebServiceProtocolFromDetails(t *testing.T) {
dir := createTestDir(t)
filePath := filepath.Join(dir, "test_web_protocol.txt")
writer, err := NewTXTWriter(filePath)
if err != nil {
t.Fatalf("创建TXTWriter失败: %v", err)
}
result := createTestResult(
TypeService,
"192.168.1.1:8443",
"web",
map[string]interface{}{
"plugin": "webtitle",
"is_web": true,
"port": 8443,
"protocol": "https",
"title": "Home",
"status": 200,
},
)
if err := writer.Write(result); err != nil {
t.Fatalf("Write()失败: %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("Close()失败: %v", err)
}
content := readFileContent(t, filePath)
if !strings.Contains(content, "https://192.168.1.1:8443") {
t.Fatalf("TXT输出缺少HTTPS URL,内容:\n%s", content)
}
if strings.Contains(content, "http://192.168.1.1:8443") {
t.Fatalf("TXT输出不应把HTTPS目标降级为HTTP,内容:\n%s", content)
}
}
// TestJSONWriter_FlushAndFormat 测试JSON的Flush和GetFormat
func TestJSONWriter_FlushAndFormat(t *testing.T) {
dir := createTestDir(t)
@@ -1534,3 +1728,420 @@ func TestManager_ConcurrentSave(t *testing.T) {
t.Logf("✓ 并发保存测试通过(%d个goroutine,每个%d次,输出%d行)",
numGoroutines, savesPerGoroutine, len(lines))
}
// =============================================================================
// TXTWriter - 内部格式化函数覆盖率测试
// =============================================================================
// newTestTXTWriter 创建用于单元测试的 TXTWriter(写到临时文件,调用方负责 Close)
func newTestTXTWriter(t *testing.T) *TXTWriter {
t.Helper()
w, err := NewTXTWriter(filepath.Join(t.TempDir(), "unit.txt"))
if err != nil {
t.Fatalf("创建 TXTWriter 失败: %v", err)
}
return w
}
// TestFormatServiceLine 覆盖 formatServiceLine 的各分支
func TestFormatServiceLine(t *testing.T) {
w := newTestTXTWriter(t)
defer w.Close()
tests := []struct {
name string
details map[string]interface{}
want []string // 输出中必须包含的子串
notwant []string // 输出中不应包含的子串
}{
{
name: "非web服务带service和banner",
details: map[string]interface{}{
"port": 22,
"service": "ssh",
"banner": "OpenSSH_8.0",
},
want: []string{"ssh", "OpenSSH_8.0"},
notwant: []string{"http://", "https://"},
},
{
name: "非web服务只有service",
details: map[string]interface{}{
"port": 3306,
"service": "mysql",
},
want: []string{"mysql"},
notwant: []string{"http://"},
},
{
name: "非web服务无banner",
details: map[string]interface{}{
"port": 21,
"service": "ftp",
},
want: []string{"ftp"},
},
{
name: "service=http 走 web 分支",
details: map[string]interface{}{
"port": 80,
"service": "http",
"title": "Home",
"status": 200,
},
want: []string{"http://", "Home"},
notwant: []string{"ssh"},
},
{
name: "service=https 走 web 分支",
details: map[string]interface{}{
"port": 443,
"service": "https",
"title": "Secure",
"status": 200,
},
want: []string{"https://", "Secure"},
},
{
name: "is_web=true 走 web 分支",
details: map[string]interface{}{
"port": 8080,
"is_web": true,
"title": "Dashboard",
"status": 302,
},
want: []string{"http://", "Dashboard"},
},
{
name: "有 status 字段触发 web 分支",
details: map[string]interface{}{
"port": 8080,
"status": 200,
},
want: []string{"http://"},
},
{
name: "有 server 字段触发 web 分支",
details: map[string]interface{}{
"port": 8080,
"server": "nginx",
},
want: []string{"http://", "nginx"},
},
{
name: "banner 含控制字符被转义",
details: map[string]interface{}{
"port": 9999,
"service": "custom",
"banner": "hello\nworld\r\n",
},
want: []string{"\\n", "\\r"},
notwant: []string{"http://"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := &ScanResult{
Target: "192.168.1.1",
Type: TypeService,
Details: tt.details,
}
got := w.formatServiceLine(result)
for _, s := range tt.want {
if !strings.Contains(got, s) {
t.Errorf("formatServiceLine() = %q,缺少 %q", got, s)
}
}
for _, s := range tt.notwant {
if strings.Contains(got, s) {
t.Errorf("formatServiceLine() = %q,不应含 %q", got, s)
}
}
})
}
}
// TestGetFingerprints 覆盖 getFingerprints 的各类型分支
func TestGetFingerprints(t *testing.T) {
w := newTestTXTWriter(t)
defer w.Close()
tests := []struct {
name string
details map[string]interface{}
want string
}{
{
name: "nil fingerprints",
details: map[string]interface{}{},
want: "",
},
{
name: "[]string 非空",
details: map[string]interface{}{"fingerprints": []string{"nginx", "php"}},
want: "[nginx,php]",
},
{
name: "[]string 空slice",
details: map[string]interface{}{"fingerprints": []string{}},
want: "",
},
{
name: "[]interface{} 非空",
details: map[string]interface{}{"fingerprints": []interface{}{"wordpress", "jquery"}},
want: "[wordpress,jquery]",
},
{
name: "[]interface{} 含数字",
details: map[string]interface{}{"fingerprints": []interface{}{"apache", 2}},
want: "[apache,2]",
},
{
name: "[]interface{} 空slice",
details: map[string]interface{}{"fingerprints": []interface{}{}},
want: "",
},
{
name: "不支持的类型返回空",
details: map[string]interface{}{"fingerprints": "just-a-string"},
want: "",
},
{
name: "单个元素",
details: map[string]interface{}{"fingerprints": []string{"tomcat"}},
want: "[tomcat]",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := &ScanResult{Target: "1.2.3.4", Details: tt.details}
got := w.getFingerprints(result)
if got != tt.want {
t.Errorf("getFingerprints() = %qwant %q", got, tt.want)
}
})
}
}
// TestFormatVulnLine 覆盖 formatVulnLine 的各分支
func TestFormatVulnLine(t *testing.T) {
w := newTestTXTWriter(t)
defer w.Close()
tests := []struct {
name string
target string
status string
details map[string]interface{}
want string
}{
{
name: "weak_credential 带 service",
target: "192.168.1.1:22",
details: map[string]interface{}{
"type": "weak_credential",
"service": "ssh",
"username": "root",
"password": "123456",
},
want: "192.168.1.1:22 ssh root/123456",
},
{
name: "weak_credential 不带 service",
target: "192.168.1.1:3306",
details: map[string]interface{}{
"type": "weak_credential",
"username": "admin",
"password": "pass",
},
want: "192.168.1.1:3306 admin/pass",
},
{
name: "有 vulnerability 字段",
target: "10.0.0.1",
details: map[string]interface{}{
"type": "poc",
"vulnerability": "CVE-2024-1234",
},
want: "10.0.0.1 CVE-2024-1234",
},
{
name: "无 vulnerability 字段回退到 status",
target: "10.0.0.2",
status: "VULNERABLE",
details: map[string]interface{}{
"type": "unknown",
},
want: "10.0.0.2 VULNERABLE",
},
{
name: "空 details 回退到 status",
target: "10.0.0.3",
status: "poc_hit",
details: map[string]interface{}{},
want: "10.0.0.3 poc_hit",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := &ScanResult{
Target: tt.target,
Status: tt.status,
Type: TypeVuln,
Details: tt.details,
}
got := w.formatVulnLine(result)
if got != tt.want {
t.Errorf("formatVulnLine() = %qwant %q", got, tt.want)
}
})
}
}
// TestIsWebService 覆盖 isWebService 的各判断分支
func TestIsWebService(t *testing.T) {
w := newTestTXTWriter(t)
defer w.Close()
tests := []struct {
name string
details map[string]interface{}
want bool
}{
{
name: "is_web=true",
details: map[string]interface{}{"is_web": true},
want: true,
},
{
name: "is_web=false 无其他标志",
details: map[string]interface{}{"is_web": false},
want: false,
},
{
name: "有 status 字段",
details: map[string]interface{}{"status": 200},
want: true,
},
{
name: "status=nil 不触发",
details: map[string]interface{}{},
want: false,
},
{
name: "有非空 server 字段",
details: map[string]interface{}{"server": "nginx"},
want: true,
},
{
name: "空 server 字段不触发",
details: map[string]interface{}{"server": ""},
want: false,
},
{
name: "service=http",
details: map[string]interface{}{"service": "http"},
want: true,
},
{
name: "service=https",
details: map[string]interface{}{"service": "https"},
want: true,
},
{
name: "service=ssh 不是 web",
details: map[string]interface{}{"service": "ssh"},
want: false,
},
{
name: "nil Details",
details: nil,
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := &ScanResult{Target: "1.2.3.4", Details: tt.details}
got := w.isWebService(result)
if got != tt.want {
t.Errorf("isWebService() = %vwant %v", got, tt.want)
}
})
}
}
// TestWebProtocol 覆盖 webProtocol 的各判断分支
func TestWebProtocol(t *testing.T) {
w := newTestTXTWriter(t)
defer w.Close()
tests := []struct {
name string
target string
details map[string]interface{}
want string
}{
{
name: "protocol=https 直接返回",
target: "1.2.3.4:8443",
details: map[string]interface{}{"protocol": "https"},
want: "https",
},
{
name: "protocol=http 直接返回",
target: "1.2.3.4:8080",
details: map[string]interface{}{"protocol": "http"},
want: "http",
},
{
name: "protocol=HTTPS 大小写不敏感",
target: "1.2.3.4:443",
details: map[string]interface{}{"protocol": "HTTPS"},
want: "https",
},
{
name: "service=https 回退",
target: "1.2.3.4:8080",
details: map[string]interface{}{"service": "https"},
want: "https",
},
{
name: "target 含 :443 回退 https",
target: "example.com:443",
details: map[string]interface{}{},
want: "https",
},
{
name: "无任何标志默认 http",
target: "1.2.3.4:8080",
details: map[string]interface{}{},
want: "http",
},
{
name: "service=http 默认 http",
target: "1.2.3.4:80",
details: map[string]interface{}{"service": "http"},
want: "http",
},
{
name: "protocol 为其他值走 service 分支",
target: "1.2.3.4:9000",
details: map[string]interface{}{"protocol": "tcp", "service": "https"},
want: "https",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := &ScanResult{Target: tt.target, Details: tt.details}
got := w.webProtocol(result, tt.target)
if got != tt.want {
t.Errorf("webProtocol() = %qwant %q", got, tt.want)
}
})
}
}
+17 -1
View File
@@ -15,10 +15,18 @@ import (
// ResultOutput 全局输出管理器
var ResultOutput *output.Manager
// StdoutWriter silent模式下的NDJSON stdout写入器
var StdoutWriter *output.StdoutNDJSONWriter
// InitOutput 初始化输出系统
func InitOutput() error {
fv := GetFlagVars()
// silent模式:初始化NDJSON stdout写入器(独立于文件输出)
if fv.Silent {
StdoutWriter = output.NewStdoutNDJSONWriter()
}
// 用户通过-no flag禁用保存时,跳过文件初始化避免不必要的资源开销
if fv.DisableSave {
return nil
@@ -59,6 +67,9 @@ func InitOutput() error {
// CloseOutput 关闭输出系统
func CloseOutput() error {
if StdoutWriter != nil {
_ = StdoutWriter.Close()
}
if ResultOutput == nil {
return nil
}
@@ -80,8 +91,13 @@ func SaveResult(result *output.ScanResult) error {
"details": result.Details,
})
// silent模式:NDJSON实时输出到stdout
if StdoutWriter != nil {
_ = StdoutWriter.WriteResult(result)
}
// 用户禁用保存或输出未初始化时,跳过文件保存
if GetFlagVars().DisableSave || ResultOutput == nil {
if GetGlobalConfig().Output.DisableSave || ResultOutput == nil {
return nil
}
return ResultOutput.SaveResult(result)
+171
View File
@@ -0,0 +1,171 @@
package common
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/shadow1ng/fscan/common/output"
)
func readTestFile(t *testing.T, path string) string {
t.Helper()
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return string(content)
}
func preserveOutputAPIGlobals(t *testing.T) {
t.Helper()
globalMu.RLock()
oldConfig := globalConfig
oldState := globalState
globalMu.RUnlock()
oldFlagVars := flagVars
oldResultOutput := ResultOutput
oldStdoutWriter := StdoutWriter
t.Cleanup(func() {
if ResultOutput != nil && ResultOutput != oldResultOutput {
_ = ResultOutput.Close()
}
if StdoutWriter != nil && StdoutWriter != oldStdoutWriter {
_ = StdoutWriter.Close()
}
ClearResultCallback()
globalMu.Lock()
globalConfig = oldConfig
globalState = oldState
globalMu.Unlock()
flagVars = oldFlagVars
ResultOutput = oldResultOutput
StdoutWriter = oldStdoutWriter
})
ClearResultCallback()
flagVars = &FlagVars{}
ResultOutput = nil
StdoutWriter = nil
SetGlobalConfig(NewConfig())
SetGlobalState(NewState())
}
func TestInitOutputValidationAndDefaultExtension(t *testing.T) {
preserveOutputAPIGlobals(t)
flagVars = &FlagVars{DisableSave: true}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput disable save error = %v", err)
}
if ResultOutput != nil {
t.Fatalf("ResultOutput = %#v, want nil when save is disabled", ResultOutput)
}
flagVars = &FlagVars{OutputFormat: "txt"}
if err := InitOutput(); err == nil || !strings.Contains(err.Error(), "output file not specified") {
t.Fatalf("missing output error = %v", err)
}
flagVars = &FlagVars{Outputfile: "out.bad", OutputFormat: "xml"}
if err := InitOutput(); err == nil || !strings.Contains(err.Error(), "invalid output format") {
t.Fatalf("invalid format error = %v", err)
}
dir := t.TempDir()
t.Chdir(dir)
flagVars = &FlagVars{Outputfile: "result.txt", OutputFormat: "json"}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput json error = %v", err)
}
if ResultOutput == nil {
t.Fatal("ResultOutput should be initialized")
}
if err := SaveResult(&output.ScanResult{
Time: time.Date(2026, 6, 13, 1, 2, 3, 0, time.UTC),
Type: output.TypeHost,
Target: "127.0.0.1",
Status: "ALIVE",
}); err != nil {
t.Fatalf("SaveResult json error = %v", err)
}
if err := CloseOutput(); err != nil {
t.Fatalf("CloseOutput error = %v", err)
}
if content := readTestFile(t, filepath.Join(dir, "result.json")); !strings.Contains(content, "127.0.0.1") {
t.Fatalf("result.json content = %q, want saved target", content)
}
}
func TestCloseOutputWithStdoutWriter(t *testing.T) {
preserveOutputAPIGlobals(t)
// 初始化 silent 模式以创建 StdoutWriter
flagVars = &FlagVars{Silent: true, DisableSave: true}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput silent error = %v", err)
}
if StdoutWriter == nil {
t.Fatal("StdoutWriter 应在 Silent 模式下被初始化")
}
// CloseOutput 应正常关闭 StdoutWriter
if err := CloseOutput(); err != nil {
t.Fatalf("CloseOutput with StdoutWriter error = %v", err)
}
}
func TestSaveResultFacadeCallbackAndDisabledSave(t *testing.T) {
preserveOutputAPIGlobals(t)
cfg := NewConfig()
cfg.Output.DisableSave = true
SetGlobalConfig(cfg)
flagVars = &FlagVars{DisableSave: true}
if err := InitOutput(); err != nil {
t.Fatalf("InitOutput disable save error = %v", err)
}
called := false
SetResultCallback(func(payload interface{}) {
called = true
data, ok := payload.(map[string]interface{})
if !ok {
t.Fatalf("callback payload type = %T", payload)
}
if data["type"] != string(output.TypeVuln) || data["target"] != "http://example.com" {
t.Fatalf("callback payload = %#v", data)
}
})
if err := SaveResult(nil); err != nil {
t.Fatalf("SaveResult nil error = %v", err)
}
if called {
t.Fatal("nil result should not notify callback")
}
if err := SaveResult(&output.ScanResult{
Type: output.TypeVuln,
Target: "http://example.com",
Status: "vulnerable",
Details: map[string]interface{}{"type": "poc"},
}); err != nil {
t.Fatalf("SaveResult disabled save error = %v", err)
}
if !called {
t.Fatal("callback was not notified")
}
if err := CloseOutput(); err != nil {
t.Fatalf("CloseOutput disabled save error = %v", err)
}
}
+3
View File
@@ -45,6 +45,9 @@ func applyLogLevel() {
StartTime: GetGlobalState().GetStartTime(),
LevelColors: logging.GetDefaultLevelColors(),
}
if fv.Debug {
config.DebugLogFile = "fscan_debug.log"
}
newLogger := logging.NewLogger(config)
newLogger.SetCoordinatedOutput(LogWithProgress)
-9
View File
@@ -21,15 +21,6 @@ const (
MaxPort = 65535
)
// =============================================================================
// IP/主机解析常量
// =============================================================================
const (
// SimpleMaxHosts 最大主机数量限制
SimpleMaxHosts = 10000
)
// =============================================================================
// 哈希验证常量
// =============================================================================
+577
View File
@@ -0,0 +1,577 @@
package parsers
import (
"bufio"
"context"
"fmt"
"net"
"os"
"strconv"
"strings"
"github.com/shadow1ng/fscan/common/i18n"
)
const DefaultHostBatchSize = 4096
type HostIterator struct {
sources []hostSource
current hostSource
exclude *hostMatcher
}
func NewHostIterator(host string, filename string, nohosts ...string) (*HostIterator, error) {
var sources []hostSource
if filename != "" {
fileSrc, err := newFileHostSource(filename)
if err != nil {
return nil, err
}
sources = append(sources, fileSrc)
}
hostSources, err := newHostSources(host)
if err != nil {
closeHostSources(sources)
return nil, err
}
sources = append(sources, hostSources...)
matcher := newHostMatcher()
for _, exclude := range nohosts {
if strings.TrimSpace(exclude) == "" {
continue
}
if err := matcher.add(exclude); err != nil {
closeHostSources(sources)
return nil, err
}
}
return &HostIterator{
sources: sources,
exclude: matcher,
}, nil
}
func (it *HostIterator) Close() error {
if it == nil {
return nil
}
var firstErr error
if it.current != nil {
firstErr = it.current.Close()
it.current = nil
}
for _, src := range it.sources {
if err := src.Close(); err != nil && firstErr == nil {
firstErr = err
}
}
it.sources = nil
return firstErr
}
func (it *HostIterator) Next() (string, bool, error) {
for {
if it.current == nil {
if len(it.sources) == 0 {
return "", false, nil
}
it.current = it.sources[0]
it.sources = it.sources[1:]
}
host, ok, err := it.current.Next()
if err != nil {
return "", false, err
}
if !ok {
if err := it.current.Close(); err != nil {
return "", false, err
}
it.current = nil
continue
}
if it.exclude != nil && it.exclude.match(host) {
continue
}
return host, true, nil
}
}
func (it *HostIterator) NextBatch(ctx context.Context, size int) ([]string, error) {
if size <= 0 {
size = DefaultHostBatchSize
}
batch := make([]string, 0, size)
seen := make(map[string]struct{}, size)
for len(batch) < size {
select {
case <-ctx.Done():
return batch, ctx.Err()
default:
}
host, ok, err := it.Next()
if err != nil {
return batch, err
}
if !ok {
return batch, nil
}
if _, exists := seen[host]; exists {
continue
}
seen[host] = struct{}{}
batch = append(batch, host)
}
return batch, nil
}
type hostSource interface {
Next() (string, bool, error)
Close() error
}
type singleHostSource struct {
host string
done bool
}
func (s *singleHostSource) Next() (string, bool, error) {
if s.done {
return "", false, nil
}
s.done = true
return s.host, true, nil
}
func (s *singleHostSource) Close() error { return nil }
type cidrHostSource struct {
current uint32
end uint32
done bool
}
func (s *cidrHostSource) Next() (string, bool, error) {
if s.done || s.current > s.end {
return "", false, nil
}
host := uint32ToIP(s.current)
if s.current == s.end {
s.done = true
} else {
s.current++
}
return host, true, nil
}
func (s *cidrHostSource) Close() error { return nil }
type fileHostSource struct {
file *os.File
scanner *bufio.Scanner
current hostSource
}
func newFileHostSource(filename string) (*fileHostSource, error) {
file, err := os.Open(filename)
if err != nil {
return nil, err
}
src := &fileHostSource{
file: file,
scanner: bufio.NewScanner(file),
}
src.scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
return src, nil
}
func (s *fileHostSource) Next() (string, bool, error) {
for {
if s.current != nil {
host, ok, err := s.current.Next()
if err != nil {
return "", false, err
}
if ok {
return host, true, nil
}
_ = s.current.Close()
s.current = nil
}
if !s.scanner.Scan() {
if err := s.scanner.Err(); err != nil {
return "", false, err
}
return "", false, nil
}
line := strings.TrimSpace(s.scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
sources, err := newHostSources(line)
if err != nil || len(sources) == 0 {
continue
}
if len(sources) == 1 {
s.current = sources[0]
continue
}
s.current = &multiHostSource{sources: sources}
}
}
func (s *fileHostSource) Close() error {
if s.current != nil {
_ = s.current.Close()
s.current = nil
}
if s.file == nil {
return nil
}
err := s.file.Close()
s.file = nil
return err
}
type multiHostSource struct {
sources []hostSource
current hostSource
}
func (s *multiHostSource) Next() (string, bool, error) {
for {
if s.current == nil {
if len(s.sources) == 0 {
return "", false, nil
}
s.current = s.sources[0]
s.sources = s.sources[1:]
}
host, ok, err := s.current.Next()
if err != nil {
return "", false, err
}
if ok {
return host, true, nil
}
_ = s.current.Close()
s.current = nil
}
}
func (s *multiHostSource) Close() error {
if s.current != nil {
_ = s.current.Close()
s.current = nil
}
closeHostSources(s.sources)
s.sources = nil
return nil
}
func newHostSources(host string) ([]hostSource, error) {
var sources []hostSource
for _, h := range strings.Split(host, ",") {
h = strings.TrimSpace(h)
if h == "" {
continue
}
src, err := newHostSource(h)
if err != nil {
closeHostSources(sources)
return nil, err
}
sources = append(sources, src)
}
return sources, nil
}
func newHostSource(host string) (hostSource, error) {
switch {
case host == "192":
return newCIDRHostSource("192.168.0.0/16")
case host == "172":
return newCIDRHostSource("172.16.0.0/12")
case host == "10":
return newCIDRHostSource("10.0.0.0/8")
case strings.Contains(host, "/"):
src, err := newCIDRHostSource(host)
if err != nil {
return nil, fmt.Errorf(i18n.Tr("parser_cidr_failed", host)+": %w", err)
}
return src, nil
case strings.Contains(host, "-") && !strings.Contains(host, ":") && looksLikeIPRange(host):
src, err := newRangeHostSource(host)
if err != nil {
return nil, fmt.Errorf(i18n.Tr("parser_ip_range_failed", host)+": %w", err)
}
return src, nil
default:
return &singleHostSource{host: host}, nil
}
}
func newCIDRHostSource(cidr string) (hostSource, error) {
_, ipNet, err := net.ParseCIDR(cidr)
if err != nil {
return nil, err
}
start, ok := ipToUint32(ipNet.IP)
if !ok {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
ones, bits := ipNet.Mask.Size()
if bits != 32 {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
size := uint64(1) << uint(32-ones)
end := start + uint32(size-1)
if size > 2 {
start++
end--
}
return &cidrHostSource{current: start, end: end}, nil
}
func newRangeHostSource(rangeStr string) (hostSource, error) {
parts := strings.Split(rangeStr, "-")
if len(parts) != 2 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_fmt", rangeStr))
}
startIPStr := strings.TrimSpace(parts[0])
endIPStr := strings.TrimSpace(parts[1])
startIP := net.ParseIP(startIPStr)
if startIP == nil {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_start_ip", startIPStr))
}
if len(endIPStr) < 4 || !strings.Contains(endIPStr, ".") {
endNum, err := strconv.Atoi(endIPStr)
if err != nil || endNum > 255 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_end_val", endIPStr))
}
parts := strings.Split(startIPStr, ".")
if len(parts) != 4 {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_fmt", startIPStr))
}
parts[3] = strconv.Itoa(endNum)
endIPStr = strings.Join(parts, ".")
}
start, ok := ipToUint32(startIP)
if !ok {
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
end, ok := ipToUint32(net.ParseIP(endIPStr))
if !ok {
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_end_ip", endIPStr))
}
if start > end {
return nil, fmt.Errorf("%s", i18n.GetText("parser_start_gt_end"))
}
return &cidrHostSource{current: start, end: end}, nil
}
func closeHostSources(sources []hostSource) {
for _, src := range sources {
_ = src.Close()
}
}
type hostMatcher struct {
exact map[string]struct{}
ranges []ipRange
}
type ipRange struct {
start uint32
end uint32
}
func newHostMatcher() *hostMatcher {
return &hostMatcher{exact: make(map[string]struct{})}
}
func (m *hostMatcher) add(input string) error {
for _, h := range strings.Split(input, ",") {
h = strings.TrimSpace(h)
if h == "" {
continue
}
switch {
case h == "192":
if err := m.addCIDR("192.168.0.0/16"); err != nil {
return err
}
case h == "172":
if err := m.addCIDR("172.16.0.0/12"); err != nil {
return err
}
case h == "10":
if err := m.addCIDR("10.0.0.0/8"); err != nil {
return err
}
case strings.Contains(h, "/"):
if err := m.addCIDR(h); err != nil {
return err
}
case strings.Contains(h, "-") && !strings.Contains(h, ":") && looksLikeIPRange(h):
if err := m.addRange(h); err != nil {
return err
}
default:
m.exact[h] = struct{}{}
}
}
return nil
}
func (m *hostMatcher) addCIDR(cidr string) error {
src, err := newCIDRHostSource(cidr)
if err != nil {
return err
}
rangeSrc, ok := src.(*cidrHostSource)
if !ok {
return fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
m.ranges = append(m.ranges, ipRange{start: rangeSrc.current, end: rangeSrc.end})
return nil
}
func (m *hostMatcher) addRange(rangeStr string) error {
src, err := newRangeHostSource(rangeStr)
if err != nil {
return err
}
rangeSrc, ok := src.(*cidrHostSource)
if !ok {
return fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
m.ranges = append(m.ranges, ipRange{start: rangeSrc.current, end: rangeSrc.end})
return nil
}
func (m *hostMatcher) match(host string) bool {
if _, ok := m.exact[host]; ok {
return true
}
ip, ok := ipToUint32(net.ParseIP(host))
if !ok {
return false
}
for _, r := range m.ranges {
if ip >= r.start && ip <= r.end {
return true
}
}
return false
}
func ipToUint32(ip net.IP) (uint32, bool) {
ip4 := ip.To4()
if ip4 == nil {
return 0, false
}
return uint32(ip4[0])<<24 | uint32(ip4[1])<<16 | uint32(ip4[2])<<8 | uint32(ip4[3]), true
}
func uint32ToIP(v uint32) string {
return fmt.Sprintf("%d.%d.%d.%d", byte(v>>24), byte(v>>16), byte(v>>8), byte(v))
}
// EstimateHostCount 快速估算主机总数(不消费 iterator)
func EstimateHostCount(host string, filename string) int64 {
var total int64
if filename != "" {
if f, err := os.Open(filename); err == nil {
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
total += estimateHostEntry(line)
}
_ = f.Close()
}
}
for _, h := range strings.Split(host, ",") {
h = strings.TrimSpace(h)
if h != "" {
total += estimateHostEntry(h)
}
}
return total
}
func estimateHostEntry(entry string) int64 {
switch {
case entry == "192":
return 65536 // /16
case entry == "172":
return 1 << 20 // /12
case entry == "10":
return 1 << 24 // /8
case strings.Contains(entry, "/"):
_, ipNet, err := net.ParseCIDR(entry)
if err != nil {
return 1
}
ones, bits := ipNet.Mask.Size()
if bits != 32 {
return 1
}
size := int64(1) << uint(32-ones)
if size > 2 {
size -= 2
}
return size
case strings.Contains(entry, "-") && !strings.Contains(entry, ":") && looksLikeIPRange(entry):
parts := strings.SplitN(entry, "-", 2)
startIP := net.ParseIP(strings.TrimSpace(parts[0]))
if startIP == nil {
return 1
}
startU, ok := ipToUint32(startIP)
if !ok {
return 1
}
endStr := strings.TrimSpace(parts[1])
var endU uint32
if len(endStr) < 4 || !strings.Contains(endStr, ".") {
n, err := strconv.Atoi(endStr)
if err != nil || n > 255 {
return 1
}
endU = (startU & 0xFFFFFF00) | uint32(n)
} else {
endIP := net.ParseIP(endStr)
if endIP == nil {
return 1
}
endU, ok = ipToUint32(endIP)
if !ok {
return 1
}
}
if endU < startU {
return 1
}
return int64(endU-startU) + 1
default:
return 1
}
}
+944
View File
@@ -0,0 +1,944 @@
package parsers
import (
"bufio"
"context"
"errors"
"net"
"os"
"reflect"
"strings"
"testing"
)
func TestHostIteratorCIDRBatch(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/30", "", "")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.2"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorDoesNotExpandWholeRangeAtOnce(t *testing.T) {
iter, err := NewHostIterator("10", "", "")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 3)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"10.0.0.1", "10.0.0.2", "10.0.0.3"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorExcludeCIDR(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/29", "", "192.168.1.2-192.168.1.4")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.5", "192.168.1.6"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorAcceptsMultipleExcludeSources(t *testing.T) {
iter, err := NewHostIterator("192.168.1.0/29", "", "192.168.1.2", "192.168.1.5")
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.3", "192.168.1.4", "192.168.1.6"}
if !reflect.DeepEqual(batch, want) {
t.Fatalf("batch = %#v, want %#v", batch, want)
}
}
func TestHostIteratorReadsLongHostFileLine(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
longPrefix := strings.Repeat("a", 70*1024)
host := longPrefix + ".example.com"
if err := os.WriteFile(path, []byte(host+"\n"), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
iter, err := NewHostIterator("", path)
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 1)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
if !reflect.DeepEqual(batch, []string{host}) {
t.Fatalf("batch = %#v, want long host", batch)
}
}
func TestMultiHostSourceAndMatcherCIDR(t *testing.T) {
src := &multiHostSource{sources: []hostSource{
&singleHostSource{host: "192.168.1.1"},
&singleHostSource{host: "192.168.1.2"},
}}
host, ok, err := src.Next()
if err != nil || !ok || host != "192.168.1.1" {
t.Fatalf("first Next = %q/%v/%v", host, ok, err)
}
host, ok, err = src.Next()
if err != nil || !ok || host != "192.168.1.2" {
t.Fatalf("second Next = %q/%v/%v", host, ok, err)
}
host, ok, err = src.Next()
if err != nil || ok || host != "" {
t.Fatalf("exhausted Next = %q/%v/%v", host, ok, err)
}
if err := src.Close(); err != nil {
t.Fatalf("Close error = %v", err)
}
matcher := newHostMatcher()
if err := matcher.add("192.168.1.0/30,example.com"); err != nil {
t.Fatalf("matcher add error = %v", err)
}
if !matcher.match("192.168.1.1") || !matcher.match("192.168.1.2") || !matcher.match("example.com") {
t.Fatal("matcher should match CIDR hosts and exact host")
}
if matcher.match("192.168.1.3") || matcher.match("nope.example") {
t.Fatal("matcher matched hosts outside its rules")
}
if err := matcher.add("2001:db8::/126"); err == nil {
t.Fatal("IPv6 CIDR should be rejected by IPv4-only matcher")
}
}
func TestCloseHostSourcesIgnoresCloseErrors(t *testing.T) {
first := &closeTrackingSource{err: errors.New("close failed")}
second := &closeTrackingSource{}
closeHostSources([]hostSource{first, second})
if !first.closed || !second.closed {
t.Fatalf("sources closed = %v/%v, want both true", first.closed, second.closed)
}
}
type closeTrackingSource struct {
closed bool
err error
}
func (s *closeTrackingSource) Next() (string, bool, error) {
return "", false, nil
}
func (s *closeTrackingSource) Close() error {
s.closed = true
return s.err
}
// =============================================================================
// newHostSource 分支覆盖
// =============================================================================
// TestNewHostSource_Shortcuts 验证 192/172/10 快捷方式展开为正确 CIDR
func TestNewHostSource_Shortcuts(t *testing.T) {
cases := []struct {
input string
wantFirst string
}{
{"192", "192.168.0.1"},
{"172", "172.16.0.1"},
{"10", "10.0.0.1"},
}
for _, c := range cases {
t.Run(c.input, func(t *testing.T) {
src, err := newHostSource(c.input)
if err != nil {
t.Fatalf("newHostSource(%q) error = %v", c.input, err)
}
defer src.Close()
host, ok, err := src.Next()
if err != nil || !ok {
t.Fatalf("Next() = %q/%v/%v", host, ok, err)
}
if host != c.wantFirst {
t.Errorf("first host = %q, 期望 %q", host, c.wantFirst)
}
})
}
}
// TestNewHostSource_CIDRBranch 验证含 "/" 走 CIDR 分支
func TestNewHostSource_CIDRBranch(t *testing.T) {
src, err := newHostSource("10.0.0.0/30")
if err != nil {
t.Fatalf("newHostSource CIDR error = %v", err)
}
defer src.Close()
host, ok, _ := src.Next()
if !ok || host != "10.0.0.1" {
t.Errorf("CIDR first host = %q, 期望 10.0.0.1", host)
}
}
// TestNewHostSource_InvalidCIDR 无效 CIDR 返回错误
func TestNewHostSource_InvalidCIDR(t *testing.T) {
_, err := newHostSource("999.0.0.0/24")
if err == nil {
t.Error("无效 CIDR 应返回 error")
}
}
// TestNewHostSource_RangeBranch 验证 a-b 格式走 range 分支
func TestNewHostSource_RangeBranch(t *testing.T) {
src, err := newHostSource("192.168.1.5-192.168.1.7")
if err != nil {
t.Fatalf("newHostSource range error = %v", err)
}
defer src.Close()
var got []string
for {
h, ok, err := src.Next()
if err != nil {
t.Fatalf("Next() error = %v", err)
}
if !ok {
break
}
got = append(got, h)
}
want := []string{"192.168.1.5", "192.168.1.6", "192.168.1.7"}
if !reflect.DeepEqual(got, want) {
t.Errorf("range hosts = %v, 期望 %v", got, want)
}
}
// TestNewHostSource_RangeShortTail 验证短尾写法 x.x.x.a-b
func TestNewHostSource_RangeShortTail(t *testing.T) {
src, err := newHostSource("10.0.0.3-5")
if err != nil {
t.Fatalf("newHostSource short-tail range error = %v", err)
}
defer src.Close()
var got []string
for {
h, ok, err := src.Next()
if err != nil {
t.Fatalf("Next() error = %v", err)
}
if !ok {
break
}
got = append(got, h)
}
want := []string{"10.0.0.3", "10.0.0.4", "10.0.0.5"}
if !reflect.DeepEqual(got, want) {
t.Errorf("short-tail range = %v, 期望 %v", got, want)
}
}
// TestNewHostSource_SingleHost 验证普通主机名走 singleHostSource 分支
func TestNewHostSource_SingleHost(t *testing.T) {
src, err := newHostSource("example.com")
if err != nil {
t.Fatalf("newHostSource single error = %v", err)
}
defer src.Close()
host, ok, err := src.Next()
if err != nil || !ok || host != "example.com" {
t.Errorf("single host = %q/%v/%v, 期望 example.com/true/nil", host, ok, err)
}
// 第二次应该耗尽
_, ok, _ = src.Next()
if ok {
t.Error("singleHostSource 第二次 Next 应返回 ok=false")
}
}
// =============================================================================
// hostMatcher.add 分支覆盖
// =============================================================================
// TestHostMatcherAdd_192Shortcut 验证 add("192") 展开为 192.168.0.0/16
func TestHostMatcherAdd_192Shortcut(t *testing.T) {
m := newHostMatcher()
if err := m.add("192"); err != nil {
t.Fatalf("add(192) error = %v", err)
}
if !m.match("192.168.1.100") {
t.Error("192.168.1.100 应命中 192.168.0.0/16")
}
if m.match("10.0.0.1") {
t.Error("10.0.0.1 不应命中")
}
}
// TestHostMatcherAdd_172Shortcut 验证 add("172")
func TestHostMatcherAdd_172Shortcut(t *testing.T) {
m := newHostMatcher()
if err := m.add("172"); err != nil {
t.Fatalf("add(172) error = %v", err)
}
if !m.match("172.16.0.1") {
t.Error("172.16.0.1 应命中 172.16.0.0/12")
}
}
// TestHostMatcherAdd_10Shortcut 验证 add("10")
func TestHostMatcherAdd_10Shortcut(t *testing.T) {
m := newHostMatcher()
if err := m.add("10"); err != nil {
t.Fatalf("add(10) error = %v", err)
}
if !m.match("10.1.2.3") {
t.Error("10.1.2.3 应命中 10.0.0.0/8")
}
}
// TestHostMatcherAdd_CIDR 验证 add 处理 CIDR 字符串
func TestHostMatcherAdd_CIDR(t *testing.T) {
m := newHostMatcher()
if err := m.add("192.168.5.0/24"); err != nil {
t.Fatalf("add CIDR error = %v", err)
}
if !m.match("192.168.5.10") {
t.Error("192.168.5.10 应命中 /24")
}
if m.match("192.168.6.10") {
t.Error("192.168.6.10 不应命中")
}
}
// TestHostMatcherAdd_Range 验证 add 处理 a-b 范围
func TestHostMatcherAdd_Range(t *testing.T) {
m := newHostMatcher()
if err := m.add("10.0.0.10-10.0.0.20"); err != nil {
t.Fatalf("add range error = %v", err)
}
if !m.match("10.0.0.15") {
t.Error("10.0.0.15 应命中范围")
}
if m.match("10.0.0.9") || m.match("10.0.0.21") {
t.Error("边界外不应命中")
}
}
// TestHostMatcherAdd_ExactHost 验证 add 处理普通主机名(exact 分支)
func TestHostMatcherAdd_ExactHost(t *testing.T) {
m := newHostMatcher()
if err := m.add("myhost.local"); err != nil {
t.Fatalf("add exact error = %v", err)
}
if !m.match("myhost.local") {
t.Error("exact 主机名应命中")
}
if m.match("other.local") {
t.Error("其他主机名不应命中")
}
}
// TestHostMatcherAdd_MultipleComma 验证逗号分隔多个值
func TestHostMatcherAdd_MultipleComma(t *testing.T) {
m := newHostMatcher()
if err := m.add("host1.com, host2.com, 192.168.1.0/30"); err != nil {
t.Fatalf("add comma-separated error = %v", err)
}
if !m.match("host1.com") || !m.match("host2.com") || !m.match("192.168.1.1") {
t.Error("逗号分隔的值应全部命中")
}
}
// TestHostMatcherAdd_EmptyEntry 逗号中间空串不报错
func TestHostMatcherAdd_EmptyEntry(t *testing.T) {
m := newHostMatcher()
if err := m.add(",,,"); err != nil {
t.Fatalf("全空逗号不应报错: %v", err)
}
}
// TestHostMatcherAdd_InvalidCIDR 无效 CIDR 返回 error
func TestHostMatcherAdd_InvalidCIDR(t *testing.T) {
m := newHostMatcher()
if err := m.add("999.0.0.0/8"); err == nil {
t.Error("无效 CIDR 应返回 error")
}
}
// =============================================================================
// fileHostSource.Next 分支覆盖
// =============================================================================
// TestFileHostSourceNext_SkipsEmptyAndComments 验证空行和注释行被跳过
func TestFileHostSourceNext_SkipsEmptyAndComments(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
content := "\n# this is a comment\n\n \n10.0.0.1\n# another comment\n10.0.0.2\n"
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
iter, err := NewHostIterator("", path)
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"10.0.0.1", "10.0.0.2"}
if !reflect.DeepEqual(batch, want) {
t.Errorf("batch = %v, 期望 %v", batch, want)
}
}
// TestFileHostSourceNext_MultipleSources 验证文件中每行多个 host(逗号分隔)走 multiHostSource 分支
func TestFileHostSourceNext_MultipleSources(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
// 一行两个 host,触发 multiHostSource 分支
content := "10.0.0.1,10.0.0.2\n10.0.0.3\n"
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
iter, err := NewHostIterator("", path)
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
want := []string{"10.0.0.1", "10.0.0.2", "10.0.0.3"}
if !reflect.DeepEqual(batch, want) {
t.Errorf("batch = %v, 期望 %v", batch, want)
}
}
// TestFileHostSourceNext_InvalidLineSkipped 无效行(解析失败)被跳过不报错
func TestFileHostSourceNext_InvalidLineSkipped(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
// 包含无效 CIDR,应被跳过
content := "999.0.0.0/8\n10.0.0.1\n"
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
iter, err := NewHostIterator("", path)
if err != nil {
t.Fatalf("NewHostIterator error = %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
// 无效行被跳过,只返回有效行
if len(batch) != 1 || batch[0] != "10.0.0.1" {
t.Errorf("batch = %v, 期望 [10.0.0.1]", batch)
}
}
// =============================================================================
// NewHostIterator 错误路径
// =============================================================================
// TestNewHostIterator_InvalidFilename 不存在的文件应返回 error
func TestNewHostIterator_InvalidFilename(t *testing.T) {
_, err := NewHostIterator("", "/nonexistent/path/hosts.txt")
if err == nil {
t.Error("不存在的文件应返回 error")
}
}
// TestNewHostIterator_InvalidHost host 解析失败时应返回 error(并关闭已打开的文件 source)
func TestNewHostIterator_InvalidHost_WithFile(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
if err := os.WriteFile(path, []byte("10.0.0.1\n"), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
// 无效 CIDR 会让 newHostSources 失败
_, err := NewHostIterator("999.0.0.0/8", path)
if err == nil {
t.Error("无效 host 应返回 error")
}
}
// TestNewHostIterator_InvalidExclude exclude 参数无效时应返回 error
func TestNewHostIterator_InvalidExclude(t *testing.T) {
_, err := NewHostIterator("10.0.0.1", "", "999.0.0.0/8")
if err == nil {
t.Error("无效 exclude 应返回 error")
}
}
// TestNewHostIterator_EmptyExcludeSkipped 空白 exclude 条目应被跳过,不报错
func TestNewHostIterator_EmptyExcludeSkipped(t *testing.T) {
iter, err := NewHostIterator("10.0.0.1", "", " ", "")
if err != nil {
t.Fatalf("空白 exclude 不应报错: %v", err)
}
defer iter.Close()
host, ok, err := iter.Next()
if err != nil || !ok || host != "10.0.0.1" {
t.Errorf("Next() = %q/%v/%v", host, ok, err)
}
}
// =============================================================================
// Close 路径
// =============================================================================
// TestClose_Nil nil HostIterator Close 不 panic
func TestClose_Nil(t *testing.T) {
var it *HostIterator
if err := it.Close(); err != nil {
t.Errorf("nil Close 应返回 nil, 得到 %v", err)
}
}
// TestClose_WithCurrent 有 current source 时 Close 应关闭它
func TestClose_WithCurrent(t *testing.T) {
src := &closeTrackingSource{}
it := &HostIterator{current: src}
if err := it.Close(); err != nil {
t.Errorf("Close error = %v", err)
}
if !src.closed {
t.Error("current source 应被关闭")
}
if it.current != nil {
t.Error("Close 后 current 应为 nil")
}
}
// TestClose_SourcesError Close 中 source 返回 error 应被记录
func TestClose_SourcesError(t *testing.T) {
errSrc := &closeTrackingSource{err: errors.New("close error")}
it := &HostIterator{sources: []hostSource{errSrc}}
err := it.Close()
if err == nil {
t.Error("source Close 失败时应返回 error")
}
if !errSrc.closed {
t.Error("出错的 source 也应被调用 Close")
}
}
// TestClose_CurrentErrorThenSources current Close 报错,后续 source Close 成功,返回 current 的 error
func TestClose_CurrentErrorThenSources(t *testing.T) {
currentSrc := &closeTrackingSource{err: errors.New("current close error")}
otherSrc := &closeTrackingSource{}
it := &HostIterator{
current: currentSrc,
sources: []hostSource{otherSrc},
}
err := it.Close()
if err == nil {
t.Error("应返回 current 的 error")
}
if !currentSrc.closed || !otherSrc.closed {
t.Error("两个 source 都应被关闭")
}
}
// =============================================================================
// Next 错误路径
// =============================================================================
// errorSource 让 Next() 返回 error
type errorSource struct {
err error
}
func (s *errorSource) Next() (string, bool, error) { return "", false, s.err }
func (s *errorSource) Close() error { return nil }
// errorOnCloseSource Next 返回 ok=falseClose 返回 error
type errorOnCloseSource struct {
err error
}
func (s *errorOnCloseSource) Next() (string, bool, error) { return "", false, nil }
func (s *errorOnCloseSource) Close() error { return s.err }
// TestNext_SourceNextError source.Next() 返回 error 时 iter.Next 应透传
func TestNext_SourceNextError(t *testing.T) {
it := &HostIterator{
sources: []hostSource{&errorSource{err: errors.New("next error")}},
}
_, _, err := it.Next()
if err == nil {
t.Error("source Next error 应透传")
}
}
// TestNext_SourceCloseError 源耗尽时 Close 报错应透传
func TestNext_SourceCloseError(t *testing.T) {
it := &HostIterator{
sources: []hostSource{&errorOnCloseSource{err: errors.New("close error")}},
}
_, _, err := it.Next()
if err == nil {
t.Error("source 耗尽时 Close error 应透传")
}
}
// =============================================================================
// NextBatch 边界条件
// =============================================================================
// TestNextBatch_ZeroSize size=0 应使用 DefaultHostBatchSize(实际受源数量限制)
func TestNextBatch_ZeroSize(t *testing.T) {
iter, err := NewHostIterator("10.0.0.1", "")
if err != nil {
t.Fatalf("NewHostIterator: %v", err)
}
defer iter.Close()
// size=0 触发默认 DefaultHostBatchSize 分支,源只有一个 host
batch, err := iter.NextBatch(context.Background(), 0)
if err != nil {
t.Fatalf("NextBatch(0) error = %v", err)
}
if len(batch) != 1 || batch[0] != "10.0.0.1" {
t.Errorf("batch = %v, 期望 [10.0.0.1]", batch)
}
}
// TestNextBatch_NegativeSize size<0 也应使用默认值
func TestNextBatch_NegativeSize(t *testing.T) {
iter, err := NewHostIterator("10.0.0.2", "")
if err != nil {
t.Fatalf("NewHostIterator: %v", err)
}
defer iter.Close()
batch, err := iter.NextBatch(context.Background(), -1)
if err != nil {
t.Fatalf("NextBatch(-1) error = %v", err)
}
if len(batch) != 1 || batch[0] != "10.0.0.2" {
t.Errorf("batch = %v, 期望 [10.0.0.2]", batch)
}
}
// TestNextBatch_ContextCancelled context 取消应立即返回
func TestNextBatch_ContextCancelled(t *testing.T) {
iter, err := NewHostIterator("10.0.0.0/8", "")
if err != nil {
t.Fatalf("NewHostIterator: %v", err)
}
defer iter.Close()
ctx, cancel := context.WithCancel(context.Background())
cancel() // 立即取消
_, err = iter.NextBatch(ctx, 100)
if err == nil {
t.Error("已取消的 context 应返回 error")
}
}
// TestNextBatch_DeduplicatesHosts 重复 host 只保留一个
func TestNextBatch_DeduplicatesHosts(t *testing.T) {
// 两个相同的单 host source
it := &HostIterator{
sources: []hostSource{
&singleHostSource{host: "10.0.0.1"},
&singleHostSource{host: "10.0.0.1"},
},
}
batch, err := it.NextBatch(context.Background(), 10)
if err != nil {
t.Fatalf("NextBatch error = %v", err)
}
if len(batch) != 1 || batch[0] != "10.0.0.1" {
t.Errorf("batch = %v, 期望去重为 [10.0.0.1]", batch)
}
}
// TestNextBatch_NextError Next 报错时应透传
func TestNextBatch_NextError(t *testing.T) {
it := &HostIterator{
sources: []hostSource{&errorSource{err: errors.New("iter error")}},
}
_, err := it.NextBatch(context.Background(), 10)
if err == nil {
t.Error("Next error 应透传到 NextBatch")
}
}
// =============================================================================
// newRangeHostSource 错误路径
// =============================================================================
// TestNewRangeHostSource_TooManyDashes 超过一个 "-" 应报错(实际按首个切分:a-b-c 被 Split 成 3 段)
func TestNewRangeHostSource_TooManyDashes(t *testing.T) {
// "a-b-c" Split by "-" 得到 3 段,len != 2,应报错
_, err := newRangeHostSource("10.0.0.1-10.0.0.5-extra")
if err == nil {
t.Error("三段格式应报错")
}
}
// TestNewRangeHostSource_InvalidStartIP 起始 IP 无效
func TestNewRangeHostSource_InvalidStartIP(t *testing.T) {
_, err := newRangeHostSource("notanip-10.0.0.5")
if err == nil {
t.Error("无效起始 IP 应报错")
}
}
// TestNewRangeHostSource_InvalidShortTailNonNumeric 短尾不是数字应报错
func TestNewRangeHostSource_InvalidShortTailNonNumeric(t *testing.T) {
// 尾部 "xyz" 不是数字
_, err := newRangeHostSource("10.0.0.1-xyz")
if err == nil {
t.Error("非数字短尾应报错")
}
}
// TestNewRangeHostSource_InvalidShortTailOver255 短尾超过 255 应报错
func TestNewRangeHostSource_InvalidShortTailOver255(t *testing.T) {
_, err := newRangeHostSource("10.0.0.1-300")
if err == nil {
t.Error("短尾 >255 应报错")
}
}
// TestNewRangeHostSource_StartGTEnd 起始 > 结束应报错
func TestNewRangeHostSource_StartGTEnd(t *testing.T) {
_, err := newRangeHostSource("10.0.0.200-10.0.0.100")
if err == nil {
t.Error("start > end 应报错")
}
}
// TestNewRangeHostSource_InvalidFullEndIP 完整结束 IP 无效(如 "10.0.0.999"
func TestNewRangeHostSource_InvalidFullEndIP(t *testing.T) {
// end IP 包含 "." 但无效
_, err := newRangeHostSource("10.0.0.1-10.0.0.999")
if err == nil {
t.Error("无效结束 IP 应报错")
}
}
// TestNewRangeHostSource_ShortTailStartGTEnd 短尾导致 start > end 应报错
func TestNewRangeHostSource_ShortTailStartGTEnd(t *testing.T) {
_, err := newRangeHostSource("10.0.0.200-100")
if err == nil {
t.Error("短尾结果 start > end 应报错")
}
}
// =============================================================================
// hostMatcher.addRange 错误路径
// =============================================================================
// TestAddRange_InvalidRange addRange 传入无效范围应报错
func TestAddRange_InvalidRange(t *testing.T) {
m := newHostMatcher()
if err := m.addRange("notvalid-range"); err == nil {
t.Error("无效 range 应返回 error")
}
}
// TestAddRange_ValidRange addRange 正常路径
func TestAddRange_ValidRange(t *testing.T) {
m := newHostMatcher()
if err := m.addRange("10.0.0.10-10.0.0.20"); err != nil {
t.Fatalf("addRange error = %v", err)
}
if !m.match("10.0.0.10") || !m.match("10.0.0.20") {
t.Error("addRange 边界值应命中")
}
}
// =============================================================================
// hostMatcher.add 错误路径(shortcut 分支中 addCIDR 失败)
// =============================================================================
// TestHostMatcherAdd_InvalidRange add 的 range 格式无效
func TestHostMatcherAdd_InvalidRange(t *testing.T) {
m := newHostMatcher()
// 构造一个 looksLikeIPRange 通过但 newRangeHostSource 失败的字符串
// "10.0.0.200-10.0.0.100" start>end 会报错
if err := m.add("10.0.0.200-10.0.0.100"); err == nil {
t.Error("无效 range (start>end) 应返回 error")
}
}
// =============================================================================
// newCIDRHostSource IPv6 路径
// =============================================================================
// TestNewCIDRHostSource_IPv6Rejected IPv6 CIDR 应报错
func TestNewCIDRHostSource_IPv6Rejected(t *testing.T) {
_, err := newCIDRHostSource("2001:db8::/32")
if err == nil {
t.Error("IPv6 CIDR 应被拒绝")
}
}
// =============================================================================
// fileHostSource.Close 路径
// =============================================================================
// TestFileHostSource_CloseWithCurrent fileHostSource.Close 时 current != nil 分支
func TestFileHostSource_CloseWithCurrent(t *testing.T) {
dir := t.TempDir()
path := dir + "/hosts.txt"
// 写入一个 CIDR,这样 fileHostSource 会持有 current source
if err := os.WriteFile(path, []byte("10.0.0.0/30\n"), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
src, err := newFileHostSource(path)
if err != nil {
t.Fatalf("newFileHostSource: %v", err)
}
// 触发 current 被设置
_, _, _ = src.Next()
// 此时 current 应非 nilClose 应正常关闭它
if err := src.Close(); err != nil {
t.Errorf("Close with current error = %v", err)
}
}
// TestFileHostSource_CloseNilFile file 已经为 nil 时 Close 直接返回 nil
func TestFileHostSource_CloseNilFile(t *testing.T) {
src := &fileHostSource{file: nil}
if err := src.Close(); err != nil {
t.Errorf("nil file Close error = %v", err)
}
}
// =============================================================================
// multiHostSource.Close 路径
// =============================================================================
// TestMultiHostSource_CloseWithCurrent Close 时 current != nil 分支
func TestMultiHostSource_CloseWithCurrent(t *testing.T) {
inner := &closeTrackingSource{}
ms := &multiHostSource{current: inner}
if err := ms.Close(); err != nil {
t.Errorf("Close error = %v", err)
}
if !inner.closed {
t.Error("current 应被关闭")
}
if ms.current != nil {
t.Error("Close 后 current 应为 nil")
}
}
// =============================================================================
// ipToUint32 IPv6 路径
// =============================================================================
// TestIpToUint32_IPv6ReturnsFalse IPv6 地址应返回 false
func TestIpToUint32_IPv6ReturnsFalse(t *testing.T) {
ip := net.ParseIP("2001:db8::1")
_, ok := ipToUint32(ip)
if ok {
t.Error("IPv6 地址应返回 ok=false")
}
}
// TestIpToUint32_NilReturnsFalse nil IP 应返回 false
func TestIpToUint32_NilReturnsFalse(t *testing.T) {
_, ok := ipToUint32(nil)
if ok {
t.Error("nil IP 应返回 ok=false")
}
}
// =============================================================================
// 剩余未覆盖路径
// =============================================================================
// TestFileHostSource_CurrentNextError fileHostSource.Next 中 current.Next() 报错应透传
func TestFileHostSource_CurrentNextError(t *testing.T) {
src := &fileHostSource{
current: &errorSource{err: errors.New("inner error")},
// scanner 为 nil——不会走到 scanner 分支
scanner: bufio.NewScanner(strings.NewReader("")),
}
_, _, err := src.Next()
if err == nil {
t.Error("current.Next() 报错应透传")
}
}
// TestMultiHostSource_InnerNextError multiHostSource.Next 中内部 source.Next() 报错应透传
func TestMultiHostSource_InnerNextError(t *testing.T) {
ms := &multiHostSource{
sources: []hostSource{&errorSource{err: errors.New("inner error")}},
}
_, _, err := ms.Next()
if err == nil {
t.Error("内部 source.Next() 报错应透传到 multiHostSource.Next")
}
}
// TestNewHostSource_RangeError newHostSource range 分支中 newRangeHostSource 失败
func TestNewHostSource_RangeError(t *testing.T) {
// start > endlooksLikeIPRange 通过(前半部分是有效 IP),但 newRangeHostSource 返回错误
_, err := newHostSource("10.0.0.200-10.0.0.100")
if err == nil {
t.Error("start>end range 应返回 error")
}
}
// TestNewCIDRHostSource_IPv6DirectCall 直接调用 newCIDRHostSource 传入 IPv6 CIDR
func TestNewCIDRHostSource_IPv6DirectCall(t *testing.T) {
// IPv6 CIDR —— bits=128 != 32,触发 line 332-334
_, err := newCIDRHostSource("::1/128")
if err == nil {
t.Error("IPv6 CIDR 应被 newCIDRHostSource 拒绝 (bits!=32)")
}
}
+57 -10
View File
@@ -386,6 +386,15 @@ func TestParsePort_PortGroups(t *testing.T) {
}
}
func TestParsePortGroupsRequireWholeToken(t *testing.T) {
if got := ParsePort("web8080"); len(got) != 0 {
t.Fatalf("ParsePort(web8080) = %v, want empty invalid token", got)
}
if got := ParsePort("web,8080"); len(got) == 0 || got[len(got)-1] != 28018 {
t.Fatalf("ParsePort(web,8080) = %v, want expanded web group", got)
}
}
// TestParsePort_WhitespaceHandling 测试空格处理
func TestParsePort_WhitespaceHandling(t *testing.T) {
tests := []struct {
@@ -590,6 +599,18 @@ func TestParseIP_IPRange(t *testing.T) {
}
}
func TestParseIP_IPRangeNoLimit(t *testing.T) {
result, err := parseIPRangeString("192.168.1.1-5")
if err != nil {
t.Fatalf("parseIPRangeString error = %v", err)
}
expected := []string{"192.168.1.1", "192.168.1.2", "192.168.1.3", "192.168.1.4", "192.168.1.5"}
if !reflect.DeepEqual(result, expected) {
t.Errorf("parseIPRangeString = %v, want %v", result, expected)
}
}
// TestParseIP_FromFile 测试从文件读取
//
// 验证:文件中的IP列表被正确读取
@@ -684,6 +705,18 @@ func TestParseIP_Exclude(t *testing.T) {
}
}
func TestParseIPMultipleExcludeSources(t *testing.T) {
result, err := ParseIP("192.168.1.1-192.168.1.4", "", "192.168.1.2", "192.168.1.4")
if err != nil {
t.Fatalf("ParseIP error = %v", err)
}
expected := []string{"192.168.1.1", "192.168.1.3"}
if !reflect.DeepEqual(result, expected) {
t.Fatalf("ParseIP with multiple excludes = %v, want %v", result, expected)
}
}
// TestParseIP_Deduplicate 测试去重
func TestParseIP_Deduplicate(t *testing.T) {
result, err := ParseIP("192.168.1.1,192.168.1.1,192.168.1.2,192.168.1.2", "", "")
@@ -788,7 +821,9 @@ func TestParsePortRange(t *testing.T) {
// TestExcludeHosts 测试排除主机
func TestExcludeHosts(t *testing.T) {
hosts := []string{"host1", "host2", "host3", "host4"}
exclude := []string{"host2", "host4"}
exclude := newHostMatcher()
exclude.exact["host2"] = struct{}{}
exclude.exact["host4"] = struct{}{}
result := excludeFromList(hosts, exclude)
expected := []string{"host1", "host3"}
@@ -803,7 +838,7 @@ func TestExcludeHosts(t *testing.T) {
// TestExcludeHosts_EmptyExclude 测试空排除列表
func TestExcludeHosts_EmptyExclude(t *testing.T) {
hosts := []string{"host1", "host2"}
result := excludeFromList(hosts, []string{})
result := excludeFromList(hosts, nil)
if !reflect.DeepEqual(result, hosts) {
t.Errorf("excludeFromList(空排除列表) 应该返回原列表")
@@ -878,12 +913,6 @@ func TestParseIP_InternalNetworkShortcuts(t *testing.T) {
100, // 172.16.0.0/12 应该很多
"172.",
},
{
"10简写",
"10",
100, // 10.0.0.0/8 应该很多
"10.",
},
}
for _, tt := range tests {
@@ -969,6 +998,24 @@ func TestParseIP_FullIPRange(t *testing.T) {
}
}
func TestParseIP_FullIPRangeComplete(t *testing.T) {
result, err := parseIPRangeString("192.168.1.1-192.168.1.5")
if err != nil {
t.Fatalf("parseIPRangeString error = %v", err)
}
expected := []string{
"192.168.1.1",
"192.168.1.2",
"192.168.1.3",
"192.168.1.4",
"192.168.1.5",
}
if !reflect.DeepEqual(result, expected) {
t.Errorf("parseIPRangeString no limit = %v, want %v", result, expected)
}
}
// TestParseIP_InvalidCIDR 测试无效CIDR
func TestParseIP_InvalidCIDR(t *testing.T) {
tests := []struct {
@@ -977,7 +1024,7 @@ func TestParseIP_InvalidCIDR(t *testing.T) {
expectErr bool
}{
{"无效掩码/33", "192.168.1.0/33", true},
{"效掩码/0", "192.168.1.0/0", false}, // /0 技术上是有效的
{"效掩码/32", "192.168.1.1/32", false},
{"格式错误", "192.168.1.0/abc", true},
{"缺少掩码", "192.168.1.0/", true},
}
@@ -1173,7 +1220,7 @@ test:
{"admin", "password123"},
{"root", "toor"},
{"user", "pass:with:colons"}, // 密码可以包含冒号
{"test", ""}, // 空密码
{"test", ""}, // 空密码
}
if len(result) != len(tests) {
+49 -49
View File
@@ -11,6 +11,7 @@ import (
"strings"
"github.com/shadow1ng/fscan/common/config"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -37,7 +38,7 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
if filename != "" {
fileHosts, err := ReadLinesFromFile(filename)
if err != nil {
return nil, fmt.Errorf("读取主机文件失败: %w", err)
return nil, fmt.Errorf(i18n.GetText("parser_read_hosts_failed")+": %w", err)
}
for _, h := range fileHosts {
parsed, err := parseHostString(h)
@@ -52,18 +53,27 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
if host != "" {
hostList, err := parseHostString(host)
if err != nil {
return nil, fmt.Errorf("解析主机失败: %w", err)
return nil, fmt.Errorf(i18n.GetText("parser_parse_host_failed")+": %w", err)
}
hosts = append(hosts, hostList...)
}
// 处理排除主机
if len(nohosts) > 0 && nohosts[0] != "" {
excludeList, err := parseHostString(nohosts[0])
if err != nil {
return nil, fmt.Errorf("解析排除主机失败: %w", err)
if len(nohosts) > 0 {
matcher := newHostMatcher()
hasExclude := false
for _, exclude := range nohosts {
if strings.TrimSpace(exclude) == "" {
continue
}
hasExclude = true
if err := matcher.add(exclude); err != nil {
return nil, fmt.Errorf(i18n.GetText("parser_parse_exclude_failed")+": %w", err)
}
}
if hasExclude {
hosts = excludeFromList(hosts, matcher)
}
hosts = excludeFromList(hosts, excludeList)
}
// 去重和排序
@@ -71,7 +81,7 @@ func ParseIP(host string, filename string, nohosts ...string) ([]string, error)
sort.Strings(hosts)
if len(hosts) == 0 {
return nil, fmt.Errorf("没有找到有效的主机")
return nil, fmt.Errorf("%s", i18n.GetText("parser_no_valid_hosts"))
}
return hosts, nil
@@ -89,33 +99,33 @@ func parseHostString(host string) ([]string, error) {
switch {
case h == "192":
cidrHosts, err := parseIPCIDR("192.168.0.0/16", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("192.168.0.0/16")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case h == "172":
cidrHosts, err := parseIPCIDR("172.16.0.0/12", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("172.16.0.0/12")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case h == "10":
cidrHosts, err := parseIPCIDR("10.0.0.0/8", SimpleMaxHosts)
cidrHosts, err := parseIPCIDR("10.0.0.0/8")
if err != nil {
return nil, err
}
hosts = append(hosts, cidrHosts...)
case strings.Contains(h, "/"):
cidrHosts, err := parseIPCIDR(h, SimpleMaxHosts)
cidrHosts, err := parseIPCIDR(h)
if err != nil {
return nil, fmt.Errorf("CIDR解析失败 %s: %w", h, err)
return nil, fmt.Errorf(i18n.Tr("parser_cidr_failed", h)+": %w", err)
}
hosts = append(hosts, cidrHosts...)
case strings.Contains(h, "-") && !strings.Contains(h, ":") && looksLikeIPRange(h):
rangeHosts, err := parseIPRangeString(h, SimpleMaxHosts)
rangeHosts, err := parseIPRangeString(h)
if err != nil {
return nil, fmt.Errorf("IP范围解析失败 %s: %w", h, err)
return nil, fmt.Errorf(i18n.Tr("parser_ip_range_failed", h)+": %w", err)
}
hosts = append(hosts, rangeHosts...)
default:
@@ -190,11 +200,14 @@ func parsePortRange(rangeStr string) []int {
// expandPortGroups 展开端口组
func expandPortGroups(ports string) string {
portGroups := config.GetPortGroups()
result := ports
for group, portList := range portGroups {
result = strings.ReplaceAll(result, group, portList)
parts := strings.Split(ports, ",")
for i, part := range parts {
token := strings.TrimSpace(part)
if portList, ok := portGroups[token]; ok {
parts[i] = portList
}
}
return result
return strings.Join(parts, ",")
}
// =============================================================================
@@ -211,6 +224,7 @@ func ReadLinesFromFile(filename string) ([]string, error) {
var lines []string
scanner := bufio.NewScanner(file)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line != "" && !strings.HasPrefix(line, "#") {
@@ -288,7 +302,7 @@ func ParseHashFile(filename string) ([]string, [][]byte, error) {
// =============================================================================
// parseIPCIDR 解析CIDR网段
func parseIPCIDR(cidr string, maxTargets int) ([]string, error) {
func parseIPCIDR(cidr string) ([]string, error) {
_, ipNet, err := net.ParseCIDR(cidr)
if err != nil {
return nil, err
@@ -298,17 +312,11 @@ func parseIPCIDR(cidr string, maxTargets int) ([]string, error) {
ip := make(net.IP, len(ipNet.IP))
copy(ip, ipNet.IP)
count := 0
for ipNet.Contains(ip) {
ips = append(ips, ip.String())
count++
if count >= maxTargets {
break
}
incrementIP(ip)
}
// 移除网络地址和广播地址
if len(ips) > 2 {
ips = ips[1 : len(ips)-1]
}
@@ -330,10 +338,10 @@ func looksLikeIPRange(s string) bool {
}
// parseIPRangeString 解析IP范围字符串
func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
func parseIPRangeString(rangeStr string) ([]string, error) {
parts := strings.Split(rangeStr, "-")
if len(parts) != 2 {
return nil, fmt.Errorf("无效的IP范围格式: %s", rangeStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_fmt", rangeStr))
}
startIPStr := strings.TrimSpace(parts[0])
@@ -341,7 +349,7 @@ func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
startIP := net.ParseIP(startIPStr)
if startIP == nil {
return nil, fmt.Errorf("无效的起始IP地址: %s", startIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_start_ip", startIPStr))
}
// 处理简写格式 (如: 192.168.1.1-100)
@@ -352,28 +360,28 @@ func parseIPRangeString(rangeStr string, maxTargets int) ([]string, error) {
// 处理完整格式 (如: 192.168.1.1-192.168.1.100)
endIP := net.ParseIP(endIPStr)
if endIP == nil {
return nil, fmt.Errorf("无效的结束IP地址: %s", endIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_end_ip", endIPStr))
}
return parseIPFullRange(startIP, endIP, maxTargets)
return parseIPFullRange(startIP, endIP)
}
// parseIPShortRange 解析短格式IP范围
func parseIPShortRange(startIPStr, endSuffix string) ([]string, error) {
endNum, err := strconv.Atoi(endSuffix)
if err != nil || endNum > 255 {
return nil, fmt.Errorf("无效的IP范围结束值: %s", endSuffix)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_end_val", endSuffix))
}
ipParts := strings.Split(startIPStr, ".")
if len(ipParts) != 4 {
return nil, fmt.Errorf("无效的IP地址格式: %s", startIPStr)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_fmt", startIPStr))
}
prefixIP := strings.Join(ipParts[0:3], ".")
startNum, err := strconv.Atoi(ipParts[3])
if err != nil || startNum > endNum {
return nil, fmt.Errorf("无效的IP范围: %s-%s", startIPStr, endSuffix)
return nil, fmt.Errorf("%s", i18n.Tr("parser_invalid_ip_range_val", startIPStr, endSuffix))
}
var allIP []string
@@ -385,30 +393,27 @@ func parseIPShortRange(startIPStr, endSuffix string) ([]string, error) {
}
// parseIPFullRange 解析完整格式的IP范围
func parseIPFullRange(startIP, endIP net.IP, maxTargets int) ([]string, error) {
func parseIPFullRange(startIP, endIP net.IP) ([]string, error) {
start4 := startIP.To4()
end4 := endIP.To4()
if start4 == nil || end4 == nil {
return nil, fmt.Errorf("仅支持IPv4地址范围")
return nil, fmt.Errorf("%s", i18n.GetText("parser_ipv4_only"))
}
startInt := (int(start4[0]) << 24) | (int(start4[1]) << 16) | (int(start4[2]) << 8) | int(start4[3])
endInt := (int(end4[0]) << 24) | (int(end4[1]) << 16) | (int(end4[2]) << 8) | int(end4[3])
if startInt > endInt {
return nil, fmt.Errorf("起始IP大于结束IP")
return nil, fmt.Errorf("%s", i18n.GetText("parser_start_gt_end"))
}
var ips []string
current := make(net.IP, len(start4))
copy(current, start4)
count := 0
for {
ips = append(ips, current.String())
count++
if current.Equal(end4) || count >= maxTargets {
if current.Equal(end4) {
break
}
incrementIP(current)
@@ -428,19 +433,14 @@ func incrementIP(ip net.IP) {
}
// excludeFromList 从列表中排除指定项
func excludeFromList(hosts, excludeList []string) []string {
if len(excludeList) == 0 {
func excludeFromList(hosts []string, matcher *hostMatcher) []string {
if matcher == nil {
return hosts
}
excludeMap := make(map[string]struct{}, len(excludeList))
for _, e := range excludeList {
excludeMap[e] = struct{}{}
}
result := make([]string, 0, len(hosts))
for _, h := range hosts {
if _, found := excludeMap[h]; !found {
if !matcher.match(h) {
result = append(result, h)
}
}
+78 -56
View File
@@ -28,11 +28,11 @@ ProgressManager.go - 固定底部进度条管理器
type ProgressManager struct {
mu sync.RWMutex
enabled bool
total int64
current int64
total atomic.Int64
current atomic.Int64
description string
startTime time.Time
isActive bool
isActive atomic.Bool
terminalHeight int
reservedLines int // 为进度条保留的行数
lastContentLine int // 最后一行内容的位置
@@ -48,6 +48,10 @@ type ProgressManager struct {
// 进度条更新控制(减少 Windows 终端的重复输出)
lastRenderedPercent int
// 引用,避免读全局
state *State
noColor bool
}
// =============================================================================
@@ -102,20 +106,22 @@ func GetProgressManager() *ProgressManager {
// InitProgress 初始化进度条
func (pm *ProgressManager) InitProgress(total int64, description string) {
fv := GetFlagVars()
if fv.DisableProgress || fv.Silent {
cfg := GetGlobalConfig()
if cfg.Output.DisableProgress || cfg.Output.Silent || cfg.Output.NoColor {
pm.enabled = false
return
}
pm.state = GetGlobalState()
pm.noColor = cfg.Output.NoColor
pm.mu.Lock()
defer pm.mu.Unlock()
pm.total = total
pm.current = 0
pm.total.Store(total)
pm.current.Store(0)
pm.description = description
pm.startTime = time.Now()
pm.isActive = true
pm.isActive.Store(true)
pm.enabled = true
pm.lastActivity = time.Now()
pm.spinnerIndex = 0
@@ -133,20 +139,28 @@ func (pm *ProgressManager) InitProgress(total int64, description string) {
// UpdateProgress 更新进度
func (pm *ProgressManager) UpdateProgress(increment int64) {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
// 原子累加,避免高并发下的锁竞争
newCurrent := pm.current.Add(increment)
if newCurrent > pm.total.Load() {
pm.current.Store(pm.total.Load())
}
// 节流渲染:距上次渲染不足 50ms 则跳过
now := time.Now()
pm.mu.RLock()
lastAct := pm.lastActivity
pm.mu.RUnlock()
if now.Sub(lastAct) < 50*time.Millisecond {
return
}
pm.mu.Lock()
defer pm.mu.Unlock()
pm.current += increment
if pm.current > pm.total {
pm.current = pm.total
}
// 更新活跃时间
pm.lastActivity = time.Now()
pm.lastActivity = now
pm.mu.Unlock()
pm.renderProgress()
}
@@ -157,14 +171,14 @@ func (pm *ProgressManager) UpdateProgress(increment int64) {
// FinishProgress 完成进度条
func (pm *ProgressManager) FinishProgress() {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
pm.mu.Lock()
defer pm.mu.Unlock()
pm.current = pm.total
pm.current.Store(pm.total.Load())
pm.renderProgress()
// 停止活跃指示器
@@ -175,7 +189,7 @@ func (pm *ProgressManager) FinishProgress() {
// 清理进度条区域,恢复正常输出
pm.clearProgressArea()
pm.isActive = false
pm.isActive.Store(false)
}
// setupProgressSpace 设置进度条空间
@@ -205,20 +219,21 @@ func (pm *ProgressManager) generateProgressBar() string {
// 获取发包统计
packetInfo := pm.getPacketInfo()
if pm.total == 0 {
if pm.total.Load() == 0 {
spinner := pm.getActivityIndicator()
base := fmt.Sprintf("%s %s 等待中...", pm.description, spinner)
base := fmt.Sprintf("%s %s %s", pm.description, spinner, i18n.GetText("progress_waiting"))
if packetInfo != "" {
return base + " " + packetInfo
}
return base
}
percentage := float64(pm.current) / float64(pm.total) * 100
percentage := float64(pm.current.Load()) / float64(pm.total.Load()) * 100
elapsed := time.Since(pm.startTime)
current := pm.current.Load()
// 计算速度
speed := float64(pm.current) / elapsed.Seconds()
speed := float64(current) / elapsed.Seconds()
speedStr := ""
if speed > 0 {
speedStr = fmt.Sprintf(" %.0f/s", speed)
@@ -226,8 +241,8 @@ func (pm *ProgressManager) generateProgressBar() string {
// 计算预估剩余时间
var eta string
if pm.current > 0 && pm.current < pm.total {
totalTime := elapsed * time.Duration(pm.total) / time.Duration(pm.current)
if current > 0 && current < pm.total.Load() {
totalTime := elapsed * time.Duration(pm.total.Load()) / time.Duration(current)
remaining := totalTime - elapsed
if remaining > 0 {
eta = fmt.Sprintf(" ETA:%s", formatDuration(remaining))
@@ -239,7 +254,7 @@ func (pm *ProgressManager) generateProgressBar() string {
// 计算固定部分的宽度
fixedPart := fmt.Sprintf("%s %s %5.1f%% [] (%d/%d)%s%s %s",
pm.description, spinner, percentage, pm.current, pm.total, speedStr, eta, packetInfo)
pm.description, spinner, percentage, current, pm.total.Load(), speedStr, eta, packetInfo)
fixedWidth := displayWidth(fixedPart)
// 计算进度条槽位可用宽度(预留2字符余量)
@@ -266,7 +281,7 @@ func (pm *ProgressManager) generateProgressBar() string {
// 构建最终进度条
result := fmt.Sprintf("%s %s %5.1f%% %s (%d/%d)%s%s",
pm.description, spinner, percentage, bar, pm.current, pm.total, speedStr, eta)
pm.description, spinner, percentage, bar, current, pm.total.Load(), speedStr, eta)
if packetInfo != "" {
result += " " + packetInfo
@@ -277,13 +292,16 @@ func (pm *ProgressManager) generateProgressBar() string {
// getPacketInfo 获取发包统计信息(简化版)
func (pm *ProgressManager) getPacketInfo() string {
packetCount := GetGlobalState().GetPacketCount()
if pm.state == nil {
return ""
}
packetCount := pm.state.GetPacketCount()
if packetCount == 0 {
return ""
}
tcpSuccess := GetGlobalState().GetTCPSuccessPacketCount()
tcpFailed := GetGlobalState().GetTCPFailedPacketCount()
tcpSuccess := pm.state.GetTCPSuccessPacketCount()
tcpFailed := pm.state.GetTCPFailedPacketCount()
// 简化格式:TCP:成功/失败
if tcpSuccess > 0 || tcpFailed > 0 {
@@ -301,13 +319,16 @@ func (pm *ProgressManager) showCompletionInfo() {
fmt.Print("\n")
completionMsg := i18n.GetText("progress_scan_completed")
if GetFlagVars().NoColor {
fmt.Printf("[完成] %s %d/%d (耗时: %s)\n",
completionMsg, pm.total, pm.total, formatDuration(elapsed))
doneMsg := i18n.GetText("progress_done")
durationMsg := i18n.GetText("progress_duration")
total := pm.total.Load()
if pm.noColor {
fmt.Printf("[%s] %s: %d/%d (%s: %s)\n",
doneMsg, completionMsg, total, total, durationMsg, formatDuration(elapsed))
} else {
fmt.Printf("%s[完成] %s %d/%d%s %s(耗时: %s)%s\n",
AnsiGreen, completionMsg, pm.total, pm.total, AnsiReset,
AnsiGray, formatDuration(elapsed), AnsiReset)
fmt.Printf("%s[%s] %s: %d/%d%s %s(%s: %s)%s\n",
AnsiGreen, doneMsg, completionMsg, total, total, AnsiReset,
AnsiGray, durationMsg, formatDuration(elapsed), AnsiReset)
}
}
@@ -321,7 +342,7 @@ func (pm *ProgressManager) clearProgressArea() {
func (pm *ProgressManager) IsActive() bool {
pm.mu.RLock()
defer pm.mu.RUnlock()
return pm.isActive && pm.enabled
return pm.isActive.Load() && pm.enabled
}
// getTerminalHeight 获取终端高度
@@ -458,10 +479,10 @@ func (pm *ProgressManager) GetPercent() float64 {
pm.mu.RLock()
defer pm.mu.RUnlock()
if !pm.isActive || pm.total == 0 {
if !pm.isActive.Load() || pm.total.Load() == 0 {
return 0
}
return float64(pm.current) / float64(pm.total) * 100
return float64(pm.current.Load()) / float64(pm.total.Load()) * 100
}
// =============================================================================
@@ -470,6 +491,10 @@ func (pm *ProgressManager) GetPercent() float64 {
// LogWithProgress 在进度条活跃时协调日志输出
func LogWithProgress(message string) {
if cfg := GetGlobalConfig(); cfg != nil && cfg.Output.Silent {
return
}
pm := GetProgressManager()
if !pm.IsActive() {
// 如果进度条不活跃,直接输出
@@ -492,14 +517,14 @@ func LogWithProgress(message string) {
// renderProgressUnsafe 不加锁的进度条渲染(内部使用)
func (pm *ProgressManager) renderProgressUnsafe() {
if !pm.enabled || !pm.isActive {
if !pm.enabled || !pm.isActive.Load() {
return
}
// 计算当前百分比(避免除零)
currentPercent := 0
if pm.total > 0 {
currentPercent = int((pm.current * 100) / pm.total)
if pm.total.Load() > 0 {
currentPercent = int((pm.current.Load() * 100) / pm.total.Load())
}
// 只在百分比变化时更新,减少不必要的渲染
@@ -532,7 +557,7 @@ func (pm *ProgressManager) renderProgressUnsafe() {
fmt.Print(clearStr)
// 输出进度条(带颜色,如果启用)
if GetFlagVars().NoColor {
if pm.noColor {
fmt.Print(progressBar)
} else {
fmt.Printf("%s%s%s", AnsiCyan, progressBar, AnsiReset)
@@ -561,7 +586,7 @@ func (pm *ProgressManager) startActivityIndicator() {
select {
case <-pm.activityTicker.C:
// 只有在活跃状态下才更新指示器
if pm.isActive && pm.enabled {
if pm.isActive.Load() && pm.enabled {
pm.mu.Lock()
pm.spinnerIndex = (pm.spinnerIndex + 1) % len(spinnerChars)
pm.mu.Unlock()
@@ -618,8 +643,8 @@ ConcurrencyMonitor - 并发监控器
// ConcurrencyMonitor 并发监控器
type ConcurrencyMonitor struct {
// 主扫描器层级
activePluginTasks int64 // 当前活跃的插件任务数
totalPluginTasks int64 // 总插件任务数
activePluginTasks atomic.Int64 // 当前活跃的插件任务数
totalPluginTasks atomic.Int64 // 总插件任务数
// 插件内连接层级已移除 - 原代码为死代码,无任何调用者
}
@@ -634,10 +659,7 @@ var (
// GetConcurrencyMonitor 获取全局并发监控器
func GetConcurrencyMonitor() *ConcurrencyMonitor {
concurrencyMutex.Do(func() {
globalConcurrencyMonitor = &ConcurrencyMonitor{
activePluginTasks: 0,
totalPluginTasks: 0,
}
globalConcurrencyMonitor = &ConcurrencyMonitor{}
})
return globalConcurrencyMonitor
}
@@ -648,18 +670,18 @@ func GetConcurrencyMonitor() *ConcurrencyMonitor {
// StartPluginTask 开始插件任务
func (m *ConcurrencyMonitor) StartPluginTask() {
atomic.AddInt64(&m.activePluginTasks, 1)
atomic.AddInt64(&m.totalPluginTasks, 1)
m.activePluginTasks.Add(1)
m.totalPluginTasks.Add(1)
}
// FinishPluginTask 完成插件任务
func (m *ConcurrencyMonitor) FinishPluginTask() {
atomic.AddInt64(&m.activePluginTasks, -1)
m.activePluginTasks.Add(-1)
}
// GetPluginTaskStats 获取插件任务统计
func (m *ConcurrencyMonitor) GetPluginTaskStats() (active int64, total int64) {
return atomic.LoadInt64(&m.activePluginTasks), atomic.LoadInt64(&m.totalPluginTasks)
return m.activePluginTasks.Load(), m.totalPluginTasks.Load()
}
// =============================================================================
+102
View File
@@ -0,0 +1,102 @@
package common
import (
"strings"
"testing"
"time"
)
func TestProgressTextHelpers(t *testing.T) {
tests := []struct {
name string
in string
want int
}{
{name: "ascii", in: "abc", want: 3},
{name: "cjk", in: "中文", want: 4},
{name: "mixed", in: "a中", want: 3},
{name: "symbol", in: "★", want: 2},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := displayWidth(tt.in); got != tt.want {
t.Fatalf("displayWidth(%q) = %d, want %d", tt.in, got, tt.want)
}
})
}
truncateTests := []struct {
name string
in string
width int
want string
}{
{name: "exact mixed width", in: "abc中文", width: 5, want: "abc中"},
{name: "wide char does not fit", in: "中文", width: 1, want: ""},
{name: "zero width", in: "abc", width: 0, want: ""},
{name: "negative width", in: "abc", width: -1, want: ""},
}
for _, tt := range truncateTests {
t.Run(tt.name, func(t *testing.T) {
if got := truncateToWidth(tt.in, tt.width); got != tt.want {
t.Fatalf("truncateToWidth(%q, %d) = %q, want %q", tt.in, tt.width, got, tt.want)
}
})
}
if got := stripAnsiCodes("\033[31mred\033[0m plain"); got != "red plain" {
t.Fatalf("stripAnsiCodes removed ANSI = %q, want %q", got, "red plain")
}
if got := stripAnsiCodes("plain"); got != "plain" {
t.Fatalf("stripAnsiCodes plain = %q, want plain", got)
}
}
func TestFormatDuration(t *testing.T) {
tests := []struct {
name string
in time.Duration
want string
}{
{name: "seconds", in: 1500 * time.Millisecond, want: "1.5s"},
{name: "minutes", in: 90 * time.Second, want: "1.5m"},
{name: "hours", in: 150 * time.Minute, want: "2.5h"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := formatDuration(tt.in); got != tt.want {
t.Fatalf("formatDuration(%s) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
func TestConcurrencyMonitorTaskStats(t *testing.T) {
monitor := &ConcurrencyMonitor{}
if status := monitor.GetConcurrencyStatus(); status != "" {
t.Fatalf("initial status = %q, want empty", status)
}
monitor.StartPluginTask()
monitor.StartPluginTask()
active, total := monitor.GetPluginTaskStats()
if active != 2 || total != 2 {
t.Fatalf("stats after start = active %d total %d, want 2/2", active, total)
}
if status := monitor.GetConcurrencyStatus(); !strings.HasSuffix(status, ":2") {
t.Fatalf("status after start = %q, want suffix :2", status)
}
monitor.FinishPluginTask()
active, total = monitor.GetPluginTaskStats()
if active != 1 || total != 2 {
t.Fatalf("stats after one finish = active %d total %d, want 1/2", active, total)
}
monitor.FinishPluginTask()
if status := monitor.GetConcurrencyStatus(); status != "" {
t.Fatalf("status after all finish = %q, want empty", status)
}
}
+18 -16
View File
@@ -2,6 +2,8 @@ package proxy
import (
"time"
"github.com/shadow1ng/fscan/common/i18n"
)
/*
@@ -151,41 +153,41 @@ const (
// 错误消息常量
// =============================================================================
const (
var (
// ErrMsgUnsupportedProxyType Manager错误消息 - 不支持的代理类型
ErrMsgUnsupportedProxyType = "不支持的代理类型"
ErrMsgUnsupportedProxyType = i18n.GetText("proxy_unsupported_type")
// ErrMsgEmptyConfig 配置不能为空
ErrMsgEmptyConfig = "配置不能为空"
ErrMsgEmptyConfig = i18n.GetText("proxy_empty_config")
// ErrMsgSOCKS5ParseFailed SOCKS5错误消息 - 地址解析失败
ErrMsgSOCKS5ParseFailed = "SOCKS5代理地址解析失败"
ErrMsgSOCKS5ParseFailed = i18n.GetText("proxy_socks5_parse_failed")
// ErrMsgSOCKS5CreateFailed 拨号器创建失败
ErrMsgSOCKS5CreateFailed = "SOCKS5拨号器创建失败"
ErrMsgSOCKS5CreateFailed = i18n.GetText("proxy_socks5_create_failed")
// ErrMsgSOCKS5ConnTimeout 连接超时
ErrMsgSOCKS5ConnTimeout = "SOCKS5连接超时"
ErrMsgSOCKS5ConnTimeout = i18n.GetText("proxy_socks5_conn_timeout")
// ErrMsgSOCKS5ConnFailed 连接失败
ErrMsgSOCKS5ConnFailed = "SOCKS5连接失败"
ErrMsgSOCKS5ConnFailed = i18n.GetText("proxy_socks5_conn_failed")
// ErrMsgDirectConnFailed 直连错误消息 - 直连失败
ErrMsgDirectConnFailed = "直连失败"
ErrMsgDirectConnFailed = i18n.GetText("proxy_direct_conn_failed")
// ErrMsgHTTPConnFailed HTTP代理错误消息 - 连接失败
ErrMsgHTTPConnFailed = "连接HTTP代理服务器失败"
ErrMsgHTTPConnFailed = i18n.GetText("proxy_http_conn_failed")
// ErrMsgHTTPSetWriteTimeout 设置写超时失败
ErrMsgHTTPSetWriteTimeout = "设置写超时失败"
ErrMsgHTTPSetWriteTimeout = i18n.GetText("proxy_http_set_write_timeout")
// ErrMsgHTTPSendConnectFail 发送CONNECT请求失败
ErrMsgHTTPSendConnectFail = "发送CONNECT请求失败"
ErrMsgHTTPSendConnectFail = i18n.GetText("proxy_http_send_connect_failed")
// ErrMsgHTTPSetReadTimeout 设置读超时失败
ErrMsgHTTPSetReadTimeout = "设置读超时失败"
ErrMsgHTTPSetReadTimeout = i18n.GetText("proxy_http_set_read_timeout")
// ErrMsgHTTPReadRespFailed 读取响应失败
ErrMsgHTTPReadRespFailed = "读取HTTP响应失败"
ErrMsgHTTPReadRespFailed = i18n.GetText("proxy_http_read_response_failed")
// ErrMsgHTTPProxyAuthFailed 代理认证失败
ErrMsgHTTPProxyAuthFailed = "HTTP代理连接失败,状态码: %d"
ErrMsgHTTPProxyAuthFailed = i18n.GetText("proxy_http_status_failed")
// ErrMsgTLSTCPConnFailed TLS错误消息 - TCP连接失败
ErrMsgTLSTCPConnFailed = "建立TCP连接失败"
ErrMsgTLSTCPConnFailed = i18n.GetText("proxy_tls_tcp_conn_failed")
// ErrMsgTLSHandshakeFailed TLS握手失败
ErrMsgTLSHandshakeFailed = "TLS握手失败"
ErrMsgTLSHandshakeFailed = i18n.GetText("proxy_tls_handshake_failed")
)
// =============================================================================
+9
View File
@@ -19,6 +19,9 @@ var (
// proxyProbed 标记代理是否已经探测过(避免重复探测)
proxyProbed atomic.Bool
// currentProxyType 当前代理类型
currentProxyType atomic.Int32
)
// SetProxyEnabled 设置代理启用状态
@@ -61,6 +64,11 @@ func IsProxyProbed() bool {
return proxyProbed.Load()
}
// IsSOCKS5Proxy 检查当前代理是否为SOCKS5类型
func IsSOCKS5Proxy() bool {
return proxyEnabled.Load() && ProxyType(currentProxyType.Load()) == ProxyTypeSOCKS5
}
// AutoConfigureProxy 自动配置代理相关行为
// 根据代理类型和状态自动调整扫描策略
func AutoConfigureProxy(config *ProxyConfig) {
@@ -74,6 +82,7 @@ func AutoConfigureProxy(config *ProxyConfig) {
// 启用代理标记
SetProxyEnabled(true)
currentProxyType.Store(int32(config.Type))
// SOCKS5代理默认假设非标准(后续由探测函数验证)
if config.Type == ProxyTypeSOCKS5 {
+13 -9
View File
@@ -7,7 +7,7 @@ import (
"fmt"
"net"
"net/http"
"sync/atomic"
"strings"
"time"
)
@@ -24,27 +24,27 @@ func (h *httpDialer) Dial(network, address string) (net.Conn, error) {
func (h *httpDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&h.stats.TotalConnections, 1)
h.stats.addTotal(1)
// 连接到HTTP代理服务器
proxyConn, err := h.baseDial.DialContext(ctx, NetworkTCP, h.config.Address)
if err != nil {
atomic.AddInt64(&h.stats.FailedConnections, 1)
h.stats.LastError = err.Error()
h.stats.addFailed(1)
h.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgHTTPConnFailed, ErrCodeHTTPConnFailed, err)
}
// 发送CONNECT请求
if err := h.sendConnectRequest(proxyConn, address); err != nil {
_ = proxyConn.Close() // 错误处理路径,Close错误可忽略
atomic.AddInt64(&h.stats.FailedConnections, 1)
h.stats.LastError = err.Error()
h.stats.addFailed(1)
h.stats.setLastError(err.Error())
return nil, err
}
duration := time.Since(start)
h.stats.LastConnectTime = start
atomic.AddInt64(&h.stats.ActiveConnections, 1)
h.stats.setLastConnectTime(start)
h.stats.addActive(1)
h.updateAverageConnectTime(duration)
return &trackedConn{
@@ -55,6 +55,9 @@ func (h *httpDialer) DialContext(ctx context.Context, network, address string) (
// sendConnectRequest 发送HTTP CONNECT请求
func (h *httpDialer) sendConnectRequest(conn net.Conn, address string) error {
if strings.ContainsAny(address, "\r\n") {
return NewProxyError(ErrTypeProtocol, "invalid CONNECT target", ErrCodeHTTPReadRespFailed, nil)
}
// 构建CONNECT请求
req := fmt.Sprintf(HTTPConnectRequestFormat, address, address)
@@ -108,7 +111,8 @@ func (h *httpDialer) sendConnectRequest(conn net.Conn, address string) error {
// updateAverageConnectTime 更新平均连接时间
func (h *httpDialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
h.stats.mu.Lock()
defer h.stats.mu.Unlock()
if h.stats.AverageConnectTime == 0 {
h.stats.AverageConnectTime = duration
} else {
+23
View File
@@ -0,0 +1,23 @@
package proxy
import (
"net"
"testing"
"time"
)
func TestHTTPDialerRejectsConnectTargetWithLineBreak(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
dialer := &httpDialer{
config: &ProxyConfig{Timeout: time.Second},
stats: &ProxyStats{},
}
err := dialer.sendConnectRequest(client, "example.com:80\r\nX-Injected: yes")
if err == nil {
t.Fatal("sendConnectRequest() error = nil, want invalid target error")
}
}
+26 -37
View File
@@ -4,9 +4,7 @@ import (
"context"
"fmt"
"net"
"net/url"
"sync"
"sync/atomic"
"time"
"golang.org/x/net/proxy"
@@ -128,9 +126,7 @@ func (m *manager) Stats() *ProxyStats {
m.mu.RLock()
defer m.mu.RUnlock()
// 返回副本以避免并发问题
statsCopy := *m.stats
return &statsCopy
return m.stats.snapshot()
}
// createDirectDialer 创建直连拨号器
@@ -146,6 +142,9 @@ func (m *manager) createDirectDialer() Dialer {
func (m *manager) createSOCKS5Dialer() (Dialer, error) {
// 检查缓存
cacheKey := fmt.Sprintf(CacheKeySOCKS5, m.config.Address)
if m.config.Username != "" || m.config.Password != "" {
cacheKey = fmt.Sprintf("%s_%s_%s", cacheKey, m.config.Username, m.config.Password)
}
m.cacheMu.RLock()
if time.Now().Before(m.cacheExpiry) {
if cached, exists := m.dialerCache[cacheKey]; exists {
@@ -155,18 +154,6 @@ func (m *manager) createSOCKS5Dialer() (Dialer, error) {
}
m.cacheMu.RUnlock()
// 解析代理地址
proxyURL := fmt.Sprintf(SOCKS5URLFormat, m.config.Address)
if m.config.Username != "" {
proxyURL = fmt.Sprintf(SOCKS5URLAuthFormat,
m.config.Username, m.config.Password, m.config.Address)
}
u, err := url.Parse(proxyURL)
if err != nil {
return nil, NewProxyError(ErrTypeConfig, ErrMsgSOCKS5ParseFailed, ErrCodeSOCKS5ParseFailed, err)
}
// 创建基础拨号器
baseDial := &net.Dialer{
Timeout: m.config.Timeout,
@@ -175,16 +162,14 @@ func (m *manager) createSOCKS5Dialer() (Dialer, error) {
// 创建SOCKS5拨号器
var auth *proxy.Auth
if u.User != nil {
if m.config.Username != "" || m.config.Password != "" {
auth = &proxy.Auth{
User: u.User.Username(),
}
if password, hasPassword := u.User.Password(); hasPassword {
auth.Password = password
User: m.config.Username,
Password: m.config.Password,
}
}
socksDialer, err := proxy.SOCKS5(NetworkTCP, u.Host, auth, baseDial)
socksDialer, err := proxy.SOCKS5(NetworkTCP, m.config.Address, auth, baseDial)
if err != nil {
return nil, NewProxyError(ErrTypeConnection, ErrMsgSOCKS5CreateFailed, ErrCodeSOCKS5CreateFailed, err)
}
@@ -248,7 +233,7 @@ func (d *directDialer) Dial(network, address string) (net.Conn, error) {
func (d *directDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&d.stats.TotalConnections, 1)
d.stats.addTotal(1)
dialer := &net.Dialer{
Timeout: d.timeout,
@@ -264,15 +249,16 @@ func (d *directDialer) DialContext(ctx context.Context, network, address string)
conn, err := dialer.DialContext(ctx, network, address)
duration := time.Since(start)
d.stats.LastConnectTime = start
d.stats.setLastConnectTime(start)
if err != nil {
atomic.AddInt64(&d.stats.FailedConnections, 1)
d.stats.LastError = err.Error()
d.stats.addFailed(1)
d.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgDirectConnFailed, ErrCodeDirectConnFailed, err)
}
atomic.AddInt64(&d.stats.ActiveConnections, 1)
d.stats.addActive(1)
d.updateAverageConnectTime(duration)
return &trackedConn{
@@ -294,7 +280,7 @@ func (s *socks5Dialer) Dial(network, address string) (net.Conn, error) {
func (s *socks5Dialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
start := time.Now()
atomic.AddInt64(&s.stats.TotalConnections, 1)
s.stats.addTotal(1)
// 创建一个带超时的上下文
dialCtx, cancel := context.WithTimeout(ctx, s.config.Timeout)
@@ -322,20 +308,21 @@ func (s *socks5Dialer) DialContext(ctx context.Context, network, address string)
select {
case <-dialCtx.Done():
atomic.AddInt64(&s.stats.FailedConnections, 1)
s.stats.LastError = dialCtx.Err().Error()
s.stats.addFailed(1)
s.stats.setLastError(dialCtx.Err().Error())
return nil, NewProxyError(ErrTypeTimeout, ErrMsgSOCKS5ConnTimeout, ErrCodeSOCKS5ConnTimeout, dialCtx.Err())
case result := <-connChan:
duration := time.Since(start)
s.stats.LastConnectTime = start
s.stats.setLastConnectTime(start)
if result.err != nil {
atomic.AddInt64(&s.stats.FailedConnections, 1)
s.stats.LastError = result.err.Error()
s.stats.addFailed(1)
s.stats.setLastError(result.err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgSOCKS5ConnFailed, ErrCodeSOCKS5ConnFailed, result.err)
}
atomic.AddInt64(&s.stats.ActiveConnections, 1)
s.stats.addActive(1)
s.updateAverageConnectTime(duration)
return &trackedConn{
@@ -347,7 +334,8 @@ func (s *socks5Dialer) DialContext(ctx context.Context, network, address string)
// updateAverageConnectTime 更新平均连接时间
func (d *directDialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
d.stats.mu.Lock()
defer d.stats.mu.Unlock()
if d.stats.AverageConnectTime == 0 {
d.stats.AverageConnectTime = duration
} else {
@@ -356,7 +344,8 @@ func (d *directDialer) updateAverageConnectTime(duration time.Duration) {
}
func (s *socks5Dialer) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
s.stats.mu.Lock()
defer s.stats.mu.Unlock()
if s.stats.AverageConnectTime == 0 {
s.stats.AverageConnectTime = duration
} else {
+156
View File
@@ -1,6 +1,9 @@
package proxy
import (
"fmt"
"io"
"net"
"testing"
"time"
)
@@ -242,6 +245,73 @@ func TestGetDialer_HTTPS(t *testing.T) {
t.Logf("✓ GetDialer 返回HTTPS代理拨号器")
}
func TestGetDialer_SOCKS5AuthSpecialChars(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen failed: %v", err)
}
defer ln.Close()
type credential struct {
user string
pass string
}
authCh := make(chan credential, 1)
errCh := make(chan error, 1)
go func() {
conn, err := ln.Accept()
if err != nil {
errCh <- err
return
}
defer conn.Close()
user, pass, err := handleTestSOCKS5Auth(conn)
if err != nil {
errCh <- err
return
}
authCh <- credential{user: user, pass: pass}
}()
origProbed := IsProxyProbed()
SetProxyProbed(true)
defer SetProxyProbed(origProbed)
config := &ProxyConfig{
Type: ProxyTypeSOCKS5,
Address: ln.Addr().String(),
Username: "user",
Password: "p@ss:word#1",
Timeout: time.Second,
}
manager := NewProxyManager(config)
dialer, err := manager.GetDialer()
if err != nil {
t.Fatalf("GetDialer failed: %v", err)
}
conn, err := dialer.Dial("tcp", "127.0.0.1:80")
if err != nil {
t.Fatalf("SOCKS5 dial failed: %v", err)
}
_ = conn.Close()
select {
case got := <-authCh:
if got.user != config.Username || got.pass != config.Password {
t.Fatalf("auth = %q/%q, want %q/%q", got.user, got.pass, config.Username, config.Password)
}
case err := <-errCh:
t.Fatalf("SOCKS5 test server failed: %v", err)
case <-time.After(time.Second):
t.Fatal("timed out waiting for SOCKS5 auth")
}
t.Logf("✓ SOCKS5认证支持特殊字符密码")
}
// =============================================================================
// GetTLSDialer - TLS拨号器获取测试
// =============================================================================
@@ -559,3 +629,89 @@ func TestDirectDialer_LocalAddr_Loopback(t *testing.T) {
t.Logf("✓ LocalAddr 绑定正常工作(连接失败是预期的): %v", err)
}
}
func handleTestSOCKS5Auth(conn net.Conn) (string, string, error) {
header := make([]byte, 2)
if _, err := io.ReadFull(conn, header); err != nil {
return "", "", err
}
if header[0] != 0x05 {
return "", "", fmt.Errorf("unexpected socks version: %d", header[0])
}
methods := make([]byte, int(header[1]))
if _, err := io.ReadFull(conn, methods); err != nil {
return "", "", err
}
hasAuth := false
for _, method := range methods {
if method == 0x02 {
hasAuth = true
break
}
}
if !hasAuth {
return "", "", fmt.Errorf("client did not offer username/password auth")
}
if _, err := conn.Write([]byte{0x05, 0x02}); err != nil {
return "", "", err
}
authHeader := make([]byte, 2)
if _, err := io.ReadFull(conn, authHeader); err != nil {
return "", "", err
}
if authHeader[0] != 0x01 {
return "", "", fmt.Errorf("unexpected auth version: %d", authHeader[0])
}
userBytes := make([]byte, int(authHeader[1]))
if _, err := io.ReadFull(conn, userBytes); err != nil {
return "", "", err
}
passLen := make([]byte, 1)
if _, err := io.ReadFull(conn, passLen); err != nil {
return "", "", err
}
passBytes := make([]byte, int(passLen[0]))
if _, err := io.ReadFull(conn, passBytes); err != nil {
return "", "", err
}
if _, err := conn.Write([]byte{0x01, 0x00}); err != nil {
return "", "", err
}
reqHeader := make([]byte, 4)
if _, err := io.ReadFull(conn, reqHeader); err != nil {
return "", "", err
}
if reqHeader[0] != 0x05 || reqHeader[1] != 0x01 {
return "", "", fmt.Errorf("unexpected request header: %v", reqHeader)
}
if err := discardSOCKS5Address(conn, reqHeader[3]); err != nil {
return "", "", err
}
if _, err := conn.Write([]byte{0x05, 0x00, 0x00, 0x01, 0, 0, 0, 0, 0, 0}); err != nil {
return "", "", err
}
return string(userBytes), string(passBytes), nil
}
func discardSOCKS5Address(conn net.Conn, atyp byte) error {
switch atyp {
case 0x01:
_, err := io.CopyN(io.Discard, conn, 6)
return err
case 0x03:
length := make([]byte, 1)
if _, err := io.ReadFull(conn, length); err != nil {
return err
}
_, err := io.CopyN(io.Discard, conn, int64(length[0])+2)
return err
case 0x04:
_, err := io.CopyN(io.Discard, conn, 18)
return err
default:
return fmt.Errorf("unsupported atyp: %d", atyp)
}
}
+10 -9
View File
@@ -49,8 +49,8 @@ func (t *tlsDialerWrapper) DialTLSContext(ctx context.Context, network, address
// 进行TLS握手
if err := tlsConn.Handshake(); err != nil {
_ = tcpConn.Close() // TLS握手失败,Close错误可忽略
atomic.AddInt64(&t.stats.FailedConnections, 1)
t.stats.LastError = err.Error()
t.stats.addFailed(1)
t.stats.setLastError(err.Error())
return nil, NewProxyError(ErrTypeConnection, ErrMsgTLSHandshakeFailed, ErrCodeTLSHandshakeFailed, err)
}
@@ -71,7 +71,8 @@ func (t *tlsDialerWrapper) DialTLSContext(ctx context.Context, network, address
// updateAverageConnectTime 更新平均连接时间
func (t *tlsDialerWrapper) updateAverageConnectTime(duration time.Duration) {
// 简单的移动平均
t.stats.mu.Lock()
defer t.stats.mu.Unlock()
if t.stats.AverageConnectTime == 0 {
t.stats.AverageConnectTime = duration
} else {
@@ -81,16 +82,16 @@ func (t *tlsDialerWrapper) updateAverageConnectTime(duration time.Duration) {
// trackedConn 带统计的连接
type trackedConn struct {
bytesSent atomic.Int64
bytesRecv atomic.Int64
net.Conn
stats *ProxyStats
bytesSent int64
bytesRecv int64
stats *ProxyStats
}
func (tc *trackedConn) Read(b []byte) (n int, err error) {
n, err = tc.Conn.Read(b)
if n > 0 {
atomic.AddInt64(&tc.bytesRecv, int64(n))
tc.bytesRecv.Add(int64(n))
}
return n, err
}
@@ -98,13 +99,13 @@ func (tc *trackedConn) Read(b []byte) (n int, err error) {
func (tc *trackedConn) Write(b []byte) (n int, err error) {
n, err = tc.Conn.Write(b)
if n > 0 {
atomic.AddInt64(&tc.bytesSent, int64(n))
tc.bytesSent.Add(int64(n))
}
return n, err
}
func (tc *trackedConn) Close() error {
atomic.AddInt64(&tc.stats.ActiveConnections, -1)
tc.stats.addActive(-1)
return tc.Conn.Close()
}
+48
View File
@@ -4,6 +4,7 @@ import (
"context"
"crypto/tls"
"net"
"sync"
"time"
)
@@ -98,6 +99,7 @@ type ProxyStats struct {
TotalConnections int64 `json:"total_connections"`
ActiveConnections int64 `json:"active_connections"`
FailedConnections int64 `json:"failed_connections"`
mu sync.Mutex `json:"-"`
AverageConnectTime time.Duration `json:"average_connect_time"`
LastConnectTime time.Time `json:"last_connect_time"`
LastError string `json:"last_error,omitempty"`
@@ -105,6 +107,52 @@ type ProxyStats struct {
ProxyAddress string `json:"proxy_address"`
}
func (s *ProxyStats) addTotal(delta int64) {
s.mu.Lock()
s.TotalConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) addActive(delta int64) {
s.mu.Lock()
s.ActiveConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) addFailed(delta int64) {
s.mu.Lock()
s.FailedConnections += delta
s.mu.Unlock()
}
func (s *ProxyStats) setLastConnectTime(t time.Time) {
s.mu.Lock()
s.LastConnectTime = t
s.mu.Unlock()
}
func (s *ProxyStats) setLastError(err string) {
s.mu.Lock()
s.LastError = err
s.mu.Unlock()
}
func (s *ProxyStats) snapshot() *ProxyStats {
s.mu.Lock()
defer s.mu.Unlock()
return &ProxyStats{
TotalConnections: s.TotalConnections,
ActiveConnections: s.ActiveConnections,
FailedConnections: s.FailedConnections,
AverageConnectTime: s.AverageConnectTime,
LastConnectTime: s.LastConnectTime,
LastError: s.LastError,
ProxyType: s.ProxyType,
ProxyAddress: s.ProxyAddress,
}
}
// ProxyError 代理错误类型
//
//nolint:revive // 保持与现有代码的向后兼容性
+62
View File
@@ -0,0 +1,62 @@
package common
import "testing"
func TestResultCallbackLifecycle(t *testing.T) {
ClearResultCallback()
t.Cleanup(ClearResultCallback)
called := false
SetResultCallback(func(result interface{}) {
called = true
if result != "payload" {
t.Fatalf("callback payload = %#v", result)
}
})
NotifyResult("payload")
if !called {
t.Fatal("callback was not called")
}
called = false
ClearResultCallback()
NotifyResult("payload")
if called {
t.Fatal("callback should not be called after ClearResultCallback")
}
}
func TestStateRuntimeTargetsAndShellFlags(t *testing.T) {
state := NewState()
urls := []string{"http://example.com", "https://example.org"}
state.SetURLs(urls)
if got := state.GetURLs(); len(got) != 2 || got[0] != urls[0] || got[1] != urls[1] {
t.Fatalf("urls = %#v", got)
}
hostPorts := []string{"127.0.0.1:80", "[::1]:443"}
state.SetHostPorts(hostPorts)
if got := state.GetHostPorts(); len(got) != 2 || got[0] != hostPorts[0] || got[1] != hostPorts[1] {
t.Fatalf("hostPorts = %#v", got)
}
state.ClearHostPorts()
if got := state.GetHostPorts(); got != nil {
t.Fatalf("hostPorts after clear = %#v, want nil", got)
}
state.SetForwardShellActive(true)
state.SetReverseShellActive(true)
state.SetSocks5ProxyActive(true)
if !state.IsForwardShellActive() || !state.IsReverseShellActive() || !state.IsSocks5ProxyActive() {
t.Fatal("shell/proxy flags should be active")
}
state.SetForwardShellActive(false)
state.SetReverseShellActive(false)
state.SetSocks5ProxyActive(false)
if state.IsForwardShellActive() || state.IsReverseShellActive() || state.IsSocks5ProxyActive() {
t.Fatal("shell/proxy flags should be inactive")
}
}
+231
View File
@@ -0,0 +1,231 @@
package common
import (
"context"
"fmt"
"net"
"net/http"
"strings"
"sync"
"time"
"github.com/shadow1ng/fscan/common/i18n"
"github.com/shadow1ng/fscan/common/output"
"github.com/shadow1ng/fscan/common/proxy"
)
// ResultSink receives structured scan results for one scan session.
type ResultSink func(result *output.ScanResult) error
// ScanSession 封装单次扫描的全部上下文
// 一次扫描一个 session,并发扫描各自独立
type ScanSession struct {
Config *Config // 不可变,创建后只读
State *State // 可变,原子操作,每会话独立
Params *FlagVars // 原始参数,只读
ResultSink ResultSink // 可选,覆盖全局输出
PauseGate func(ctx context.Context) error
// 每会话 dialer(按 timeout 懒初始化,取决于代理配置)
dialerMu sync.Mutex
dialers map[time.Duration]proxy.Dialer
dialerErrs map[time.Duration]error
}
// NewScanSession 从已构建的 Config、State 和 FlagVars 创建会话
func NewScanSession(config *Config, state *State, params *FlagVars) *ScanSession {
return &ScanSession{
Config: config,
State: state,
Params: params,
}
}
// SaveResult saves a scan result through the session sink if present, otherwise
// falls back to the process-wide output pipeline used by the CLI.
func (s *ScanSession) SaveResult(result *output.ScanResult) error {
if s != nil && s.ResultSink != nil {
return s.ResultSink(result)
}
return SaveResult(result)
}
func (s *ScanSession) loggingEnabled() bool {
return s == nil || s.Config == nil || !s.Config.Output.Silent
}
// LogDebug writes through the session's logging policy.
func (s *ScanSession) LogDebug(msg string) {
if s.loggingEnabled() {
LogDebug(msg)
}
}
// LogInfo writes through the session's logging policy.
func (s *ScanSession) LogInfo(msg string) {
if s.loggingEnabled() {
LogInfo(msg)
}
}
// LogSuccess writes through the session's logging policy.
func (s *ScanSession) LogSuccess(result string) {
if s.loggingEnabled() {
LogSuccess(result)
}
}
// LogVuln writes through the session's logging policy.
func (s *ScanSession) LogVuln(result string) {
if s.loggingEnabled() {
LogVuln(result)
}
}
// LogError writes through the session's logging policy.
func (s *ScanSession) LogError(errMsg string) {
if s.loggingEnabled() {
LogError(errMsg)
}
}
// DialTCP 创建 TCP 连接,内含限速检查、代理、计数
func (s *ScanSession) DialTCP(ctx context.Context, network, address string, timeout time.Duration) (net.Conn, error) {
// 检查发包限制
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
s.LogError(i18n.Tr("tcp_connection_restricted", address, err.Error()))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
// 获取 dialer
dialer, err := s.getDialer(timeout)
if err != nil {
s.LogError(i18n.Tr("proxy_dialer_failed", err))
s.State.IncrementTCPFailedPacketCount()
return nil, err
}
conn, err := dialer.DialContext(ctx, network, address)
if err != nil {
s.State.IncrementTCPFailedPacketCount()
s.LogDebug(i18n.Tr("connection_failed", address, err))
return nil, err
}
// SO_LINGER=0: 连接关闭时立即发送 RST,避免 TIME_WAIT 堆积
if tc, ok := conn.(*net.TCPConn); ok {
_ = tc.SetLinger(0)
}
s.State.IncrementTCPSuccessPacketCount()
return conn, nil
}
// DialUDP creates a connected UDP socket with rate limiting and packet counting.
// UDP cannot be proxied; if a proxy is configured the connection is made directly.
func (s *ScanSession) DialUDP(ctx context.Context, address string, timeout time.Duration) (net.Conn, error) {
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
conn, err := net.DialTimeout("udp", address, timeout)
if err != nil {
s.State.IncrementUDPPacketCount()
return nil, err
}
_ = conn.SetDeadline(time.Now().Add(timeout))
s.State.IncrementUDPPacketCount()
return conn, nil
}
// HTTPDo executes an HTTP request with the session's packet limits and counters.
func (s *ScanSession) HTTPDo(client *http.Client, req *http.Request) (*http.Response, error) {
if ok, err := CanSendPacketWith(s.Config, s.State); !ok {
s.LogError(i18n.Tr("http_request_restricted", req.URL.String(), err.Error()))
return nil, fmt.Errorf("%s", i18n.Tr("network_rate_limited", err.Error()))
}
resp, err := client.Do(req)
if err != nil {
s.State.IncrementTCPFailedPacketCount()
return nil, err
}
s.State.IncrementTCPSuccessPacketCount()
return resp, nil
}
// ProxyEnabled reports whether this scan session uses a network proxy.
func (s *ScanSession) ProxyEnabled() bool {
if s == nil || s.Config == nil {
return false
}
return s.Config.Network.Socks5Proxy != "" || s.Config.Network.HTTPProxy != ""
}
// IsSOCKS5Proxy reports whether this scan session uses SOCKS5.
func (s *ScanSession) IsSOCKS5Proxy() bool {
return s != nil && s.Config != nil && s.Config.Network.Socks5Proxy != ""
}
// ProxyReliable reports whether the session proxy should be treated as reliable.
func (s *ScanSession) ProxyReliable() bool {
if !s.ProxyEnabled() || !s.IsSOCKS5Proxy() {
return true
}
return proxy.IsProxyReliable()
}
func (s *ScanSession) getDialer(timeout time.Duration) (proxy.Dialer, error) {
if timeout <= 0 {
timeout = s.Config.Timeout
}
s.dialerMu.Lock()
defer s.dialerMu.Unlock()
if s.dialers == nil {
s.dialers = make(map[time.Duration]proxy.Dialer)
s.dialerErrs = make(map[time.Duration]error)
}
if dialer, ok := s.dialers[timeout]; ok {
return dialer, s.dialerErrs[timeout]
}
cfg := s.createProxyConfig(timeout)
manager := proxy.NewProxyManager(cfg)
dialer, err := manager.GetDialer()
s.dialers[timeout] = dialer
s.dialerErrs[timeout] = err
return dialer, err
}
func (s *ScanSession) createProxyConfig(timeout time.Duration) *proxy.ProxyConfig {
cfg := proxy.DefaultProxyConfig()
cfg.Timeout = timeout
cfg.LocalAddr = s.Config.Network.Iface
// 优先 SOCKS5
if s.Config.Network.Socks5Proxy != "" {
cfg.Type = proxy.ProxyTypeSOCKS5
socks5URL := s.Config.Network.Socks5Proxy
if !strings.HasPrefix(socks5URL, "socks5://") {
socks5URL = "socks5://" + socks5URL
}
cfg.Address, cfg.Username, cfg.Password = parseProxyURL(socks5URL, s.Config.Network.Socks5Proxy)
return cfg
}
// 其次 HTTP
if s.Config.Network.HTTPProxy != "" {
if strings.HasPrefix(s.Config.Network.HTTPProxy, "https://") {
cfg.Type = proxy.ProxyTypeHTTPS
} else {
cfg.Type = proxy.ProxyTypeHTTP
}
cfg.Address, cfg.Username, cfg.Password = parseProxyURL(s.Config.Network.HTTPProxy, s.Config.Network.HTTPProxy)
return cfg
}
cfg.Type = proxy.ProxyTypeNone
return cfg
}
+249
View File
@@ -0,0 +1,249 @@
package common
import (
"io"
"net/http"
"strings"
"testing"
"time"
"github.com/shadow1ng/fscan/common/output"
)
func TestScanSessionLogMethodsHonorSilentConfig(t *testing.T) {
loggerMu.Lock()
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
t.Cleanup(func() {
loggerMu.Lock()
silentLoggerRefs = 0
resetLoggerLocked()
loggerMu.Unlock()
})
cfg := NewConfig()
cfg.Output.Silent = true
session := NewScanSession(cfg, NewState(), &FlagVars{})
session.LogDebug("debug")
session.LogInfo("info")
session.LogSuccess("success")
session.LogVuln("vuln")
session.LogError("error")
loggerMu.Lock()
defer loggerMu.Unlock()
if globalLogger != nil {
t.Fatal("silent session log methods initialized global logger")
}
}
func TestScanSessionDialerCacheIsTimeoutAware(t *testing.T) {
cfg := NewConfig()
cfg.Timeout = 5 * time.Second
session := NewScanSession(cfg, NewState(), &FlagVars{})
shortTimeout := 100 * time.Millisecond
longTimeout := 2 * time.Second
shortDialer, err := session.getDialer(shortTimeout)
if err != nil {
t.Fatal(err)
}
shortDialerAgain, err := session.getDialer(shortTimeout)
if err != nil {
t.Fatal(err)
}
longDialer, err := session.getDialer(longTimeout)
if err != nil {
t.Fatal(err)
}
if shortDialer != shortDialerAgain {
t.Fatal("same timeout should reuse the session dialer")
}
if shortDialer == longDialer {
t.Fatal("different timeouts should not share one session dialer")
}
if got := session.createProxyConfig(shortTimeout).Timeout; got != shortTimeout {
t.Fatalf("proxy timeout = %v, want %v", got, shortTimeout)
}
}
func TestScanSessionHTTPDoUsesSessionState(t *testing.T) {
previousState := GetGlobalState()
globalState := NewState()
SetGlobalState(globalState)
t.Cleanup(func() { SetGlobalState(previousState) })
sessionState := NewState()
session := NewScanSession(NewConfig(), sessionState, &FlagVars{})
client := &http.Client{
Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusNoContent,
Body: io.NopCloser(strings.NewReader("")),
Header: make(http.Header),
}, nil
}),
}
req, err := http.NewRequest(http.MethodHead, "http://example.com", nil)
if err != nil {
t.Fatal(err)
}
resp, err := session.HTTPDo(client, req)
if err != nil {
t.Fatal(err)
}
_ = resp.Body.Close()
if got := sessionState.GetTCPSuccessPacketCount(); got != 1 {
t.Fatalf("session TCP success count = %d, want 1", got)
}
if got := globalState.GetTCPSuccessPacketCount(); got != 0 {
t.Fatalf("global TCP success count = %d, want 0", got)
}
}
func TestScanSessionProxyStateComesFromConfig(t *testing.T) {
direct := NewScanSession(NewConfig(), NewState(), &FlagVars{})
if direct.ProxyEnabled() {
t.Fatal("direct session should not report proxy enabled")
}
if direct.IsSOCKS5Proxy() {
t.Fatal("direct session should not report SOCKS5")
}
if !direct.ProxyReliable() {
t.Fatal("direct session should be reliable")
}
httpCfg := NewConfig()
httpCfg.Network.HTTPProxy = "http://127.0.0.1:8080"
httpSession := NewScanSession(httpCfg, NewState(), &FlagVars{})
if !httpSession.ProxyEnabled() {
t.Fatal("HTTP proxy session should report proxy enabled")
}
if httpSession.IsSOCKS5Proxy() {
t.Fatal("HTTP proxy session should not report SOCKS5")
}
if !httpSession.ProxyReliable() {
t.Fatal("HTTP proxy session should be reliable")
}
socksCfg := NewConfig()
socksCfg.Network.Socks5Proxy = "127.0.0.1:1080"
socksSession := NewScanSession(socksCfg, NewState(), &FlagVars{})
if !socksSession.ProxyEnabled() {
t.Fatal("SOCKS5 proxy session should report proxy enabled")
}
if !socksSession.IsSOCKS5Proxy() {
t.Fatal("SOCKS5 proxy session should report SOCKS5")
}
}
func TestParseProxyURLFallsBackWhenHostIsEmpty(t *testing.T) {
host, username, password := parseProxyURL("127.0.0.1:8080", "127.0.0.1:8080")
if host != "127.0.0.1:8080" {
t.Fatalf("host = %q, want fallback address", host)
}
if username != "" || password != "" {
t.Fatalf("unexpected credentials: %q/%q", username, password)
}
}
func TestParseProxyURLExtractsAuthWithoutScheme(t *testing.T) {
host, username, password := parseProxyURL("user:[email protected]:8080", "user:[email protected]:8080")
if host != "127.0.0.1:8080" {
t.Fatalf("host = %q, want proxy address", host)
}
if username != "user" || password != "pass" {
t.Fatalf("credentials = %q/%q, want user/pass", username, password)
}
}
// TestScanSessionSaveResultUsesSink 测试 SaveResult 通过 ResultSink 分发
func TestScanSessionSaveResultUsesSink(t *testing.T) {
preserveOutputAPIGlobals(t)
cfg := NewConfig()
cfg.Output.DisableSave = true
SetGlobalConfig(cfg)
flagVars = &FlagVars{DisableSave: true}
_ = InitOutput()
var sinkGot *output.ScanResult
session := NewScanSession(cfg, NewState(), &FlagVars{})
session.ResultSink = func(r *output.ScanResult) error {
sinkGot = r
return nil
}
result := &output.ScanResult{
Type: output.TypeHost,
Target: "10.0.0.1",
Status: "ALIVE",
}
if err := session.SaveResult(result); err != nil {
t.Fatalf("session.SaveResult error = %v", err)
}
if sinkGot != result {
t.Fatalf("ResultSink 未被调用或参数不符: got %v", sinkGot)
}
}
// TestScanSessionSaveResultFallsBackToGlobal 测试无 sink 时回退到全局 SaveResult
func TestScanSessionSaveResultFallsBackToGlobal(t *testing.T) {
preserveOutputAPIGlobals(t)
cfg := NewConfig()
cfg.Output.DisableSave = true
SetGlobalConfig(cfg)
flagVars = &FlagVars{DisableSave: true}
_ = InitOutput()
called := false
SetResultCallback(func(payload interface{}) {
called = true
})
session := NewScanSession(cfg, NewState(), &FlagVars{})
// 不设置 ResultSink,应回退到全局
result := &output.ScanResult{
Type: output.TypeHost,
Target: "10.0.0.2",
Status: "ALIVE",
}
if err := session.SaveResult(result); err != nil {
t.Fatalf("session.SaveResult (fallback) error = %v", err)
}
if !called {
t.Fatal("回退到全局 SaveResult 时应触发 ResultCallback")
}
}
// TestScanSessionLogMethodsEnabledByDefault 测试非 Silent 配置下 Log 方法不被屏蔽
func TestScanSessionLogMethodsEnabledByDefault(t *testing.T) {
cfg := NewConfig()
cfg.Output.Silent = false
session := NewScanSession(cfg, NewState(), &FlagVars{})
if !session.loggingEnabled() {
t.Fatal("非 Silent 配置下 loggingEnabled 应返回 true")
}
}
// TestNilScanSessionLoggingEnabled 测试 nil session 的 loggingEnabled
func TestNilScanSessionLoggingEnabled(t *testing.T) {
var session *ScanSession
if !session.loggingEnabled() {
t.Fatal("nil session 的 loggingEnabled 应返回 true(安全降级)")
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
+67 -49
View File
@@ -23,17 +23,17 @@ state.go - 运行时状态管理
// State 扫描器运行时状态 - 线程安全
type State struct {
// 计数器 - 原子操作
packetCount int64
tcpPacketCount int64
tcpSuccessPacketCount int64
tcpFailedPacketCount int64
udpPacketCount int64
httpPacketCount int64
resourceExhaustedCount int64
packetCount atomic.Int64
tcpPacketCount atomic.Int64
tcpSuccessPacketCount atomic.Int64
tcpFailedPacketCount atomic.Int64
udpPacketCount atomic.Int64
httpPacketCount atomic.Int64
resourceExhaustedCount atomic.Int64
// 任务计数
end int64
num int64
end atomic.Int64
num atomic.Int64
// 时间
startTime time.Time
@@ -56,6 +56,10 @@ type State struct {
forwardShellActive int32 // 使用int32以便原子操作
reverseShellActive int32
socks5ProxyActive int32
// 服务识别缓存(per-session,避免跨扫描污染)
// key: "host:port", value: interface{}core.ServiceInfo 指针)
serviceCache sync.Map
}
// NewState 创建新的状态对象
@@ -71,38 +75,38 @@ func NewState() *State {
// IncrementPacketCount 增加总包计数
func (s *State) IncrementPacketCount() int64 {
return atomic.AddInt64(&s.packetCount, 1)
return s.packetCount.Add(1)
}
// IncrementTCPSuccessPacketCount 增加TCP成功连接包计数
func (s *State) IncrementTCPSuccessPacketCount() int64 {
atomic.AddInt64(&s.tcpSuccessPacketCount, 1)
atomic.AddInt64(&s.tcpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.tcpSuccessPacketCount.Add(1)
s.tcpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementTCPFailedPacketCount 增加TCP失败连接包计数
func (s *State) IncrementTCPFailedPacketCount() int64 {
atomic.AddInt64(&s.tcpFailedPacketCount, 1)
atomic.AddInt64(&s.tcpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.tcpFailedPacketCount.Add(1)
s.tcpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementUDPPacketCount 增加UDP包计数
func (s *State) IncrementUDPPacketCount() int64 {
atomic.AddInt64(&s.udpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.udpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementHTTPPacketCount 增加HTTP包计数
func (s *State) IncrementHTTPPacketCount() int64 {
atomic.AddInt64(&s.httpPacketCount, 1)
return atomic.AddInt64(&s.packetCount, 1)
s.httpPacketCount.Add(1)
return s.packetCount.Add(1)
}
// IncrementResourceExhaustedCount 增加资源耗尽错误计数
func (s *State) IncrementResourceExhaustedCount() {
atomic.AddInt64(&s.resourceExhaustedCount, 1)
s.resourceExhaustedCount.Add(1)
}
// =============================================================================
@@ -111,48 +115,48 @@ func (s *State) IncrementResourceExhaustedCount() {
// GetPacketCount 获取总包计数
func (s *State) GetPacketCount() int64 {
return atomic.LoadInt64(&s.packetCount)
return s.packetCount.Load()
}
// GetTCPPacketCount 获取TCP包计数
func (s *State) GetTCPPacketCount() int64 {
return atomic.LoadInt64(&s.tcpPacketCount)
return s.tcpPacketCount.Load()
}
// GetTCPSuccessPacketCount 获取TCP成功连接包计数
func (s *State) GetTCPSuccessPacketCount() int64 {
return atomic.LoadInt64(&s.tcpSuccessPacketCount)
return s.tcpSuccessPacketCount.Load()
}
// GetTCPFailedPacketCount 获取TCP失败连接包计数
func (s *State) GetTCPFailedPacketCount() int64 {
return atomic.LoadInt64(&s.tcpFailedPacketCount)
return s.tcpFailedPacketCount.Load()
}
// GetUDPPacketCount 获取UDP包计数
func (s *State) GetUDPPacketCount() int64 {
return atomic.LoadInt64(&s.udpPacketCount)
return s.udpPacketCount.Load()
}
// GetHTTPPacketCount 获取HTTP包计数
func (s *State) GetHTTPPacketCount() int64 {
return atomic.LoadInt64(&s.httpPacketCount)
return s.httpPacketCount.Load()
}
// GetResourceExhaustedCount 获取资源耗尽错误计数
func (s *State) GetResourceExhaustedCount() int64 {
return atomic.LoadInt64(&s.resourceExhaustedCount)
return s.resourceExhaustedCount.Load()
}
// ResetPacketCounters 重置所有包计数器
func (s *State) ResetPacketCounters() {
atomic.StoreInt64(&s.packetCount, 0)
atomic.StoreInt64(&s.tcpPacketCount, 0)
atomic.StoreInt64(&s.tcpSuccessPacketCount, 0)
atomic.StoreInt64(&s.tcpFailedPacketCount, 0)
atomic.StoreInt64(&s.udpPacketCount, 0)
atomic.StoreInt64(&s.httpPacketCount, 0)
atomic.StoreInt64(&s.resourceExhaustedCount, 0)
s.packetCount.Store(0)
s.tcpPacketCount.Store(0)
s.tcpSuccessPacketCount.Store(0)
s.tcpFailedPacketCount.Store(0)
s.udpPacketCount.Store(0)
s.httpPacketCount.Store(0)
s.resourceExhaustedCount.Store(0)
}
// =============================================================================
@@ -161,32 +165,32 @@ func (s *State) ResetPacketCounters() {
// GetEnd 获取结束计数
func (s *State) GetEnd() int64 {
return atomic.LoadInt64(&s.end)
return s.end.Load()
}
// GetNum 获取数量计数
func (s *State) GetNum() int64 {
return atomic.LoadInt64(&s.num)
return s.num.Load()
}
// IncrementEnd 增加结束计数
func (s *State) IncrementEnd() int64 {
return atomic.AddInt64(&s.end, 1)
return s.end.Add(1)
}
// IncrementNum 增加数量计数
func (s *State) IncrementNum() int64 {
return atomic.AddInt64(&s.num, 1)
return s.num.Add(1)
}
// SetEnd 设置结束计数
func (s *State) SetEnd(val int64) {
atomic.StoreInt64(&s.end, val)
s.end.Store(val)
}
// SetNum 设置数量计数
func (s *State) SetNum(val int64) {
atomic.StoreInt64(&s.num, val)
s.num.Store(val)
}
// =============================================================================
@@ -271,10 +275,10 @@ type PerfStatsData struct {
func (s *State) GetPerfStats() PerfStatsData {
duration := time.Since(s.startTime)
durationMs := duration.Milliseconds()
totalPackets := atomic.LoadInt64(&s.packetCount)
tcpSuccess := atomic.LoadInt64(&s.tcpSuccessPacketCount)
tcpFailed := atomic.LoadInt64(&s.tcpFailedPacketCount)
tcpTotal := atomic.LoadInt64(&s.tcpPacketCount)
totalPackets := s.packetCount.Load()
tcpSuccess := s.tcpSuccessPacketCount.Load()
tcpFailed := s.tcpFailedPacketCount.Load()
tcpTotal := s.tcpPacketCount.Load()
var pps float64
if durationMs > 0 {
@@ -291,13 +295,13 @@ func (s *State) GetPerfStats() PerfStatsData {
TCPPackets: tcpTotal,
TCPSuccess: tcpSuccess,
TCPFailed: tcpFailed,
UDPPackets: atomic.LoadInt64(&s.udpPacketCount),
HTTPPackets: atomic.LoadInt64(&s.httpPacketCount),
ResourceExhausted: atomic.LoadInt64(&s.resourceExhaustedCount),
UDPPackets: s.udpPacketCount.Load(),
HTTPPackets: s.httpPacketCount.Load(),
ResourceExhausted: s.resourceExhaustedCount.Load(),
ScanDurationMs: durationMs,
PacketsPerSecond: pps,
SuccessRate: successRate,
TargetsScanned: atomic.LoadInt64(&s.num),
TargetsScanned: s.num.Load(),
}
}
@@ -455,3 +459,17 @@ func (s *State) CheckAndIncrementPacketRate(rateLimit int64) (bool, error) {
return true, nil
}
// =============================================================================
// 服务识别缓存 - per-session,消除跨扫描污染
// =============================================================================
// CacheService 缓存服务信息
func (s *State) CacheService(key string, info interface{}) {
s.serviceCache.Store(key, info)
}
// GetCachedService 获取缓存的服务信息
func (s *State) GetCachedService(key string) (interface{}, bool) {
return s.serviceCache.Load(key)
}
+295
View File
@@ -215,6 +215,301 @@ func TestState_ConcurrentTaskCounters(t *testing.T) {
}
}
// TestState_GetOutputMutex 测试获取输出互斥锁指针
func TestState_GetOutputMutex(t *testing.T) {
s := NewState()
mu := s.GetOutputMutex()
if mu == nil {
t.Fatal("GetOutputMutex returned nil")
}
// 验证返回的指针可以正常加锁解锁
mu.Lock()
_ = 1 //nolint:staticcheck // SA2001: 故意测试空临界区
mu.Unlock()
}
// TestState_GetICMPLimiter 测试 ICMP 限速器延迟初始化
func TestState_GetICMPLimiter(t *testing.T) {
s := NewState()
limiter := s.GetICMPLimiter(0.1)
if limiter == nil {
t.Fatal("GetICMPLimiter returned nil")
}
// 再次调用应返回同一个实例(sync.Once 保证)
limiter2 := s.GetICMPLimiter(0.5)
if limiter != limiter2 {
t.Fatal("GetICMPLimiter should return the same instance on repeated calls")
}
}
// TestState_GetICMPLimiterMinRate 测试极低速率下的 ICMP 限速器
func TestState_GetICMPLimiterMinRate(t *testing.T) {
s := NewState()
// 极低速率(packetsPerSecond < 1)应被钳位到 1
limiter := s.GetICMPLimiter(0.000001)
if limiter == nil {
t.Fatal("GetICMPLimiter with tiny rate returned nil")
}
}
// TestState_GetPerfStats 测试性能统计数据
func TestState_GetPerfStats(t *testing.T) {
s := NewState()
// 初始状态:全零
stats := s.GetPerfStats()
if stats.TotalPackets != 0 {
t.Errorf("初始 TotalPackets 应为 0, 实际 %d", stats.TotalPackets)
}
if stats.SuccessRate != 0 {
t.Errorf("初始 SuccessRate 应为 0, 实际 %f", stats.SuccessRate)
}
// 增加一些计数后验证统计
s.IncrementTCPSuccessPacketCount()
s.IncrementTCPSuccessPacketCount()
s.IncrementTCPFailedPacketCount()
s.SetNum(3)
stats = s.GetPerfStats()
if stats.TotalPackets != 3 {
t.Errorf("TotalPackets 期望 3, 实际 %d", stats.TotalPackets)
}
if stats.TCPSuccess != 2 {
t.Errorf("TCPSuccess 期望 2, 实际 %d", stats.TCPSuccess)
}
if stats.TCPFailed != 1 {
t.Errorf("TCPFailed 期望 1, 实际 %d", stats.TCPFailed)
}
if stats.TargetsScanned != 3 {
t.Errorf("TargetsScanned 期望 3, 实际 %d", stats.TargetsScanned)
}
// success rate = 2/3 * 100 ≈ 66.67%
if stats.SuccessRate < 66 || stats.SuccessRate > 67 {
t.Errorf("SuccessRate 期望约 66.67, 实际 %f", stats.SuccessRate)
}
}
// TestState_GetPerfStatsJSON 测试性能统计 JSON 序列化
func TestState_GetPerfStatsJSON(t *testing.T) {
s := NewState()
s.IncrementTCPSuccessPacketCount()
json := s.GetPerfStatsJSON()
if json == "" || json == "{}" {
t.Fatalf("GetPerfStatsJSON 返回空: %q", json)
}
if len(json) < 10 {
t.Fatalf("GetPerfStatsJSON 内容过短: %q", json)
}
// 验证包含关键字段
for _, key := range []string{"total_packets", "tcp_success", "success_rate"} {
if !containsStr(json, key) {
t.Errorf("GetPerfStatsJSON 缺少字段 %q", key)
}
}
}
func containsStr(s, sub string) bool {
return len(s) >= len(sub) && (s == sub || len(s) > 0 && stringContains(s, sub))
}
func stringContains(s, sub string) bool {
for i := 0; i <= len(s)-len(sub); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
// TestState_GetPacketLimiter 测试通用发包限速器
func TestState_GetPacketLimiter(t *testing.T) {
t.Run("零速率返回nil", func(t *testing.T) {
s := NewState()
limiter := s.GetPacketLimiter(0)
if limiter != nil {
t.Fatal("零速率应返回 nil limiter")
}
})
t.Run("负速率返回nil", func(t *testing.T) {
s := NewState()
limiter := s.GetPacketLimiter(-1)
if limiter != nil {
t.Fatal("负速率应返回 nil limiter")
}
})
t.Run("正速率初始化限速器", func(t *testing.T) {
s := NewState()
limiter := s.GetPacketLimiter(600) // 600/min = 10/s
if limiter == nil {
t.Fatal("正速率应返回非 nil limiter")
}
// 再次调用返回同一实例
limiter2 := s.GetPacketLimiter(1200)
if limiter != limiter2 {
t.Fatal("GetPacketLimiter 应通过 sync.Once 复用实例")
}
})
t.Run("低速率被钳位到1pps", func(t *testing.T) {
s := NewState()
// 1/min < 1/s,应被钳位
limiter := s.GetPacketLimiter(1)
if limiter == nil {
t.Fatal("低速率钳位后应返回非 nil limiter")
}
})
}
// TestState_CacheService 测试服务识别缓存
func TestState_CacheService(t *testing.T) {
s := NewState()
// 未缓存时查询返回 false
_, ok := s.GetCachedService("192.168.1.1:80")
if ok {
t.Fatal("未缓存的 key 不应返回 ok=true")
}
// 缓存并查询
type fakeInfo struct{ Name string }
info := &fakeInfo{Name: "http"}
s.CacheService("192.168.1.1:80", info)
got, ok := s.GetCachedService("192.168.1.1:80")
if !ok {
t.Fatal("已缓存的 key 应返回 ok=true")
}
if got != info {
t.Fatalf("GetCachedService 返回 %v, 期望 %v", got, info)
}
// 不同 key 互不干扰
_, ok = s.GetCachedService("192.168.1.1:443")
if ok {
t.Fatal("不同 key 不应命中缓存")
}
}
// =============================================================================
// CheckAndIncrementPacketRate 测试
// =============================================================================
// TestCheckAndIncrementPacketRate_ZeroLimit 速率为 0 时无限制
func TestCheckAndIncrementPacketRate_ZeroLimit(t *testing.T) {
s := NewState()
for i := 0; i < 1000; i++ {
ok, err := s.CheckAndIncrementPacketRate(0)
if !ok || err != nil {
t.Fatalf("零速率限制应始终允许: ok=%v err=%v", ok, err)
}
}
}
// TestCheckAndIncrementPacketRate_NegativeLimit 负速率等同于无限制
func TestCheckAndIncrementPacketRate_NegativeLimit(t *testing.T) {
s := NewState()
ok, err := s.CheckAndIncrementPacketRate(-1)
if !ok || err != nil {
t.Fatalf("负速率应允许: ok=%v err=%v", ok, err)
}
}
// TestCheckAndIncrementPacketRate_AllowsWhenTokensAvailable 有令牌时返回 true
func TestCheckAndIncrementPacketRate_AllowsWhenTokensAvailable(t *testing.T) {
s := NewState()
// 600/min = 10/s,桶容量 20,初始满桶
ok, err := s.CheckAndIncrementPacketRate(600)
if !ok || err != nil {
t.Fatalf("初始应有令牌: ok=%v err=%v", ok, err)
}
}
// TestCheckAndIncrementPacketRate_RateLimitedAfterExhaustion 耗尽令牌后返回 false 和 PacketLimitError
func TestCheckAndIncrementPacketRate_RateLimitedAfterExhaustion(t *testing.T) {
s := NewState()
// 极低速率:1/min,桶容量为 1(钳位后 packetsPerSecond=1capacity=2
// 消耗掉所有令牌后应被限速
const limit int64 = 1
// 初始化限速器(第一次调用触发 sync.Once)
s.GetPacketLimiter(limit)
// 消耗完所有令牌(容量 <= 2)
for i := 0; i < 10; i++ {
s.CheckAndIncrementPacketRate(limit) //nolint: errcheck
}
// 此时令牌应已耗尽,下一次调用应被限速
ok, err := s.CheckAndIncrementPacketRate(limit)
if ok {
// 桶可能还剩令牌(容量 2),多耗几次再判断
for i := 0; i < 20; i++ {
ok, err = s.CheckAndIncrementPacketRate(limit)
if !ok {
break
}
}
}
if ok {
t.Fatal("令牌耗尽后应返回 ok=false")
}
if err == nil {
t.Fatal("令牌耗尽后应返回 error")
}
if !isPacketLimitError(err) {
t.Errorf("error 类型应为 PacketLimitError, 实际 %T: %v", err, err)
}
}
// isPacketLimitError 检查是否为 PacketLimitError
func isPacketLimitError(err error) bool {
_, ok := err.(*PacketLimitError)
return ok
}
// TestCheckAndIncrementPacketRate_ErrorUnwrapsToSentinel 验证 error 可 unwrap 到 sentinel
func TestCheckAndIncrementPacketRate_ErrorUnwrapsToSentinel(t *testing.T) {
s := NewState()
const limit int64 = 1
// 耗尽令牌
for i := 0; i < 50; i++ {
s.CheckAndIncrementPacketRate(limit) //nolint: errcheck
}
var lastErr error
for i := 0; i < 10; i++ {
ok, err := s.CheckAndIncrementPacketRate(limit)
if !ok {
lastErr = err
break
}
}
if lastErr == nil {
t.Skip("未能触发限速(可能令牌桶容量较大),跳过 unwrap 测试")
}
// 验证可 unwrap 到 ErrPacketRateLimited
pErr, ok := lastErr.(*PacketLimitError)
if !ok {
t.Fatalf("期望 *PacketLimitError, 实际 %T", lastErr)
}
if pErr.Sentinel != ErrPacketRateLimited {
t.Errorf("Sentinel = %v, 期望 ErrPacketRateLimited", pErr.Sentinel)
}
if pErr.Limit != limit {
t.Errorf("Limit = %d, 期望 %d", pErr.Limit, limit)
}
}
// TestState_OutputMutex 测试输出互斥锁
func TestState_OutputMutex(t *testing.T) {
s := NewState()
+250 -85
View File
@@ -1,147 +1,312 @@
package core
import (
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/panjf2000/ants/v2"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// AdaptivePool 自适应线程池
// 封装 ants.PoolWithFunc,支持根据资源耗尽率动态调整线程数
// HealthSignal 健康评估结果
type HealthSignal int
const (
HealthUnknown HealthSignal = iota // 样本不足,无法判断
HealthGood // 一切正常,可以提速
HealthOK // 正常,维持现状
HealthStressed // 有压力信号,轻微降速
HealthCongested // 明确拥塞,大幅降速
)
// AdaptivePool 自适应线程池(AIMD + 慢启动)
//
// 三阶段工作模式:
// 1. 慢启动:从 target/4 起步,每个检查周期翻倍,直到达到 target 或检测到拥塞
// 2. 稳态 AIMD:健康时加性增(+5% target),拥塞时乘性减(×0.5)
// 3. 恢复上限受 ceiling 约束,不会无限增长
//
// 健康评估基于两个信号:
// - 资源耗尽率(fd/端口不足)
// - RTT 趋势(fast EMA / slow EMA
type AdaptivePool struct {
pool *ants.PoolWithFunc
state *common.State
pool *ants.PoolWithFunc
metrics *ScanMetrics
initialSize int
minSize int
maxSize int
currentSize int32 // 原子操作
// 网络环境(影响健康评估阈值)
networkEnv NetworkEnv
// 监控参数
checkInterval time.Duration
lastCheck time.Time
lastExhaustedCount int64
lastPacketCount int64
// 并发控制
target int32 // 探测推荐的目标值
ceiling int32 // 绝对上限(用户指定或探测推荐)
currentSize int32
// 阈值
exhaustedThreshold float64 // 资源耗尽率阈值(触发降级)
recoveryThreshold float64 // 恢复阈值(允许升级
// 慢启动
inSlowStart bool
ssThreshold int32 // 慢启动阈值(拥塞后降为当前值
mu sync.Mutex
// 检查定时
checkInterval time.Duration
lastCheck atomic.Int64 // UnixNano
// 增量计算
mu sync.Mutex
prevSnapshot MetricsSnapshot
}
// NewAdaptivePool 创建自适应线程池
func NewAdaptivePool(size int, fn func(interface{}), state *common.State) (*AdaptivePool, error) {
// 移除 WithPreAlloc(true),在大规模扫描时预分配可能导致内存问题
pool, err := ants.NewPoolWithFunc(size, fn)
// target: 目标并发数(来自 NetworkProfile.RecommendConcurrency
// ceiling: 最大并发上限
// metrics: 共享的扫描度量(scanSinglePort 写入,pool 读取)
func NewAdaptivePool(target, ceiling int, fn func(interface{}), metrics *ScanMetrics, env ...NetworkEnv) (*AdaptivePool, error) {
// 慢启动初始值:target 的 25%,但不低于 10
initial := target / 4
if initial < 10 {
initial = 10
}
if initial > target {
initial = target
}
pool, err := ants.NewPoolWithFunc(initial, fn)
if err != nil {
return nil, err
}
minSize := size / 4
if minSize < 10 {
minSize = 10
netEnv := EnvWAN
if len(env) > 0 {
netEnv = env[0]
}
return &AdaptivePool{
pool: pool,
state: state,
initialSize: size,
minSize: minSize,
maxSize: size,
currentSize: int32(size),
checkInterval: time.Second,
exhaustedThreshold: 0.10, // 10% 资源耗尽率触发降级
recoveryThreshold: 0.02, // 2% 以下允许恢复
pool: pool,
metrics: metrics,
networkEnv: netEnv,
target: int32(target),
ceiling: int32(ceiling),
currentSize: int32(initial),
inSlowStart: true,
ssThreshold: int32(target),
checkInterval: 500 * time.Millisecond,
}, nil
}
// Invoke 提交任务,并在适当时机检查是否需要调整线程数
// Invoke 提交任务
func (ap *AdaptivePool) Invoke(task interface{}) error {
ap.maybeAdjust()
return ap.pool.Invoke(task)
}
// maybeAdjust 检查并可能调整线程池大小
// maybeAdjust 周期性检查并调整并发数
func (ap *AdaptivePool) maybeAdjust() {
now := time.Now()
ap.mu.Lock()
if now.Sub(ap.lastCheck) < ap.checkInterval {
ap.mu.Unlock()
last := ap.lastCheck.Load()
now := time.Now().UnixNano()
if now-last < int64(ap.checkInterval) {
return
}
ap.lastCheck = now
// 获取当前计数
currentExhausted := ap.state.GetResourceExhaustedCount()
currentPackets := ap.state.GetPacketCount()
// 计算增量(本周期内的耗尽率)
deltaExhausted := currentExhausted - ap.lastExhaustedCount
deltaPackets := currentPackets - ap.lastPacketCount
ap.lastExhaustedCount = currentExhausted
ap.lastPacketCount = currentPackets
ap.mu.Unlock()
// 需要足够的样本才能判断
if deltaPackets < 100 {
if !ap.lastCheck.CompareAndSwap(last, now) {
return
}
rate := float64(deltaExhausted) / float64(deltaPackets)
currentSize := int(atomic.LoadInt32(&ap.currentSize))
ap.adjust()
}
if rate > ap.exhaustedThreshold && currentSize > ap.minSize {
// 降级:减少 20% 线程
newSize := int(float64(currentSize) * 0.8)
if newSize < ap.minSize {
newSize = ap.minSize
}
func (ap *AdaptivePool) adjust() {
health := ap.assessHealth()
if health == HealthUnknown {
return
}
// RTT 漂移微调:fast EMA 远高于 slow EMA 说明延迟持续恶化
// 压低 target 让 AIMD 的天花板跟着降,而不是只靠乘性减
ap.maybeReduceTarget()
current := int(atomic.LoadInt32(&ap.currentSize))
target := int(atomic.LoadInt32(&ap.target))
ceiling := int(atomic.LoadInt32(&ap.ceiling))
var newSize int
if ap.inSlowStart {
newSize = ap.adjustSlowStart(health, current, target)
} else {
newSize = ap.adjustAIMD(health, current, target)
}
// 下限:ceiling 的 5%,但不低于 10
minSize := ceiling / 20
if minSize < 10 {
minSize = 10
}
if newSize < minSize {
newSize = minSize
}
if newSize > ceiling {
newSize = ceiling
}
if newSize != current {
ap.tune(newSize)
common.LogInfo(fmt.Sprintf("[AdaptivePool] 资源耗尽率 %.1f%%, 线程数 %d -> %d", rate*100, currentSize, newSize))
} else if rate < ap.recoveryThreshold && currentSize < ap.maxSize {
// 恢复:增加 10% 线程(保守恢复)
newSize := int(float64(currentSize) * 1.1)
if newSize > ap.maxSize {
newSize = ap.maxSize
// 显著变化时记录日志
delta := newSize - current
if delta < 0 {
delta = -delta
}
if newSize > currentSize {
ap.tune(newSize)
if delta > current/5 {
if newSize < current {
common.LogInfo(i18n.Tr("adaptive_pool_decrease", current, newSize))
} else {
common.LogDebug(i18n.Tr("adaptive_pool_increase", current, newSize))
}
}
}
}
// tune 调整线程池大小
func (ap *AdaptivePool) adjustSlowStart(health HealthSignal, current, target int) int {
switch health {
case HealthCongested, HealthStressed:
// 退出慢启动,设置阈值
ap.ssThreshold = int32(current)
ap.inSlowStart = false
common.LogDebug(i18n.Tr("adaptive_pool_slowstart_exit", current))
return int(float64(current) * 0.5)
default:
// 翻倍
newSize := current * 2
if newSize >= target {
newSize = target
ap.inSlowStart = false
}
return newSize
}
}
func (ap *AdaptivePool) adjustAIMD(health HealthSignal, current, target int) int {
switch health {
case HealthCongested:
// 乘性减:×0.5
newSize := int(float64(current) * 0.5)
ap.ssThreshold = int32(newSize)
return newSize
case HealthStressed:
// 温和降低:×0.85
return int(float64(current) * 0.85)
case HealthGood:
// 加性增:+5% of target,至少 +1
inc := target / 20
if inc < 1 {
inc = 1
}
return current + inc
default:
return current
}
}
// assessHealth 综合健康评估
func (ap *AdaptivePool) assessHealth() HealthSignal {
snap := ap.metrics.Snapshot()
ap.mu.Lock()
prev := ap.prevSnapshot
ap.prevSnapshot = snap
ap.mu.Unlock()
// 计算本周期增量
deltaTotal := snap.Total() - prev.Total()
deltaExhausted := snap.Exhausted - prev.Exhausted
// 样本不足
if deltaTotal < 30 {
return HealthUnknown
}
exhaustRate := float64(deltaExhausted) / float64(deltaTotal)
rttRatio := ap.metrics.RTTRatio()
// 阈值根据网络环境调整:内网收紧,公网放宽
var congestExhaust, stressExhaust, congestRTT, stressRTT, goodRTT float64
switch ap.networkEnv {
case EnvLAN:
congestExhaust, stressExhaust = 0.08, 0.03
congestRTT, stressRTT, goodRTT = 1.8, 1.4, 1.15
case EnvWAN:
congestExhaust, stressExhaust = 0.15, 0.05
congestRTT, stressRTT, goodRTT = 2.5, 1.8, 1.3
default: // Internet / Slow
congestExhaust, stressExhaust = 0.25, 0.10
congestRTT, stressRTT, goodRTT = 3.5, 2.5, 1.5
}
switch {
case exhaustRate > congestExhaust:
return HealthCongested
case rttRatio > congestRTT:
return HealthCongested
case exhaustRate > stressExhaust:
return HealthStressed
case rttRatio > stressRTT:
return HealthStressed
case exhaustRate < 0.01 && rttRatio < goodRTT:
return HealthGood
default:
return HealthOK
}
}
// maybeReduceTarget 当 RTT 持续恶化时压低 target
// 不低于 ceiling 的 20%,避免过度收缩
func (ap *AdaptivePool) maybeReduceTarget() {
rttRatio := ap.metrics.RTTRatio()
if rttRatio <= 3.0 {
return
}
target := atomic.LoadInt32(&ap.target)
ceiling := atomic.LoadInt32(&ap.ceiling)
minTarget := ceiling / 5
if minTarget < 10 {
minTarget = 10
}
// 压低 10%
newTarget := int32(float64(target) * 0.9)
if newTarget < minTarget {
newTarget = minTarget
}
if newTarget < target {
atomic.StoreInt32(&ap.target, newTarget)
}
}
func (ap *AdaptivePool) tune(newSize int) {
ap.pool.Tune(newSize)
atomic.StoreInt32(&ap.currentSize, int32(newSize))
}
// Running 返回当前运行中的 goroutine 数量
func (ap *AdaptivePool) Running() int {
return ap.pool.Running()
}
func (ap *AdaptivePool) Running() int { return ap.pool.Running() }
// Cap 返回当前池容量
func (ap *AdaptivePool) Cap() int {
return int(atomic.LoadInt32(&ap.currentSize))
}
func (ap *AdaptivePool) Cap() int { return int(atomic.LoadInt32(&ap.currentSize)) }
// Release 释放线程池
func (ap *AdaptivePool) Release() {
ap.pool.Release()
}
func (ap *AdaptivePool) Release() { ap.pool.Release() }
// Wait 等待所有任务完成
// Wait 等待所有任务完成(最多等待 10 分钟)
func (ap *AdaptivePool) Wait() {
// ants 没有原生 Wait,通过 Running() == 0 轮询
deadline := time.After(10 * time.Minute)
for ap.pool.Running() > 0 {
time.Sleep(10 * time.Millisecond)
select {
case <-deadline:
common.LogError(i18n.Tr("adaptive_pool_wait_timeout"))
return
default:
time.Sleep(10 * time.Millisecond)
}
}
}
+183 -147
View File
@@ -1,160 +1,102 @@
package core
/*
adaptive_pool_test.go - AdaptivePool 高价值测试
测试重点
1. 并发安全 - 多goroutine同时调整不崩溃
2. 降级逻辑 - 资源耗尽率高时正确减少线程
3. 恢复逻辑 - 资源耗尽率低时正确增加线程
4. 边界条件 - 不超过minSize/maxSize
不测试
- 简单的getter方法太简单不值得
- ants库本身的正确性库作者负责
*/
import (
"sync/atomic"
"testing"
"time"
"github.com/shadow1ng/fscan/common"
)
// =============================================================================
// 场景1:降级逻辑测试(高价值)
// =============================================================================
// TestAdaptivePool_DowngradeOnHighExhaustion 验证资源耗尽率高时降低线程数
// 这是个核心业务逻辑:耗尽率 > 10% 时应该减少线程
func TestAdaptivePool_DowngradeOnHighExhaustion(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(100, func(interface{}) {}, state)
// newTestPool 测试辅助:创建测试用的自适应线程池
func newTestPool(t *testing.T, size int, fn func(interface{})) (*AdaptivePool, *ScanMetrics) {
t.Helper()
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(size, size, fn, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
return pool, metrics
}
// TestAdaptivePool_DowngradeOnHighExhaustion 验证资源耗尽率高时降低线程数
func TestAdaptivePool_DowngradeOnHighExhaustion(t *testing.T) {
pool, metrics := newTestPool(t, 100, func(interface{}) {})
defer pool.Release()
// 慢启动先跑到 target
pool.inSlowStart = false
pool.tune(100)
initialCap := pool.Cap()
// 模拟高资源耗尽率:20% 的包都失败了
// 需要至少100个样本才会触发调整
// 模拟高资源耗尽率:20%
for i := 0; i < 200; i++ {
state.IncrementPacketCount()
if i < 40 { // 前40个失败(20%
state.IncrementResourceExhaustedCount()
if i < 40 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
// 触发调整:提交足够多的任务让maybeAdjust被调用
// 触发调整
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
time.Sleep(time.Millisecond * 10) // 等待异步调整
time.Sleep(time.Millisecond * 30)
}
// 等待调整完成
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:线程数应该减少
if finalCap >= initialCap {
t.Errorf("应该降级: 初始 %d, 最终 %d", initialCap, finalCap)
}
// 验证:不应该降到minSize以下
minSize := initialCap / 4
if minSize < 10 {
minSize = 10
}
if finalCap < minSize {
t.Errorf("降到minSize以下: %d < %d", finalCap, minSize)
if finalCap < 10 {
t.Errorf("降到 minSize 以下: %d", finalCap)
}
t.Logf("降级成功: %d -> %d (min=%d)", initialCap, finalCap, minSize)
t.Logf("降级成功: %d -> %d", initialCap, finalCap)
}
// =============================================================================
// 场景3:恢复逻辑测试(高价值)
// =============================================================================
// TestAdaptivePool_NoRecoveryOnLowExhaustion 验证低耗尽率时不升级
// 防止线程数盲目增长
func TestAdaptivePool_NoRecoveryOnLowExhaustion(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(50, func(interface{}) {}, state)
// TestAdaptivePool_SlowStart 验证慢启动行为
func TestAdaptivePool_SlowStart(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
// 先降到minSize
for i := 0; i < 500; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount() // 100% 耗尽
// 初始应该是 target/4 = 25
initialCap := pool.Cap()
if initialCap > 30 {
t.Errorf("慢启动初始值应该 <= 30, got %d", initialCap)
}
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
reducedCap := pool.Cap()
// 现在模拟低耗尽率:只有1%失败
for i := 0; i < 500; i++ {
state.IncrementPacketCount()
if i%100 == 0 { // 只有5个失败(1%
state.IncrementResourceExhaustedCount()
}
if !pool.inSlowStart {
t.Error("应该处于慢启动状态")
}
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:即使耗尽率低,也不应该立即恢复(保守策略)
// 或者即使恢复,也很有限
if finalCap > reducedCap+5 {
t.Logf("恢复行为: %d -> %d", reducedCap, finalCap)
}
t.Logf("慢启动初始: cap=%d, inSlowStart=%v", initialCap, pool.inSlowStart)
}
// =============================================================================
// 场景4:边界条件测试(中价值)
// =============================================================================
// TestAdaptivePool_MinSizeBoundary 验证不会降到minSize以下
// TestAdaptivePool_MinSizeBoundary 验证不会降到 minSize 以下
func TestAdaptivePool_MinSizeBoundary(t *testing.T) {
state := common.NewState()
// 创建小线程池,minSize会是10
pool, err := NewAdaptivePool(40, func(interface{}) {}, state)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
pool, metrics := newTestPool(t, 40, func(interface{}) {})
defer pool.Release()
// 模拟极端的资源耗尽:100%失败
for i := 0; i < 1000; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount()
pool.inSlowStart = false
pool.tune(40)
// 极端耗尽
for i := 0; i < 500; i++ {
metrics.RecordExhausted()
}
// 触发多次调整
for i := 0; i < 50; i++ {
_ = pool.Invoke(nil)
time.Sleep(time.Millisecond)
time.Sleep(time.Millisecond * 15)
}
finalCap := pool.Cap()
// 验证:不应该低于10
if finalCap < 10 {
t.Errorf("线程数 < 10: %d", finalCap)
}
@@ -162,76 +104,170 @@ func TestAdaptivePool_MinSizeBoundary(t *testing.T) {
t.Logf("最小边界测试通过: cap=%d", finalCap)
}
// =============================================================================
// 场景5:样本不足测试(低价值但重要)
// =============================================================================
// TestAdaptivePool_NotEnoughSamples 验证样本不足时不调整
// 防止基于小样本做错误决策
func TestAdaptivePool_NotEnoughSamples(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(100, func(interface{}) {}, state)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
pool, metrics := newTestPool(t, 100, func(interface{}) {})
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
initialCap := pool.Cap()
// 只增加少量样本(<100),不足以触发调整
for i := 0; i < 50; i++ {
state.IncrementPacketCount()
state.IncrementResourceExhaustedCount() // 即使100%失败也不调整
// 只 20 个样本,不足 30 的阈值
for i := 0; i < 20; i++ {
metrics.RecordExhausted()
}
// 提交任务
for i := 0; i < 10; i++ {
_ = pool.Invoke(nil)
}
time.Sleep(time.Millisecond * 50)
finalCap := pool.Cap()
// 验证:样本不足时不应该调整
if finalCap != initialCap {
t.Errorf("样本不足时不应该调整: %d -> %d", initialCap, finalCap)
}
}
// =============================================================================
// 辅助函数
// =============================================================================
// TestAdaptivePool_Wait 验证Wait方法正确等待所有任务完成
// TestAdaptivePool_Wait 验证 Wait 方法
func TestAdaptivePool_Wait(t *testing.T) {
state := common.NewState()
pool, err := NewAdaptivePool(10, func(interface{}) {
pool, _ := newTestPool(t, 10, func(interface{}) {
time.Sleep(time.Millisecond * 50)
}, state)
})
defer pool.Release()
pool.inSlowStart = false
pool.tune(10)
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
start := time.Now()
pool.Wait()
duration := time.Since(start)
if duration > 300*time.Millisecond {
t.Errorf("Wait 耗时过长: %v", duration)
}
t.Logf("Wait 测试通过: %v", duration)
}
// =============================================================================
// maybeReduceTarget 补充覆盖
// =============================================================================
// TestMaybeReduceTarget_NoOpWhenRTTLow rttRatio <= 3.0 时不修改 target
func TestMaybeReduceTarget_NoOpWhenRTTLow(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
// 提交任务
for i := 0; i < 20; i++ {
_ = pool.Invoke(nil)
}
initialTarget := atomic.LoadInt32(&pool.target)
// Wait应该在所有任务完成后返回
start := time.Now()
pool.Wait()
duration := time.Since(start)
// RTTRatio 样本不足(< 20)返回 1.0,远低于 3.0 阈值
pool.maybeReduceTarget()
// 20个任务,每个50ms,10个线程,应该约100ms完成
if duration < 80*time.Millisecond {
t.Logf("Wait提前返回?可能测试有问题: %v", duration)
afterTarget := atomic.LoadInt32(&pool.target)
if afterTarget != initialTarget {
t.Errorf("rttRatio <= 3.0 时 target 不应改变: %d -> %d", initialTarget, afterTarget)
}
}
// TestMaybeReduceTarget_ReducesWhenRTTHigh rttRatio > 3.0 时压低 target 10%
func TestMaybeReduceTarget_ReducesWhenRTTHigh(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 200, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
// 伪造 RTT:让 fastEMA >> slowEMAratio > 3.0
// 方法:先用大 RTT 建立 fastEMA,再用小 RTT 建立 slowEMA
// 更直接:直接操作 atomic 字段(包内测试可以访问)
for i := 0; i < 25; i++ {
metrics.RecordConnect(10 * time.Millisecond) // 先建 baseline
}
// 现在把 fastEMA 人为拉高(写入一个远大于 slowEMA 的值)
pool.metrics.rttFastNs.Store(int64(400 * time.Millisecond))
pool.metrics.rttSlowNs.Store(int64(10 * time.Millisecond))
initialTarget := atomic.LoadInt32(&pool.target)
pool.maybeReduceTarget()
afterTarget := atomic.LoadInt32(&pool.target)
if afterTarget >= initialTarget {
t.Errorf("rttRatio > 3.0 时 target 应被压低: %d -> %d", initialTarget, afterTarget)
}
// 验证是 ×0.9
expected := int32(float64(initialTarget) * 0.9)
if afterTarget != expected {
t.Errorf("target 应为 %d (×0.9), 实际 %d", expected, afterTarget)
}
}
// TestMaybeReduceTarget_ClampToMinTarget target 压低后不低于 ceiling/5 或 10
func TestMaybeReduceTarget_ClampToMinTarget(t *testing.T) {
metrics := &ScanMetrics{}
// ceiling=20, minTarget = max(20/5, 10) = 10
// target=10, newTarget = int(10*0.9) = 9 → 被 clamp 到 10 → newTarget == target → 不更新
pool, err := NewAdaptivePool(10, 20, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
// 强制设置 target=10(初始值就是 10,但确认一下)
atomic.StoreInt32(&pool.target, 10)
// 伪造 rttRatio > 3.0
for i := 0; i < 25; i++ {
metrics.RecordConnect(10 * time.Millisecond)
}
pool.metrics.rttFastNs.Store(int64(400 * time.Millisecond))
pool.metrics.rttSlowNs.Store(int64(10 * time.Millisecond))
pool.maybeReduceTarget()
afterTarget := atomic.LoadInt32(&pool.target)
// newTarget=9 < minTarget=10 → clamp 到 10 → 10 == target → 不写入
if afterTarget != 10 {
t.Errorf("clamp 后 target 应保持 10, 实际 %d", afterTarget)
}
}
// TestMaybeReduceTarget_LargeCeilingMinTarget ceiling 足够大时 minTarget = ceiling/5
func TestMaybeReduceTarget_LargeCeilingMinTarget(t *testing.T) {
metrics := &ScanMetrics{}
// ceiling=100, minTarget = 100/5 = 20
// target=21 → newTarget = int(21*0.9) = 18 → clamp 到 20
pool, err := NewAdaptivePool(21, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建线程池失败: %v", err)
}
defer pool.Release()
atomic.StoreInt32(&pool.target, 21)
atomic.StoreInt32(&pool.ceiling, 100)
for i := 0; i < 25; i++ {
metrics.RecordConnect(10 * time.Millisecond)
}
pool.metrics.rttFastNs.Store(int64(400 * time.Millisecond))
pool.metrics.rttSlowNs.Store(int64(10 * time.Millisecond))
pool.maybeReduceTarget()
afterTarget := atomic.LoadInt32(&pool.target)
// newTarget=18 < minTarget=20 → store 20; 20 < 21 → 更新
if afterTarget != 20 {
t.Errorf("应 clamp 到 minTarget=20, 实际 %d", afterTarget)
}
if duration > 200*time.Millisecond {
t.Errorf("Wait耗时过长: %v", duration)
}
t.Logf("Wait测试通过: %v", duration)
}
+117
View File
@@ -0,0 +1,117 @@
package core
import (
"math"
"sync"
"time"
)
// AdaptiveTimeout 基于 RTT 采样的自适应超时计算器
// 算法:timeout = mean(RTT) + 4 * stddev(RTT)clamp 到 [min, max]
// 冷启动阶段(样本不足)返回用户配置的固定超时
type AdaptiveTimeout struct {
mu sync.Mutex
samples []float64 // 环形缓冲区,单位 ms
pos int // 写入位置
count int // 已采集总数
size int // 缓冲区容量
minTO time.Duration
maxTO time.Duration
warmup int // 冷启动所需最小样本数
cachedTO time.Duration
dirty bool
}
// NewAdaptiveTimeout 创建自适应超时计算器
// maxTimeout: 用户配置的超时上限(即原始固定超时)
func NewAdaptiveTimeout(maxTimeout time.Duration) *AdaptiveTimeout {
// minTO: 自适应超时下限,取 max(500ms, maxTimeout/5)
// 依据:高并发下 TCP 握手存在尾延迟(OS 调度抖动、backlog 溢出、端口竞争),
// 过低的下限会导致开放端口被误判为关闭(issue #503)
minTO := maxTimeout / 5
if minTO < 500*time.Millisecond {
minTO = 500 * time.Millisecond
}
return &AdaptiveTimeout{
samples: make([]float64, 64),
size: 64,
minTO: minTO,
maxTO: maxTimeout,
warmup: 10,
}
}
// Record 记录一次成功连接的 RTT
func (a *AdaptiveTimeout) Record(rtt time.Duration) {
a.mu.Lock()
a.samples[a.pos%a.size] = float64(rtt.Milliseconds())
a.pos++
a.count++
a.dirty = true
a.mu.Unlock()
}
// Timeout 获取当前推荐超时值
// 样本不足时返回 maxTO(冷启动)
// 锁外执行均值/标准差计算,减少锁持有时间
func (a *AdaptiveTimeout) Timeout() time.Duration {
a.mu.Lock()
if a.count < a.warmup {
a.mu.Unlock()
return a.maxTO
}
if !a.dirty {
cached := a.cachedTO
a.mu.Unlock()
return cached
}
n := a.size
if a.count < a.size {
n = a.count
}
// 拷贝样本到本地,释放锁后再计算
localSamples := make([]float64, n)
start := a.pos % a.size
if a.count < a.size {
copy(localSamples, a.samples[:n])
} else {
copy(localSamples[:a.size-start], a.samples[start:])
copy(localSamples[a.size-start:], a.samples[:start])
}
a.mu.Unlock()
// 锁外计算
var sum float64
for _, s := range localSamples {
sum += s
}
mean := sum / float64(n)
var variance float64
for _, s := range localSamples {
d := s - mean
variance += d * d
}
stddev := math.Sqrt(variance / float64(n))
ms := mean + 4*stddev
to := time.Duration(ms) * time.Millisecond
if to < a.minTO {
to = a.minTO
}
if to > a.maxTO {
to = a.maxTO
}
// 短暂加锁更新缓存
a.mu.Lock()
a.cachedTO = to
a.dirty = false
a.mu.Unlock()
return to
}
+42 -33
View File
@@ -1,6 +1,7 @@
package core
import (
"context"
"fmt"
"sync"
"time"
@@ -37,7 +38,7 @@ type AliveStats struct {
// NewAliveScanStrategy 创建新的存活探测扫描策略
func NewAliveScanStrategy() *AliveScanStrategy {
return &AliveScanStrategy{
BaseScanStrategy: NewBaseScanStrategy("存活探测", FilterNone),
BaseScanStrategy: NewBaseScanStrategy(i18n.GetText("scan_strategy_alive_name"), FilterNone),
startTime: time.Now(),
}
}
@@ -53,65 +54,73 @@ func (s *AliveScanStrategy) Description() string {
}
// Execute 执行存活探测扫描策略
func (s *AliveScanStrategy) Execute(config *common.Config, state *common.State, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
func (s *AliveScanStrategy) Execute(ctx context.Context, session *common.ScanSession, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
// 验证扫描目标(需要同时检查 -h 和 -hf 参数)
fv := common.GetFlagVars()
if info.Host == "" && fv.HostsFile == "" {
common.LogError(i18n.GetText("parse_error_target_empty"))
if info.Host == "" && session.Params.HostsFile == "" {
session.LogError(i18n.GetText("parse_error_target_empty"))
return
}
// 执行存活探测
s.performAliveScan(info, config, state)
// 输出统计信息
s.outputStats()
s.performAliveScan(ctx, info, session)
}
// performAliveScan 执行存活探测
func (s *AliveScanStrategy) performAliveScan(info common.HostInfo, config *common.Config, state *common.State) {
// 解析目标主机
fv := common.GetFlagVars()
hosts, err := parsers.ParseIP(info.Host, fv.HostsFile, fv.ExcludeHosts)
func (s *AliveScanStrategy) performAliveScan(ctx context.Context, info common.HostInfo, session *common.ScanSession) {
excludes, err := loadHostExcludes(session.Params)
if err != nil {
common.LogError(i18n.Tr("parse_target_failed", err))
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
if len(hosts) == 0 {
common.LogError(i18n.GetText("parse_error_no_hosts"))
iter, err := parsers.NewHostIterator(info.Host, session.Params.HostsFile, excludes...)
if err != nil {
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
defer func() {
_ = iter.Close()
}()
// 初始化统计信息
s.stats.TotalHosts = len(hosts)
s.stats.TotalHosts = 0
s.stats.AliveHosts = 0
s.stats.DeadHosts = 0
for {
hosts, err := iter.NextBatch(ctx, targetHostBatchSize(session.Config))
if err != nil {
if ctx.Err() != nil {
session.LogError(i18n.Tr("global_timeout_exceeded",
int(session.Config.GlobalTimeout.Seconds())))
return
}
session.LogError(i18n.Tr("parse_target_failed", err))
return
}
if len(hosts) == 0 {
break
}
// 执行存活检测
aliveList := CheckLive(hosts, false, config, state) // 使用ICMP探测
s.stats.TotalHosts += len(hosts)
aliveList := CheckLive(ctx, hosts, false, session)
s.stats.AliveHosts += len(aliveList)
for _, host := range aliveList {
session.LogSuccess(fmt.Sprintf("alive %s", host))
}
}
if s.stats.TotalHosts == 0 {
session.LogError(i18n.GetText("parse_error_no_hosts"))
return
}
// 更新统计信息
s.stats.AliveHosts = len(aliveList)
s.stats.DeadHosts = s.stats.TotalHosts - s.stats.AliveHosts
s.stats.ScanDuration = time.Since(s.startTime)
s.stats.AliveHostList = aliveList // 存储存活主机列表
if s.stats.TotalHosts > 0 {
s.stats.SuccessRate = float64(s.stats.AliveHosts) / float64(s.stats.TotalHosts) * 100
}
}
// outputStats 输出统计信息(精简版)
func (s *AliveScanStrategy) outputStats() {
// 只输出存活主机列表,不输出冗余统计
for _, host := range s.stats.AliveHostList {
common.LogSuccess(fmt.Sprintf("alive %s", host))
}
}
// PrepareTargets 存活探测不需要准备扫描目标
func (s *AliveScanStrategy) PrepareTargets(info common.HostInfo) []common.HostInfo {
// 存活探测不需要返回目标列表,因为它不进行后续扫描
+69 -47
View File
@@ -29,6 +29,7 @@ const (
type BaseScanStrategy struct {
strategyName string
filterType PluginFilterType
state *common.State
}
// NewBaseScanStrategy 创建基础扫描策略
@@ -39,6 +40,11 @@ func NewBaseScanStrategy(name string, filterType PluginFilterType) *BaseScanStra
}
}
// SetState 注入 session state(用于 per-session 服务缓存)
func (b *BaseScanStrategy) SetState(state *common.State) {
b.state = state
}
// GetPlugins 获取插件列表
func (b *BaseScanStrategy) GetPlugins(config *common.Config) ([]string, bool) {
scanMode := config.Mode
@@ -81,6 +87,16 @@ func (b *BaseScanStrategy) IsPluginApplicableByName(pluginName string, targetHos
return false
}
// 显式指定插件时,尊重调用方选择,不再强制使用插件默认端口过滤。
if isCustomMode {
return b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
// -full 模式下,web 插件对所有开放端口生效(跳过 IsMarkedWebService 检查)
if config.POC.Full && b.isWebPlugin(pluginName) {
return b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
// 检查端口匹配和过滤器类型
return b.isPluginApplicableToPortWithHost(pluginName, targetHost, targetPort) && b.isPluginPassesFilterType(pluginName, isCustomMode, config)
}
@@ -101,14 +117,19 @@ func (b *BaseScanStrategy) isLocalPlugin(pluginName string) bool {
return plugins.HasType(pluginName, plugins.PluginTypeLocal)
}
func (b *BaseScanStrategy) isUDPPlugin(pluginName string) bool {
return plugins.IsUDP(pluginName)
}
func (b *BaseScanStrategy) isLocalPluginExplicitlySpecified(pluginName string, config *common.Config) bool {
return config.LocalPlugin == pluginName
}
// isPluginApplicableToPortWithHost 检查插件是否适用于指定端口
// 匹配策略:端口匹配 → 服务名称匹配(解决非标准端口问题)
func (b *BaseScanStrategy) isPluginApplicableToPortWithHost(pluginName string, targetHost string, targetPort int) bool {
if b.isWebPlugin(pluginName) {
return IsMarkedWebService(targetHost, targetPort)
return IsMarkedWebServiceWithState(b.state, targetHost, targetPort)
}
pluginPorts := b.getPluginPorts(pluginName)
@@ -127,15 +148,33 @@ func (b *BaseScanStrategy) isPluginApplicableToPortWithHost(pluginName string, t
}
}
// 端口不匹配时,按指纹识别结果匹配
// 例:8881 端口上识别到 ssh 服务 → ssh 插件应该执行
if targetHost != "" && targetPort > 0 {
if info, ok := GetCachedServiceInfoWithState(b.state, targetHost, targetPort); ok && info != nil {
if strings.EqualFold(info.Name, pluginName) {
return true
}
}
}
return false
}
func (b *BaseScanStrategy) isPluginApplicableToPort(pluginName string, targetPort int) bool {
if b.isWebPlugin(pluginName) {
return true
}
return b.isPluginApplicableToPortWithHost(pluginName, "", targetPort)
}
// isPluginPassesFilterType 检查插件是否通过过滤器类型检查
func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomMode bool, config *common.Config) bool {
// UDP 插件有独立分发路径,不参与 TCP 端口匹配流水线
if b.isUDPPlugin(pluginName) {
return false
}
// 自定义模式下强制运行所有明确指定的插件
if isCustomMode {
return true
@@ -150,8 +189,8 @@ func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomM
}
return false
case FilterService:
// 服务扫描策略:排除本地插件
return !b.isLocalPlugin(pluginName)
// 服务扫描策略:排除本地插件和UDP插件(UDP有独立分发路径)
return !b.isLocalPlugin(pluginName) && !b.isUDPPlugin(pluginName)
case FilterWeb:
// Web扫描策略:只允许Web插件
return b.isWebPlugin(pluginName)
@@ -164,26 +203,9 @@ func (b *BaseScanStrategy) isPluginPassesFilterType(pluginName string, isCustomM
}
}
// LogPluginInfo 输出插件信息
func (b *BaseScanStrategy) LogPluginInfo(config *common.Config) {
allPlugins, isCustomMode := b.GetPlugins(config)
var prefix string
switch b.filterType {
case FilterLocal:
prefix = i18n.GetText("concurrency_local_plugin")
case FilterService:
prefix = i18n.GetText("concurrency_service_plugin")
case FilterWeb:
prefix = i18n.GetText("concurrency_web_plugin")
default:
prefix = i18n.GetText("concurrency_plugin")
}
// 插件信息不再输出,减少干扰
_ = allPlugins
_ = isCustomMode
_ = prefix
// LogPluginInfo 默认不输出插件信息(service 默认端口模式有意保持安静,减少干扰)。
// 子类 LocalScanStrategy / ServiceScanStrategy 按需重写。
func (b *BaseScanStrategy) LogPluginInfo(config *common.Config, session *common.ScanSession) {
}
// formatPluginList 格式化插件列表(超过5个时精简显示)
@@ -191,7 +213,7 @@ func formatPluginList(plugins []string) string {
if len(plugins) <= 5 {
return strings.Join(plugins, ", ")
}
return fmt.Sprintf("%s ... 等%d个", strings.Join(plugins[:5], ", "), len(plugins))
return i18n.Tr("plugin_list_summary", strings.Join(plugins[:5], ", "), len(plugins))
}
// ValidateConfiguration 验证扫描配置
@@ -200,14 +222,14 @@ func (b *BaseScanStrategy) ValidateConfiguration() error {
}
// LogScanStart 输出扫描开始信息(已精简,仅在非服务扫描模式下显示)
func (b *BaseScanStrategy) LogScanStart() {
func (b *BaseScanStrategy) LogScanStart(session *common.ScanSession) {
// 服务扫描模式下不显示(插件信息已足够说明)
// 仅在本地/Web等特殊模式下显示
switch b.filterType {
case FilterLocal:
common.LogInfo(i18n.GetText("start_local_scan"))
session.LogInfo(i18n.GetText("start_local_scan"))
case FilterWeb:
common.LogInfo(i18n.GetText("start_web_scan"))
session.LogInfo(i18n.GetText("start_web_scan"))
}
}
@@ -225,9 +247,9 @@ func (b *BaseScanStrategy) getPluginsByFilterType() []string {
}
}
case FilterService:
// 服务扫描策略:排除本地插件和纯Web插件,保留服务插件
// 服务扫描策略:排除本地插件和UDP插件,保留TCP服务插件
for _, pluginName := range allPlugins {
if !b.isLocalPlugin(pluginName) {
if !b.isLocalPlugin(pluginName) && !b.isUDPPlugin(pluginName) {
filteredPlugins = append(filteredPlugins, pluginName)
}
}
@@ -238,32 +260,32 @@ func (b *BaseScanStrategy) getPluginsByFilterType() []string {
filteredPlugins = append(filteredPlugins, pluginName)
}
}
// 确保 webtitle 在 webpoc 之前执行,避免指纹识别竞态
sort.Slice(filteredPlugins, func(i, j int) bool {
// webtitle 必须在 webpoc 之前
if filteredPlugins[i] == "webtitle" {
return true
}
if filteredPlugins[j] == "webtitle" {
return false
}
if filteredPlugins[i] == "webpoc" {
return false
}
if filteredPlugins[j] == "webpoc" {
return true
}
// 其他插件保持字母顺序
return filteredPlugins[i] < filteredPlugins[j]
})
default:
// 无过滤器:返回所有插件
filteredPlugins = allPlugins
}
orderWebPlugins(filteredPlugins)
return filteredPlugins
}
func orderWebPlugins(pluginNames []string) {
sort.SliceStable(pluginNames, func(i, j int) bool {
return webPluginOrder(pluginNames[i]) < webPluginOrder(pluginNames[j])
})
}
func webPluginOrder(pluginName string) int {
switch pluginName {
case "webtitle":
return 0
case "webpoc":
return 2
default:
return 1
}
}
// parsePluginList 解析插件列表字符串
func parsePluginList(pluginStr string) []string {
if pluginStr == "" {
+241
View File
@@ -2,6 +2,9 @@ package core
import (
"testing"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/plugins"
)
// =============================================================================
@@ -261,6 +264,113 @@ func slicesEqual(a, b []string) bool {
return true
}
func TestOrderWebPlugins(t *testing.T) {
plugins := []string{"ssh", "webpoc", "redis", "webtitle", "mysql"}
orderWebPlugins(plugins)
expected := []string{"webtitle", "ssh", "redis", "mysql", "webpoc"}
if !slicesEqual(plugins, expected) {
t.Fatalf("orderWebPlugins = %#v, want %#v", plugins, expected)
}
}
func TestBaseScanStrategyPluginSelectionAndApplicability(t *testing.T) {
registerTestPlugins(t)
plugins.RegisterWithOptions("core_test_local", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeLocal}, false)
plugins.RegisterWithOptions("core_test_udp", func() plugins.Plugin { return nil }, []int{161}, []string{plugins.PluginTypeUDP}, true)
clearServiceCache()
cfg := common.NewConfig()
cfg.Mode = "ssh, missing_plugin, webtitle"
strategy := NewBaseScanStrategy("service", FilterService)
got, custom := strategy.GetPlugins(cfg)
if !custom {
t.Fatal("explicit mode should be marked as custom")
}
if !slicesEqual(got, []string{"ssh", "webtitle"}) {
t.Fatalf("custom plugins = %#v, want ssh/webtitle", got)
}
cfg.Mode = "all"
servicePlugins, custom := strategy.GetPlugins(cfg)
if custom {
t.Fatal("all mode should not be custom")
}
if !containsString(servicePlugins, "ssh") || containsString(servicePlugins, "core_test_local") || containsString(servicePlugins, "core_test_udp") {
t.Fatalf("service filtered plugins = %#v", servicePlugins)
}
if !strategy.pluginExists("ssh") || strategy.pluginExists("missing_plugin") {
t.Fatal("pluginExists returned wrong result")
}
if !strategy.isPluginApplicableToPort("ssh", 22) || strategy.isPluginApplicableToPort("ssh", 23) {
t.Fatal("port applicability for ssh is wrong")
}
CacheServiceInfo("10.0.0.9", 22222, &ServiceInfo{Name: "ssh"})
if !strategy.isPluginApplicableToPortWithHost("ssh", "10.0.0.9", 22222) {
t.Fatal("service cache should allow ssh on a non-standard port")
}
if !strategy.IsPluginApplicableByName("ssh", "10.0.0.9", 1, true, cfg) {
t.Fatal("custom mode should respect explicitly selected plugin")
}
if strategy.IsPluginApplicableByName("missing_plugin", "10.0.0.9", 22, true, cfg) {
t.Fatal("missing plugin should never be applicable")
}
}
func TestBaseScanStrategyFilterTypes(t *testing.T) {
plugins.RegisterWithOptions("core_test_local_filter", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeLocal}, false)
plugins.RegisterWithOptions("core_test_web_filter", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeWeb}, true)
plugins.RegisterWithOptions("core_test_udp_filter", func() plugins.Plugin { return nil }, []int{53}, []string{plugins.PluginTypeUDP}, true)
cfg := common.NewConfig()
localStrategy := NewBaseScanStrategy("local", FilterLocal)
if localStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) {
t.Fatal("local plugin should require explicit -local selection")
}
cfg.LocalPlugin = "core_test_local_filter"
if !localStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) {
t.Fatal("explicit local plugin should pass local filter")
}
serviceStrategy := NewBaseScanStrategy("service", FilterService)
if !serviceStrategy.isPluginPassesFilterType("ssh", false, cfg) {
t.Fatal("service plugin should pass service filter")
}
if serviceStrategy.isPluginPassesFilterType("core_test_local_filter", false, cfg) ||
serviceStrategy.isPluginPassesFilterType("core_test_udp_filter", false, cfg) {
t.Fatal("service filter should reject local and UDP plugins")
}
webStrategy := NewBaseScanStrategy("web", FilterWeb)
if !webStrategy.isPluginPassesFilterType("core_test_web_filter", false, cfg) ||
webStrategy.isPluginPassesFilterType("ssh", false, cfg) {
t.Fatal("web filter should only allow web plugins")
}
if webPluginOrder("webtitle") != 0 || webPluginOrder("webpoc") != 2 || webPluginOrder("other") != 1 {
t.Fatal("web plugin order changed")
}
}
func TestFormatPluginList(t *testing.T) {
if got := formatPluginList([]string{"a", "b", "c"}); got != "a, b, c" {
t.Fatalf("short plugin list = %q", got)
}
if got := formatPluginList([]string{"a", "b", "c", "d", "e", "f"}); got == "" || got == "a, b, c, d, e, f" {
t.Fatalf("long plugin list should be summarized, got %q", got)
}
}
func containsString(values []string, want string) bool {
for _, value := range values {
if value == want {
return true
}
}
return false
}
// TestNewBaseScanStrategy 测试构造函数
func TestNewBaseScanStrategy(t *testing.T) {
tests := []struct {
@@ -352,3 +462,134 @@ func TestBaseScanStrategy_ValidateConfiguration(t *testing.T) {
t.Errorf("ValidateConfiguration 应返回 nil, 实际: %v", err)
}
}
// =============================================================================
// IsPluginApplicableByName 补充覆盖
// =============================================================================
// TestIsPluginApplicableByName_FullModeWebPlugin 测试 -full 模式下 web 插件对任意端口生效
func TestIsPluginApplicableByName_FullModeWebPlugin(t *testing.T) {
registerTestPlugins(t)
clearServiceCache()
cfg := common.NewConfig()
cfg.POC.Full = true
strategy := NewBaseScanStrategy("service", FilterService)
// webtitle 是 web 插件;-full 模式下不检查 IsMarkedWebService,直接走 passesFilterType
// FilterService 不允许 local/udp,但允许 web 插件
got := strategy.IsPluginApplicableByName("webtitle", "10.0.0.1", 12345, false, cfg)
if !got {
t.Error("full 模式下 web 插件应对任意端口返回 true")
}
}
// TestIsPluginApplicableByName_FullModeNonWebPlugin 确认 -full 不影响非 web 插件的端口匹配
func TestIsPluginApplicableByName_FullModeNonWebPlugin(t *testing.T) {
registerTestPlugins(t)
clearServiceCache()
cfg := common.NewConfig()
cfg.POC.Full = true
strategy := NewBaseScanStrategy("service", FilterService)
// ssh 不是 web 插件,-full 无特殊逻辑,走普通端口匹配
// ssh 默认端口 22;用 99999 端口应该不匹配
got := strategy.IsPluginApplicableByName("ssh", "10.0.0.1", 99999, false, cfg)
if got {
t.Error("-full 模式对非 web 插件不应绕过端口匹配")
}
}
// =============================================================================
// isPluginApplicableToPort 补充覆盖
// =============================================================================
// TestIsPluginApplicableToPort_WebPlugin web 插件忽略端口直接返回 true
func TestIsPluginApplicableToPort_WebPlugin(t *testing.T) {
registerTestPlugins(t)
strategy := NewBaseScanStrategy("service", FilterService)
// webtitle 是 web 插件,任何端口都应返回 true
if !strategy.isPluginApplicableToPort("webtitle", 8080) {
t.Error("web 插件在任意端口应返回 true")
}
if !strategy.isPluginApplicableToPort("webtitle", 0) {
t.Error("web 插件在端口 0 也应返回 true")
}
}
// TestIsPluginApplicableToPort_NonWebPlugin 非 web 插件走端口匹配逻辑
func TestIsPluginApplicableToPort_NonWebPlugin(t *testing.T) {
registerTestPlugins(t)
clearServiceCache()
strategy := NewBaseScanStrategy("service", FilterService)
// ssh 端口 22 匹配
if !strategy.isPluginApplicableToPort("ssh", 22) {
t.Error("ssh 应匹配端口 22")
}
// ssh 端口 9999 不匹配(无服务缓存)
if strategy.isPluginApplicableToPort("ssh", 9999) {
t.Error("ssh 不应匹配端口 9999")
}
}
// =============================================================================
// isPluginPassesFilterType 补充覆盖
// =============================================================================
// TestIsPluginPassesFilterType_CustomMode isCustomMode=true 应直接跳过过滤返回 true(非 UDP)
func TestIsPluginPassesFilterType_CustomMode(t *testing.T) {
registerTestPlugins(t)
cfg := common.NewConfig()
// FilterLocal 策略下 custom mode 也应通过
localStrategy := NewBaseScanStrategy("local", FilterLocal)
if !localStrategy.isPluginPassesFilterType("ssh", true, cfg) {
t.Error("custom mode 下非 UDP 插件应直接返回 true")
}
// FilterService 策略下 custom mode 也应通过
serviceStrategy := NewBaseScanStrategy("service", FilterService)
if !serviceStrategy.isPluginPassesFilterType("ssh", true, cfg) {
t.Error("custom mode 下 service 策略应直接返回 true")
}
}
// TestIsPluginPassesFilterType_FilterNoneNonLocal FilterNone + 普通 TCP 插件 → true
func TestIsPluginPassesFilterType_FilterNoneNonLocal(t *testing.T) {
registerTestPlugins(t)
cfg := common.NewConfig()
noneStrategy := NewBaseScanStrategy("none", FilterNone)
// ssh 不是 local 插件,FilterNone 应直接返回 true
if !noneStrategy.isPluginPassesFilterType("ssh", false, cfg) {
t.Error("FilterNone + 非 local 插件应返回 true")
}
if !noneStrategy.isPluginPassesFilterType("redis", false, cfg) {
t.Error("FilterNone + 非 local 插件 redis 应返回 true")
}
}
// TestIsPluginPassesFilterType_FilterNoneLocalPlugin FilterNone + local 插件:需要 -local 显式指定
func TestIsPluginPassesFilterType_FilterNoneLocalPlugin(t *testing.T) {
plugins.RegisterWithOptions("core_test_local_none", func() plugins.Plugin { return nil }, nil, []string{plugins.PluginTypeLocal}, false)
cfg := common.NewConfig()
noneStrategy := NewBaseScanStrategy("none", FilterNone)
// 未指定 LocalPlugin,应返回 false
if noneStrategy.isPluginPassesFilterType("core_test_local_none", false, cfg) {
t.Error("FilterNone + local 插件未显式指定时应返回 false")
}
// 指定后应返回 true
cfg.LocalPlugin = "core_test_local_none"
if !noneStrategy.isPluginPassesFilterType("core_test_local_none", false, cfg) {
t.Error("FilterNone + local 插件显式指定后应返回 true")
}
}
-66
View File
@@ -1,66 +0,0 @@
package core
import (
"hash/fnv"
)
// BloomFilter 布隆过滤器,用于ICMP包去重
type BloomFilter struct {
bits []bool
size uint32
k uint32 // hash函数数量
}
// NewBloomFilter 创建布隆过滤器
// size: 预期元素数量
// falsePositiveRate: 期望的误判率(通常0.01即1%)
func NewBloomFilter(size int, falsePositiveRate float64) *BloomFilter {
// 计算最优bit数组大小: m = -n*ln(p) / (ln(2)^2)
// 简化计算:m ≈ n * 10 for p=0.01
m := uint32(size * 10)
if m < 1024 {
m = 1024 // 最小1KB
}
// 计算最优hash函数数量: k = (m/n) * ln(2)
// 简化:k ≈ 7 for p=0.01
k := uint32(7)
return &BloomFilter{
bits: make([]bool, m),
size: m,
k: k,
}
}
// Add 添加元素到过滤器
func (bf *BloomFilter) Add(data string) {
for i := uint32(0); i < bf.k; i++ {
pos := bf.hash(data, i)
bf.bits[pos] = true
}
}
// Contains 检查元素是否可能存在
// 返回true:可能存在(有误判可能)
// 返回false:一定不存在
func (bf *BloomFilter) Contains(data string) bool {
for i := uint32(0); i < bf.k; i++ {
pos := bf.hash(data, i)
if !bf.bits[pos] {
return false
}
}
return true
}
// hash 计算hash值
func (bf *BloomFilter) hash(data string, seed uint32) uint32 {
h := fnv.New32a()
_, _ = h.Write([]byte(data))
// 添加seed实现多个hash函数
for i := uint32(0); i < seed; i++ {
_, _ = h.Write([]byte{byte(i)})
}
return h.Sum32() % bf.size
}
-168
View File
@@ -1,168 +0,0 @@
package core
import (
"fmt"
"testing"
)
/*
bloom_filter_test.go - BloomFilter 高价值测试
测试重点
1. 基本正确性 - Add后Contains返回true未添加的返回false
2. 误判率验证 - 实际误判率应接近理论值(1%)
3. 大规模数据 - 模拟真实ICMP去重场景
不测试
- 内部哈希实现细节
- 精确的数学公式验证
*/
// TestBloomFilter_BasicCorrectness 基本正确性测试
func TestBloomFilter_BasicCorrectness(t *testing.T) {
bf := NewBloomFilter(1000, 0.01)
// 添加元素后应该能找到
testData := []string{
"192.168.1.1",
"10.0.0.1",
"172.16.0.1",
}
for _, data := range testData {
bf.Add(data)
}
for _, data := range testData {
if !bf.Contains(data) {
t.Errorf("已添加的元素 %s 应该返回 true", data)
}
}
// 未添加的元素(大概率)返回false
notAdded := []string{
"8.8.8.8",
"1.1.1.1",
"255.255.255.255",
}
falsePositives := 0
for _, data := range notAdded {
if bf.Contains(data) {
falsePositives++
}
}
// 3个未添加元素全部误判的概率极低(<0.0001%
if falsePositives == len(notAdded) {
t.Error("所有未添加元素都返回true,布隆过滤器可能有问题")
}
}
// TestBloomFilter_FalsePositiveRate 误判率验证
//
// 对于 n=10000, p=0.01 的布隆过滤器:
// 实际误判率应该在 0.5% - 2% 之间(允许统计波动)
func TestBloomFilter_FalsePositiveRate(t *testing.T) {
n := 10000 // 添加的元素数
bf := NewBloomFilter(n, 0.01)
// 添加n个元素
for i := 0; i < n; i++ {
bf.Add(fmt.Sprintf("added_%d", i))
}
// 测试n个未添加的元素
falsePositives := 0
testCount := n
for i := 0; i < testCount; i++ {
if bf.Contains(fmt.Sprintf("not_added_%d", i)) {
falsePositives++
}
}
actualRate := float64(falsePositives) / float64(testCount)
// 允许的误判率范围:0.1% - 3%(考虑统计波动)
if actualRate > 0.03 {
t.Errorf("误判率过高: %.2f%% (期望 < 3%%)", actualRate*100)
}
t.Logf("实际误判率: %.2f%% (%d/%d)", actualRate*100, falsePositives, testCount)
}
// TestBloomFilter_LargeScale 大规模数据测试
//
// 模拟真实的ICMP去重场景:100万个IP地址
func TestBloomFilter_LargeScale(t *testing.T) {
if testing.Short() {
t.Skip("跳过大规模测试")
}
n := 1000000 // 100万
bf := NewBloomFilter(n, 0.01)
// 添加100万个元素
for i := 0; i < n; i++ {
bf.Add(fmt.Sprintf("192.168.%d.%d", i/256, i%256))
}
// 验证已添加的元素
sampleSize := 1000
for i := 0; i < sampleSize; i++ {
idx := i * (n / sampleSize)
data := fmt.Sprintf("192.168.%d.%d", idx/256, idx%256)
if !bf.Contains(data) {
t.Errorf("已添加的元素 %s 返回 false", data)
}
}
// 测试未添加元素的误判率
falsePositives := 0
for i := 0; i < sampleSize; i++ {
if bf.Contains(fmt.Sprintf("10.%d.%d.%d", i/65536, (i/256)%256, i%256)) {
falsePositives++
}
}
actualRate := float64(falsePositives) / float64(sampleSize)
if actualRate > 0.03 {
t.Errorf("大规模场景误判率过高: %.2f%%", actualRate*100)
}
t.Logf("100万元素场景误判率: %.2f%%", actualRate*100)
}
// TestBloomFilter_NoFalseNegative 验证无假阴性
//
// 布隆过滤器的核心保证:已添加的元素必定返回true
func TestBloomFilter_NoFalseNegative(t *testing.T) {
bf := NewBloomFilter(10000, 0.01)
// 添加5000个元素
added := make([]string, 5000)
for i := range added {
added[i] = fmt.Sprintf("element_%d", i)
bf.Add(added[i])
}
// 全部验证
for _, data := range added {
if !bf.Contains(data) {
t.Fatalf("假阴性!已添加的元素 %s 返回 false", data)
}
}
}
// TestBloomFilter_EmptyFilter 空过滤器测试
func TestBloomFilter_EmptyFilter(t *testing.T) {
bf := NewBloomFilter(100, 0.01)
// 空过滤器应该对任何查询返回false
testCases := []string{"anything", "192.168.1.1", ""}
for _, tc := range testCases {
if bf.Contains(tc) {
t.Errorf("空过滤器对 %q 返回 true", tc)
}
}
}
+651
View File
@@ -0,0 +1,651 @@
package core
import (
"math"
"sync"
"testing"
"time"
)
// =============================================================================
// computeRetries 边界
// =============================================================================
func TestComputeRetries_EdgeCases(t *testing.T) {
tests := []struct {
lossRate float64
wantMin int
wantMax int
desc string
}{
{-0.5, 1, 1, "负数丢包率: 视为零"},
{-1.0, 1, 1, "负一: 视为零"},
{0.0, 1, 1, "精确零"},
{0.001, 1, 1, "精确边界 0.001"},
{0.0009, 1, 1, "低于 0.001 边界"},
{0.0011, 1, 5, "高于 0.001 边界"},
{0.95, 5, 5, "精确边界 0.95"},
{0.949, 1, 5, "低于 0.95 边界"},
{0.951, 5, 5, "高于 0.95 边界"},
{1.0, 5, 5, "精确 1.0"},
{1.5, 5, 5, "超过 1.0"},
{100.0, 5, 5, "极大值"},
{math.SmallestNonzeroFloat64, 1, 1, "最小正浮点数"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := computeRetries(tt.lossRate, EnvWAN)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeRetries(%v) = %d, want [%d, %d]",
tt.lossRate, got, tt.wantMin, tt.wantMax)
}
if got < 1 || got > 5 {
t.Errorf("computeRetries(%v) = %d, 超出 [1,6] 范围", tt.lossRate, got)
}
})
}
}
func TestComputeRetries_NaN_Inf(t *testing.T) {
// 确保不 panic
for _, v := range []float64{math.NaN(), math.Inf(1), math.Inf(-1)} {
got := computeRetries(v, EnvWAN)
if got < 1 || got > 5 {
t.Errorf("computeRetries(%v) = %d, 超出 [1,6] 范围", v, got)
}
}
}
// =============================================================================
// computeICMPRate 边界
// =============================================================================
func TestComputeICMPRate_EdgeCases(t *testing.T) {
tests := []struct {
env NetworkEnv
fdLimit int
desc string
}{
{EnvLAN, 1, "fd=1: 极小"},
{EnvLAN, -1, "fd=负数: 应被忽略"},
{EnvLAN, 0, "fd=0: 未知"},
{EnvLAN, math.MaxInt32, "fd=极大"},
{NetworkEnv(99), 1024, "未知环境类型"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
net := &NetworkProfile{Env: tt.env}
sys := &SystemProfile{FDLimit: tt.fdLimit}
got := computeICMPRate(net, sys)
if got <= 0 || math.IsNaN(got) || math.IsInf(got, 0) {
t.Errorf("computeICMPRate(env=%v, fd=%d) = %v, 无效值", tt.env, tt.fdLimit, got)
}
})
}
}
// =============================================================================
// classifyEnv 精确边界值
// =============================================================================
func TestClassifyEnv_ExactBoundaries(t *testing.T) {
tests := []struct {
median time.Duration
lossRate float64
want NetworkEnv
desc string
}{
// RTT 边界
{4999 * time.Microsecond, 0.0, EnvLAN, "4.999ms → LAN"},
{5 * time.Millisecond, 0.0, EnvWAN, "精确 5ms → WAN"},
{49999 * time.Microsecond, 0.0, EnvWAN, "49.999ms → WAN"},
{50 * time.Millisecond, 0.0, EnvInternet, "精确 50ms → Internet"},
{199999 * time.Microsecond, 0.0, EnvInternet, "199.999ms → Internet"},
{200 * time.Millisecond, 0.0, EnvSlow, "精确 200ms → Slow"},
// 丢包率边界
{1 * time.Millisecond, 0.009, EnvLAN, "丢包 0.9% → LAN"},
{1 * time.Millisecond, 0.01, EnvWAN, "精确 1% → WAN (不满足 < 0.01)"},
{1 * time.Millisecond, 0.011, EnvWAN, "丢包 1.1% → WAN (超过 LAN 阈值)"},
{20 * time.Millisecond, 0.049, EnvWAN, "丢包 4.9% → WAN"},
{20 * time.Millisecond, 0.05, EnvInternet, "精确 5% → Internet (不满足 < 0.05)"},
{20 * time.Millisecond, 0.051, EnvInternet, "丢包 5.1% → Internet"},
{1 * time.Millisecond, 0.099, EnvInternet, "丢包 9.9% → Internet"},
{1 * time.Millisecond, 0.10, EnvInternet, "精确 10% → Internet (< 判断)"},
{1 * time.Millisecond, 0.101, EnvSlow, "丢包 10.1% → Slow"},
// 零值
{0, 0.0, EnvLAN, "零 RTT 零丢包 → LAN"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := classifyEnv(tt.median, tt.lossRate)
if got != tt.want {
t.Errorf("classifyEnv(median=%v, loss=%.4f) = %v, want %v",
tt.median, tt.lossRate, got, tt.want)
}
})
}
}
// =============================================================================
// classifyNetwork 边界
// =============================================================================
func TestClassifyNetwork_EdgeCases(t *testing.T) {
t.Run("单个 RTT 样本", func(t *testing.T) {
p := classifyNetwork([]time.Duration{5 * time.Millisecond}, 0, 1)
if p.Samples != 1 {
t.Errorf("samples = %d, want 1", p.Samples)
}
// stddev 应该是 0
if p.RTTStddev != 0 {
t.Errorf("单样本 stddev = %v, want 0", p.RTTStddev)
}
})
t.Run("所有 RTT 相同", func(t *testing.T) {
rtts := make([]time.Duration, 50)
for i := range rtts {
rtts[i] = 10 * time.Millisecond
}
p := classifyNetwork(rtts, 0, 50)
if p.RTTStddev != 0 {
t.Errorf("全相同 RTT stddev = %v, want 0", p.RTTStddev)
}
if p.RTTMedian != 10*time.Millisecond {
t.Errorf("median = %v, want 10ms", p.RTTMedian)
}
})
t.Run("极大 RTT 值", func(t *testing.T) {
rtts := []time.Duration{time.Hour, time.Hour, time.Hour}
p := classifyNetwork(rtts, 0, 3)
if p.Env != EnvSlow {
t.Errorf("env = %v, want Slow", p.Env)
}
})
t.Run("混合极端值", func(t *testing.T) {
rtts := []time.Duration{time.Microsecond, time.Hour}
p := classifyNetwork(rtts, 0, 2)
// 不 panic 就行
if p.Samples != 2 {
t.Errorf("samples = %d, want 2", p.Samples)
}
})
t.Run("全部失败无响应", func(t *testing.T) {
p := classifyNetwork(nil, 100, 100)
if p.Env != EnvWAN {
t.Errorf("env = %v, want WAN (default)", p.Env)
}
})
t.Run("failures > total (异常输入)", func(t *testing.T) {
rtts := []time.Duration{time.Millisecond}
p := classifyNetwork(rtts, 10, 5) // failures > total
// lossRate = 1 - 1/5 = 0.8, 不应 panic
if p.LossRate < 0 {
t.Errorf("lossRate = %.2f, 不应为负", p.LossRate)
}
})
t.Run("total=0", func(t *testing.T) {
p := classifyNetwork(nil, 0, 0)
// 不 panic
if p.Samples != 0 {
t.Errorf("samples = %d, want 0", p.Samples)
}
})
}
// =============================================================================
// RecommendConcurrency 边界
// =============================================================================
func TestRecommendConcurrency_EdgeCases(t *testing.T) {
tests := []struct {
env NetworkEnv
loss float64
userT int
explicit bool
desc string
}{
{EnvLAN, 0.0, 0, false, "userThreadNum=0"},
{EnvLAN, 0.0, 1, false, "userThreadNum=1"},
{EnvLAN, 0.0, -1, false, "userThreadNum 负数"},
{EnvLAN, 0.0, math.MaxInt32, false, "userThreadNum 极大"},
{EnvLAN, 0.99, 600, false, "99% 丢包"},
{EnvLAN, 1.0, 600, false, "100% 丢包"},
{EnvSlow, 0.0, 1, true, "慢速+显式+1"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
p := &NetworkProfile{Env: tt.env, LossRate: tt.loss, Samples: 10}
target, ceiling := p.RecommendConcurrency(tt.userT, tt.explicit)
// 不 panic,且 target >= 1clamp 保底 10 或 userT
if target < 0 || ceiling < 0 {
t.Errorf("target=%d ceiling=%d, 不应为负", target, ceiling)
}
if tt.explicit && ceiling != tt.userT && tt.userT > 0 {
t.Errorf("显式模式 ceiling=%d, want %d", ceiling, tt.userT)
}
t.Logf("env=%v loss=%.2f userT=%d explicit=%v → target=%d ceiling=%d",
tt.env, tt.loss, tt.userT, tt.explicit, target, ceiling)
})
}
}
// =============================================================================
// ScanMetrics 边界
// =============================================================================
func TestScanMetrics_EdgeCases(t *testing.T) {
t.Run("RTT=0", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(0)
// 不 panic
if m.Total() != 1 {
t.Errorf("Total = %d, want 1", m.Total())
}
})
t.Run("负数 RTT", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(-time.Millisecond)
// 不 panic,负数 RTT 应被忽略
if m.rttSamples.Load() != 0 {
t.Errorf("负数 RTT 不应计入采样: got %d", m.rttSamples.Load())
}
})
t.Run("极大 RTT", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(time.Hour)
if m.RTTFast() != time.Hour {
t.Errorf("首个样本 RTTFast = %v, want 1h", m.RTTFast())
}
})
t.Run("EMA 首个样本初始化", func(t *testing.T) {
m := &ScanMetrics{}
m.RecordConnect(10 * time.Millisecond)
if m.rttFastNs.Load() != int64(10*time.Millisecond) {
t.Errorf("首个样本应直接设置 EMA: got %d", m.rttFastNs.Load())
}
})
t.Run("空 Snapshot", func(t *testing.T) {
m := &ScanMetrics{}
snap := m.Snapshot()
if snap.Total() != 0 {
t.Errorf("空 metrics Snapshot.Total = %d, want 0", snap.Total())
}
})
t.Run("RTTRatio 单侧为零", func(t *testing.T) {
m := &ScanMetrics{}
// 手动设置一个但不设另一个——不应该发生,但防御
m.rttFastNs.Store(1000)
m.rttSlowNs.Store(0)
m.rttSamples.Store(30)
ratio := m.RTTRatio()
if ratio != 1.0 {
t.Errorf("slow=0 时 ratio = %.2f, want 1.0", ratio)
}
})
t.Run("大量操作不溢出", func(t *testing.T) {
m := &ScanMetrics{}
for i := 0; i < 100000; i++ {
m.RecordConnect(time.Millisecond)
}
if m.Total() != 100000 {
t.Errorf("Total = %d, want 100000", m.Total())
}
ratio := m.RTTRatio()
if math.IsNaN(ratio) || math.IsInf(ratio, 0) {
t.Errorf("大量样本后 ratio = %v, 不应为 NaN/Inf", ratio)
}
})
}
// =============================================================================
// TuneConfig 边界
// =============================================================================
func TestTuneConfig_EdgeCases(t *testing.T) {
t.Run("RTTMedian=0 RTTStddev=0", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: 0, RTTStddev: 0, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// Timeout: median(0) + 4*stddev(0) = 0 → minTO = 0+200ms → clamp to 1s
if config.Timeout < time.Second {
t.Errorf("零 RTT Timeout = %v, 应该 >= 1s", config.Timeout)
}
})
t.Run("RTTStddev 远大于 RTTMedian", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvInternet, RTTMedian: 10 * time.Millisecond, RTTStddev: 5 * time.Second, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// Timeout = 10ms + 4*5s = 20.01s → clamp to 10s
if config.Timeout != 10*time.Second {
t.Errorf("极大 stddev Timeout = %v, 应该被 clamp 到 10s", config.Timeout)
}
})
t.Run("ThreadNum=0", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 0
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: time.Millisecond, RTTStddev: time.Millisecond, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
// ModuleThreadNum = 0/30 = 0 → clamp to 5
if config.ModuleThreadNum < 5 {
t.Errorf("ThreadNum=0 时 ModuleThreadNum = %d, 应该 >= 5", config.ModuleThreadNum)
}
})
t.Run("多次调用 TuneConfig", func(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Env: EnvLAN, RTTMedian: time.Millisecond, RTTStddev: time.Millisecond, LossRate: 0.0, Samples: 10},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
first := config.Timeout
// 第二次调用——已经调整过的值不等于默认值,应被视为"显式"
ep.TuneConfig(config, session)
second := config.Timeout
if first != second {
t.Errorf("多次调用 TuneConfig 不应重复调整: %v vs %v", first, second)
}
})
t.Run("fd limit = ThreadNum 精确值", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 1000}, // 1000 * 0.6 = 600
}
ep.TuneConfig(config, session)
// ThreadNum(600) == maxConcurrency(600), 不应触发约束
if config.ThreadNum != 600 {
t.Errorf("fd=1000 时 ThreadNum = %d, 不应被约束", config.ThreadNum)
}
})
t.Run("fd limit 精确低于 ThreadNum", func(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 999}, // 999 * 0.6 = 599
}
ep.TuneConfig(config, session)
if config.ThreadNum > 599 {
t.Errorf("fd=999 时 ThreadNum = %d, 应该 <= 599", config.ThreadNum)
}
})
}
// =============================================================================
// AdaptivePool 边界
// =============================================================================
func TestAdaptivePool_EdgeCases(t *testing.T) {
t.Run("target=1", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(1, 1, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// initial = max(1/4, 10) = 10 → 但 10 > target(1)... 看实现
// 实际上 initial = min(max(1/4, 10), 1) = 1... 不对
// initial = target/4 = 0, 但 < 10, 所以 initial = 10
// 但 initial > target(1)... initial = min(10, 1) = 1
// 看代码:if initial > target { initial = target }
if pool.Cap() != 1 {
t.Errorf("target=1 时 cap = %d, want 1", pool.Cap())
}
})
t.Run("target=0", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(0, 0, func(interface{}) {}, metrics)
// ants 可能拒绝 size=0
if err != nil {
t.Logf("target=0 正确返回错误: %v", err)
return
}
defer pool.Release()
t.Logf("target=0 cap = %d", pool.Cap())
})
t.Run("ceiling < target", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 50, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// initial = 100/4 = 25, 不超过 ceiling
if pool.Cap() > 50 {
t.Errorf("ceiling=50 但 cap = %d", pool.Cap())
}
})
t.Run("高频 Invoke 不 panic", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(10, 10, func(interface{}) {
time.Sleep(time.Millisecond)
}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(10)
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_ = pool.Invoke(nil)
}()
}
wg.Wait()
pool.Wait()
})
t.Run("assessHealth 零增量", func(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建失败: %v", err)
}
defer pool.Release()
// 初始化 prevSnapshot 后不产生新数据
pool.prevSnapshot = metrics.Snapshot()
health := pool.assessHealth()
if health != HealthUnknown {
t.Errorf("零增量应返回 HealthUnknown, got %v", health)
}
})
}
// =============================================================================
// pickSamples 边界
// =============================================================================
func TestPickSamples_EdgeCases(t *testing.T) {
t.Run("maxSamples=0", func(t *testing.T) {
s := pickSamples([]string{"a", "b"}, 0)
if len(s) != 0 {
t.Errorf("maxSamples=0 应返回空, got %d", len(s))
}
})
t.Run("maxSamples=1", func(t *testing.T) {
s := pickSamples([]string{"a", "b", "c"}, 1)
if len(s) != 1 {
t.Errorf("maxSamples=1 应返回 1 个, got %d", len(s))
}
})
t.Run("hosts 等于 maxSamples", func(t *testing.T) {
hosts := []string{"a", "b", "c"}
s := pickSamples(hosts, 3)
if len(s) != 3 {
t.Errorf("应返回全部, got %d", len(s))
}
})
}
// =============================================================================
// isTimeoutError / isConnectionRefused 边界
// =============================================================================
func TestIsTimeoutError_EdgeCases(t *testing.T) {
if isTimeoutError(nil) {
t.Error("nil 不应判为 timeout")
}
}
func TestIsConnectionRefused_EdgeCases(t *testing.T) {
if isConnectionRefused(nil) {
t.Error("nil 不应判为 refused")
}
}
// =============================================================================
// NetworkEnv.String 覆盖
// =============================================================================
func TestNetworkEnv_String(t *testing.T) {
for _, env := range []NetworkEnv{EnvLAN, EnvWAN, EnvInternet, EnvSlow} {
s := env.String()
if s == "" {
t.Errorf("NetworkEnv(%d).String() = 空", env)
}
}
// 未知值
s := NetworkEnv(99).String()
if s == "" {
t.Error("未知 NetworkEnv.String() = 空")
}
}
// =============================================================================
// clampInt / clampDuration 边界
// =============================================================================
func TestClampInt(t *testing.T) {
tests := []struct {
v, min, max, want int
}{
{5, 1, 10, 5},
{0, 1, 10, 1},
{15, 1, 10, 10},
{-5, -10, -1, -5},
{5, 5, 5, 5}, // min == max == v
{3, 5, 5, 5}, // v < min == max
{10, 5, 5, 5}, // v > min == max
}
for _, tt := range tests {
got := clampInt(tt.v, tt.min, tt.max)
if got != tt.want {
t.Errorf("clampInt(%d, %d, %d) = %d, want %d", tt.v, tt.min, tt.max, got, tt.want)
}
}
}
func TestClampDuration(t *testing.T) {
got := clampDuration(5*time.Second, time.Second, 10*time.Second)
if got != 5*time.Second {
t.Errorf("got %v, want 5s", got)
}
got = clampDuration(0, time.Second, 10*time.Second)
if got != time.Second {
t.Errorf("got %v, want 1s", got)
}
got = clampDuration(time.Hour, time.Second, 10*time.Second)
if got != 10*time.Second {
t.Errorf("got %v, want 10s", got)
}
}
// =============================================================================
// isExplicit 边界
// =============================================================================
func TestIsExplicit(t *testing.T) {
config := makeDefaultConfig()
// 默认值 → 非显式
if isExplicit(config, "time") {
t.Error("默认 Timeout 不应视为显式")
}
if isExplicit(config, "mt") {
t.Error("默认 ModuleThreadNum 不应视为显式")
}
if isExplicit(config, "retry") {
t.Error("默认 MaxRetries 不应视为显式")
}
if isExplicit(config, "icmp-rate") {
t.Error("默认 ICMPRate 不应视为显式")
}
if isExplicit(config, "num") {
t.Error("默认 PocNum 不应视为显式")
}
// 未知 flag
if isExplicit(config, "nonexistent") {
t.Error("未知 flag 不应视为显式")
}
// ThreadNumExplicit
config.ThreadNumExplicit = true
if !isExplicit(config, "t") {
t.Error("ThreadNumExplicit=true 应视为显式")
}
config = makeDefaultConfig()
config.TimeoutExplicit = true
config.ModuleThreadNumExplicit = true
config.MaxRetriesExplicit = true
config.Network.ICMPRateExplicit = true
config.POC.NumExplicit = true
if !isExplicit(config, "time") || !isExplicit(config, "mt") ||
!isExplicit(config, "retry") || !isExplicit(config, "icmp-rate") ||
!isExplicit(config, "num") {
t.Error("显式标记为 true 时默认值也应视为显式")
}
}
+256
View File
@@ -0,0 +1,256 @@
package core
import (
"fmt"
"math"
"runtime"
"time"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// EnvironmentProfile 综合环境探测结果
type EnvironmentProfile struct {
Net NetworkProfile
System SystemProfile
}
// SystemProfile 系统能力信息
type SystemProfile struct {
FDLimit int // 文件描述符上限(0 表示未知)
NumCPU int
}
// ProbeSystem 探测系统能力(不需要网络目标)
func ProbeSystem() SystemProfile {
p := SystemProfile{
NumCPU: runtime.NumCPU(),
}
p.FDLimit = getFDLimit()
return p
}
// TuneConfig 根据探测结果调整 Config 中的参数
// 只调整用户未显式指定的参数
// 每个参数的推导都有明确的公式和探测依据
func (ep *EnvironmentProfile) TuneConfig(config *common.Config, session *common.ScanSession) {
net := &ep.Net
sys := &ep.System
// ---------- NetworkEnv ----------
config.DetectedNetworkEnv = int(net.Env)
// ---------- ThreadNum / ThreadCeiling ----------
if !isExplicit(config, "t") {
target, ceiling := net.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
old := config.ThreadNum
config.ThreadNum = target
config.ThreadCeiling = ceiling
session.LogDebug(fmt.Sprintf("ThreadNum: %d -> %d, Ceiling: %d (env=%s)", old, target, ceiling, net.Env))
} else {
config.ThreadCeiling = config.ThreadNum
}
// ---------- Timeout ----------
// 公式: median_rtt + 4 * stddev,下限 1s,上限 10s
// 依据: 与 AdaptiveTimeout 相同的统计原理(覆盖 99.9% 的正常连接)
if !isExplicit(config, "time") && net.Samples > 0 {
computed := net.RTTMedian + 4*net.RTTStddev
// 下限:连接建立至少需要 2 个 RTT(SYN + SYN-ACK+ 处理时间
minTO := net.RTTMedian*3 + 200*time.Millisecond
if computed < minTO {
computed = minTO
}
computed = clampDuration(computed, time.Second, 10*time.Second)
old := config.Timeout
config.Timeout = computed
session.LogDebug(fmt.Sprintf("Timeout: %v -> %v (RTT median=%v stddev=%v)",
old, computed, net.RTTMedian, net.RTTStddev))
}
// ---------- ModuleThreadNum ----------
// 公式: ThreadNum / 30,下限 5,上限 50
// 依据: 插件级并发(爆破等)不应超过端口扫描并发的 ~3%
// 单个服务的连接能力远低于 TCP SYN 扫描
// 公网服务通常有限流(MaxStartups 等),并发过高适得其反
if !isExplicit(config, "mt") {
computed := config.ThreadNum / 30
computed = clampInt(computed, 5, 50)
// 高丢包环境进一步压低,避免大量连接被丢弃浪费
if net.LossRate > 0.1 {
computed = computed * 2 / 3
if computed < 5 {
computed = 5
}
}
old := config.ModuleThreadNum
config.ModuleThreadNum = computed
session.LogDebug(fmt.Sprintf("ModuleThreadNum: %d -> %d (threadNum=%d)", old, computed, config.ThreadNum))
}
// ---------- MaxRetries ----------
// 公式: ceil(log(0.01) / log(loss_rate))
// 含义: 重试 N 次后仍然全部丢包的概率 < 1%
// 例: 丢包率 5% → N=2, 丢包率 20% → N=3, 丢包率 50% → N=7
// 下限 1(零丢包也至少试一次),上限 6(避免对不可达目标死磕)
if !isExplicit(config, "retry") {
if net.Samples > 0 {
computed := computeRetries(net.LossRate, net.Env)
old := config.MaxRetries
config.MaxRetries = computed
session.LogDebug(fmt.Sprintf("MaxRetries: %d -> %d (loss_rate=%.2f%%)", old, computed, net.LossRate*100))
} else if config.MaxRetries > 2 {
// 无网络探测数据(-np 跳过存活探测),降低默认重试避免对不可达主机死磕
old := config.MaxRetries
config.MaxRetries = 2
session.LogDebug(fmt.Sprintf("MaxRetries: %d -> %d (no network probe data)", old, config.MaxRetries))
}
}
// ---------- ICMPRate ----------
// 公式: 基于 fd limit 和网络环境
// 内网 fd 充裕: 0.5(高速发包)
// 公网或 fd 紧张: 0.1(默认保守)
// 依据: ICMP 发包速率受两个约束:网络带宽和本机 fd/socket 资源
if !isExplicit(config, "icmp-rate") && net.Samples > 0 {
computed := computeICMPRate(net, sys)
old := config.Network.ICMPRate
config.Network.ICMPRate = computed
session.LogDebug(fmt.Sprintf("ICMPRate: %.2f -> %.2f (env=%s fd=%d)", old, computed, net.Env, sys.FDLimit))
}
// ---------- PocNum ----------
// 公式: 与 ModuleThreadNum 一致
// 依据: POC 检测和凭据爆破的并发约束相同——都是对目标服务发起连接
if !isExplicit(config, "num") {
old := config.POC.Num
config.POC.Num = config.ModuleThreadNum
session.LogDebug(fmt.Sprintf("PocNum: %d -> %d (follows ModuleThreadNum)", old, config.POC.Num))
}
// ---------- DisablePing ----------
// 由 probeWithICMP 自动处理(尝试 → 失败 → 降级),无需在此干预
// 总结日志
if net.Samples > 0 {
session.LogInfo(i18n.Tr("env_tune_summary",
config.Timeout.Milliseconds(),
config.ModuleThreadNum,
config.MaxRetries,
fmt.Sprintf("%.2f", config.Network.ICMPRate),
config.POC.Num))
}
// fd limit 约束:总并发不应超过 fd limit 的 60%(留余量给系统)
if sys.FDLimit > 0 {
maxConcurrency := sys.FDLimit * 6 / 10
if config.ThreadNum > maxConcurrency {
session.LogInfo(i18n.Tr("env_fd_limit", config.ThreadNum, maxConcurrency, sys.FDLimit))
config.ThreadNum = maxConcurrency
}
if config.ThreadCeiling > maxConcurrency {
config.ThreadCeiling = maxConcurrency
}
}
}
// computeRetries 基于丢包率和网络环境计算重试次数
// 内网丢包异常,用更严格的目标概率(0.5%)和更低上限
// 公网/慢速丢包常见,放宽目标概率(2%)和更高上限
func computeRetries(lossRate float64, env NetworkEnv) int {
if lossRate <= 0.001 {
return 1
}
var targetProb float64
var maxRetries int
switch env {
case EnvLAN:
targetProb = 0.005
maxRetries = 4
case EnvWAN:
targetProb = 0.01
maxRetries = 5
default:
targetProb = 0.02
maxRetries = 6
}
if lossRate >= 0.95 {
return maxRetries
}
// P(N次全失败) = lossRate^N < targetProb
n := math.Ceil(math.Log(targetProb) / math.Log(lossRate))
return clampInt(int(n), 1, maxRetries)
}
// computeICMPRate 基于环境计算 ICMP 发包速率
func computeICMPRate(net *NetworkProfile, sys *SystemProfile) float64 {
// 基准:根据 RTT 估算网络可承受的速率
// RTT 越低,网络越快,可以发更快
var base float64
switch net.Env {
case EnvLAN:
base = 0.5
case EnvWAN:
base = 0.3
case EnvInternet:
base = 0.1
default:
base = 0.05
}
// fd 约束:fd limit 低时压低速率
if sys.FDLimit > 0 && sys.FDLimit < 1024 {
base = base * float64(sys.FDLimit) / 1024.0
if base < 0.02 {
base = 0.02
}
}
return base
}
// isExplicit 检查参数是否被用户显式指定。
// 显式标记来自 CLI flag.Visit;值比较保留 SDK/测试里直接构造 Config 的旧行为。
func isExplicit(config *common.Config, flagName string) bool {
switch flagName {
case "t":
return config.ThreadNumExplicit
case "time":
return config.TimeoutExplicit || config.Timeout != 3*time.Second
case "mt":
return config.ModuleThreadNumExplicit || config.ModuleThreadNum != 20
case "retry":
return config.MaxRetriesExplicit || config.MaxRetries != 3
case "icmp-rate":
return config.Network.ICMPRateExplicit || config.Network.ICMPRate != 0.1
case "num":
return config.POC.NumExplicit || config.POC.Num != 20
}
return false
}
func clampInt(v, min, max int) int {
if v < min {
return min
}
if v > max {
return max
}
return v
}
func clampDuration(v, min, max time.Duration) time.Duration {
if v < min {
return min
}
if v > max {
return max
}
return v
}
+374
View File
@@ -0,0 +1,374 @@
package core
import (
"math"
"testing"
"time"
"github.com/shadow1ng/fscan/common"
)
// =============================================================================
// 单元测试:computeRetries — 丢包率到重试次数的推导
// =============================================================================
func TestComputeRetries(t *testing.T) {
tests := []struct {
lossRate float64
wantMin int
wantMax int
desc string
}{
{0.0, 1, 1, "零丢包: 只需 1 次"},
{0.001, 1, 1, "极低丢包: 1 次"},
{0.05, 2, 2, "5% 丢包: 0.05^2=0.0025 < 0.01"},
{0.10, 2, 3, "10% 丢包: ceil(log(0.01)/log(0.1))=2, 但边界取 ceil 可能是 3"},
{0.20, 3, 3, "20% 丢包: 0.2^3=0.008 < 0.01"},
{0.30, 3, 4, "30% 丢包"},
{0.50, 5, 5, "50% 丢包: ceil(log(0.01)/log(0.5))=7 但上限 5"},
{0.80, 5, 5, "80% 丢包: 需要很多次但上限 5"},
{0.95, 5, 5, "95% 丢包: 触顶"},
{1.0, 5, 5, "100% 丢包: 触顶"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := computeRetries(tt.lossRate, EnvWAN)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeRetries(%.2f) = %d, want [%d, %d]",
tt.lossRate, got, tt.wantMin, tt.wantMax)
}
// 验证数学正确性:lossRate^got < 0.01
// 跳过:零丢包、极高丢包(触顶上限 6 时数学不满足,属于设计取舍)
if tt.lossRate > 0.001 && tt.lossRate < 0.45 {
prob := math.Pow(tt.lossRate, float64(got))
if prob >= 0.01 {
t.Errorf("lossRate=%.2f retries=%d: P(全失败)=%.4f >= 0.01, 重试不够",
tt.lossRate, got, prob)
}
}
})
}
}
// =============================================================================
// 单元测试:computeICMPRate
// =============================================================================
func TestComputeICMPRate(t *testing.T) {
tests := []struct {
env NetworkEnv
fdLimit int
wantMin float64
wantMax float64
desc string
}{
{EnvLAN, 65536, 0.4, 0.6, "内网高 fd: 高速"},
{EnvWAN, 65536, 0.2, 0.4, "局域网高 fd: 中速"},
{EnvInternet, 65536, 0.05, 0.15, "公网: 保守"},
{EnvSlow, 65536, 0.03, 0.08, "慢速: 极保守"},
{EnvLAN, 256, 0.01, 0.2, "内网低 fd: 受限"},
{EnvLAN, 0, 0.4, 0.6, "fd 未知: 按环境"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
net := &NetworkProfile{Env: tt.env}
sys := &SystemProfile{FDLimit: tt.fdLimit}
got := computeICMPRate(net, sys)
if got < tt.wantMin || got > tt.wantMax {
t.Errorf("computeICMPRate(env=%v, fd=%d) = %.3f, want [%.3f, %.3f]",
tt.env, tt.fdLimit, got, tt.wantMin, tt.wantMax)
}
})
}
}
// =============================================================================
// 集成测试:TuneConfig — 完整参数调整流程
// =============================================================================
func TestTuneConfig_LAN(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMin: 500 * time.Microsecond,
RTTMedian: 1 * time.Millisecond,
RTTP95: 3 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// Timeout: median(1ms) + 4*stddev(0.5ms) = 3ms → clamp to 1s 下限
if config.Timeout < time.Second || config.Timeout > 2*time.Second {
t.Errorf("LAN Timeout = %v, 内网应该在 1-2s", config.Timeout)
}
// MaxRetries: 零丢包 → 1
if config.MaxRetries != 1 {
t.Errorf("LAN MaxRetries = %d, 零丢包应该是 1", config.MaxRetries)
}
// ICMPRate: 内网应该比默认 0.1 高
if config.Network.ICMPRate <= 0.1 {
t.Errorf("LAN ICMPRate = %.2f, 应该 > 0.1", config.Network.ICMPRate)
}
// ModuleThreadNum: 基于 ThreadNum/30
if config.ModuleThreadNum < 5 {
t.Errorf("LAN ModuleThreadNum = %d, 应该 >= 5", config.ModuleThreadNum)
}
t.Logf("LAN 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
func TestTuneConfig_Internet(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvInternet,
RTTMin: 50 * time.Millisecond,
RTTMedian: 100 * time.Millisecond,
RTTP95: 250 * time.Millisecond,
RTTStddev: 40 * time.Millisecond,
LossRate: 0.08,
Samples: 25,
},
System: SystemProfile{FDLimit: 1024, NumCPU: 4},
}
ep.TuneConfig(config, session)
// Timeout: median(100ms) + 4*stddev(40ms) = 260ms → 但 minTO = 3*100+200 = 500ms
if config.Timeout < 500*time.Millisecond || config.Timeout > 5*time.Second {
t.Errorf("Internet Timeout = %v, 公网应该在 500ms-5s", config.Timeout)
}
// MaxRetries: 8% 丢包 → ceil(log(0.01)/log(0.08)) ≈ 2
if config.MaxRetries < 2 || config.MaxRetries > 3 {
t.Errorf("Internet MaxRetries = %d, 8%%丢包应该是 2-3", config.MaxRetries)
}
// ICMPRate: 公网应该偏低
if config.Network.ICMPRate > 0.2 {
t.Errorf("Internet ICMPRate = %.2f, 应该 <= 0.2", config.Network.ICMPRate)
}
t.Logf("Internet 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
func TestTuneConfig_SlowLossy(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvSlow,
RTTMin: 200 * time.Millisecond,
RTTMedian: 500 * time.Millisecond,
RTTP95: 2 * time.Second,
RTTStddev: 300 * time.Millisecond,
LossRate: 0.25,
Samples: 15,
},
System: SystemProfile{FDLimit: 512, NumCPU: 2},
}
ep.TuneConfig(config, session)
// Timeout: median(500ms) + 4*stddev(300ms) = 1700ms, minTO = 500*3+200 = 1700ms
if config.Timeout < time.Second {
t.Errorf("Slow Timeout = %v, 慢速网络应该 >= 1s", config.Timeout)
}
// MaxRetries: 25% 丢包 → ceil(log(0.01)/log(0.25)) ≈ 4
if config.MaxRetries < 3 || config.MaxRetries > 5 {
t.Errorf("Slow MaxRetries = %d, 25%%丢包应该是 3-5", config.MaxRetries)
}
// ICMPRate: 慢速 + 低 fd → 应该很低
if config.Network.ICMPRate > 0.1 {
t.Errorf("Slow ICMPRate = %.2f, 应该 <= 0.1", config.Network.ICMPRate)
}
t.Logf("Slow 参数: Timeout=%v, MT=%d, Retry=%d, ICMP=%.2f, POC=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate, config.POC.Num)
}
// =============================================================================
// 集成测试:用户显式指定时不覆盖
// =============================================================================
func TestTuneConfig_ExplicitOverride(t *testing.T) {
config := makeDefaultConfig()
config.Timeout = 5 * time.Second // 用户设了 -time 5
config.ModuleThreadNum = 50 // 用户设了 -mt 50
config.MaxRetries = 1 // 用户设了 -retry 1
config.Network.ICMPRate = 0.8 // 用户设了 -icmp-rate 0.8
config.POC.Num = 100 // 用户设了 -num 100
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// 所有非默认值都不应被覆盖
if config.Timeout != 5*time.Second {
t.Errorf("用户 Timeout 被覆盖: %v", config.Timeout)
}
if config.ModuleThreadNum != 50 {
t.Errorf("用户 ModuleThreadNum 被覆盖: %d", config.ModuleThreadNum)
}
if config.MaxRetries != 1 {
t.Errorf("用户 MaxRetries 被覆盖: %d", config.MaxRetries)
}
if config.Network.ICMPRate != 0.8 {
t.Errorf("用户 ICMPRate 被覆盖: %.2f", config.Network.ICMPRate)
}
if config.POC.Num != 100 {
t.Errorf("用户 PocNum 被覆盖: %d", config.POC.Num)
}
}
func TestTuneConfig_ExplicitDefaultValues(t *testing.T) {
config := makeDefaultConfig()
config.TimeoutExplicit = true
config.ModuleThreadNumExplicit = true
config.MaxRetriesExplicit = true
config.Network.ICMPRateExplicit = true
config.POC.NumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
if config.Timeout != 3*time.Second {
t.Errorf("显式默认 Timeout 被覆盖: %v", config.Timeout)
}
if config.ModuleThreadNum != 20 {
t.Errorf("显式默认 ModuleThreadNum 被覆盖: %d", config.ModuleThreadNum)
}
if config.MaxRetries != 3 {
t.Errorf("显式默认 MaxRetries 被覆盖: %d", config.MaxRetries)
}
if config.Network.ICMPRate != 0.1 {
t.Errorf("显式默认 ICMPRate 被覆盖: %.2f", config.Network.ICMPRate)
}
if config.POC.Num != 20 {
t.Errorf("显式默认 PocNum 被覆盖: %d", config.POC.Num)
}
}
// =============================================================================
// 集成测试:fd limit 约束
// =============================================================================
func TestTuneConfig_FDLimitConstraint(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 600
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 256, NumCPU: 4},
}
ep.TuneConfig(config, session)
// 600 线程 > 256 * 0.6 = 153 → 应该被约束
maxExpected := 256 * 6 / 10
if config.ThreadNum > maxExpected {
t.Errorf("ThreadNum = %d, 应该 <= %d (fd_limit=256)", config.ThreadNum, maxExpected)
}
t.Logf("fd limit 约束: ThreadNum=%d (max=%d)", config.ThreadNum, maxExpected)
}
// =============================================================================
// 集成测试:零样本时不调整
// =============================================================================
func TestTuneConfig_NoSamples(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
origTimeout := config.Timeout
origRetry := config.MaxRetries
origICMP := config.Network.ICMPRate
ep := &EnvironmentProfile{
Net: NetworkProfile{Samples: 0},
System: SystemProfile{FDLimit: 65536},
}
ep.TuneConfig(config, session)
if config.Timeout != origTimeout {
t.Errorf("零样本不应改 Timeout: %v -> %v", origTimeout, config.Timeout)
}
// 零样本时,默认重试降到 2(避免对不可达主机死磕)
if origRetry > 2 && config.MaxRetries != 2 {
t.Errorf("零样本应降 MaxRetries 至 2: %d -> %d", origRetry, config.MaxRetries)
}
if config.Network.ICMPRate != origICMP {
t.Errorf("零样本不应改 ICMPRate: %.2f -> %.2f", origICMP, config.Network.ICMPRate)
}
}
// =============================================================================
// 辅助
// =============================================================================
func makeDefaultConfig() *common.Config {
return &common.Config{
Timeout: 3 * time.Second,
ThreadNum: 600,
ModuleThreadNum: 20,
MaxRetries: 3,
Network: common.NetworkConfig{ICMPRate: 0.1},
POC: common.POCConfig{Num: 20},
Output: common.OutputConfig{LogLevel: "base,info,success"},
}
}
func makeTestSession(config *common.Config) *common.ScanSession {
return common.NewScanSession(config, common.NewState(), &common.FlagVars{})
}
+13
View File
@@ -0,0 +1,13 @@
//go:build !windows
package core
import "syscall"
func getFDLimit() int {
var lim syscall.Rlimit
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim); err != nil {
return 0
}
return int(lim.Cur)
}
+8
View File
@@ -0,0 +1,8 @@
//go:build windows
package core
// Windows 没有 RLIMIT_NOFILE,句柄上限由系统管理
func getFDLimit() int {
return 0
}
+22
View File
@@ -0,0 +1,22 @@
package core
import (
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/parsers"
)
const maxHostBatchSize = 65536
func targetHostBatchSize(config *common.Config) int {
size := parsers.DefaultHostBatchSize
if config != nil && config.ThreadNum > 0 {
threadWindow := config.ThreadNum * 8
if threadWindow > size {
size = threadWindow
}
}
if size > maxHostBatchSize {
return maxHostBatchSize
}
return size
}
+28
View File
@@ -0,0 +1,28 @@
package core
import (
"strings"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/parsers"
)
func loadHostExcludes(params *common.FlagVars) ([]string, error) {
if params == nil {
return nil, nil
}
excludes := make([]string, 0, 1)
if strings.TrimSpace(params.ExcludeHosts) != "" {
excludes = append(excludes, params.ExcludeHosts)
}
if strings.TrimSpace(params.ExcludeHostsFile) == "" {
return excludes, nil
}
lines, err := parsers.ReadLinesFromFile(params.ExcludeHostsFile)
if err != nil {
return nil, err
}
return append(excludes, lines...), nil
}
+29
View File
@@ -0,0 +1,29 @@
package core
import (
"os"
"reflect"
"testing"
"github.com/shadow1ng/fscan/common"
)
func TestLoadHostExcludesIncludesExcludeFile(t *testing.T) {
path := t.TempDir() + "/exclude.txt"
if err := os.WriteFile(path, []byte("192.168.1.2\n# comment\n192.168.1.3\n"), 0o600); err != nil {
t.Fatalf("WriteFile error = %v", err)
}
got, err := loadHostExcludes(&common.FlagVars{
ExcludeHosts: "192.168.1.1",
ExcludeHostsFile: path,
})
if err != nil {
t.Fatalf("loadHostExcludes error = %v", err)
}
want := []string{"192.168.1.1", "192.168.1.2", "192.168.1.3"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("loadHostExcludes = %#v, want %#v", got, want)
}
}
+76 -53
View File
@@ -2,11 +2,13 @@ package core
import (
"bytes"
"context"
"errors"
"fmt"
"net"
"os/exec"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
@@ -39,7 +41,9 @@ var pingErrorKeywords = []string{
// CheckLive 检测主机存活状态
// 支持 ICMP/Ping 探测,并在响应率过低时自动启用 TCP 补充探测
func CheckLive(hostslist []string, Ping bool, config *common.Config, state *common.State) []string {
func CheckLive(ctx context.Context, hostslist []string, Ping bool, session *common.ScanSession) []string {
config := session.Config
state := session.State
// 创建局部WaitGroup
var livewg sync.WaitGroup
@@ -52,7 +56,7 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
chanHosts := make(chan string, len(hostslist))
// 处理存活主机
go handleAliveHosts(chanHosts, hostslist, Ping, &aliveHosts, &aliveHostsMu, existHosts, config, &livewg)
go handleAliveHosts(chanHosts, hostslist, Ping, &aliveHosts, &aliveHostsMu, existHosts, config, session, &livewg)
// 根据Ping参数选择检测方式
if Ping {
@@ -68,7 +72,7 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
// TCP 补充探测:当 ICMP/Ping 响应率过低时自动启用
// 这对防火墙过滤 ICMP 的环境特别有用
aliveHosts = tcpSupplementaryProbe(hostslist, aliveHosts, config)
aliveHosts = tcpSupplementaryProbe(ctx, hostslist, aliveHosts, session)
// 输出存活统计信息
printAliveStats(aliveHosts, hostslist)
@@ -78,7 +82,11 @@ func CheckLive(hostslist []string, Ping bool, config *common.Config, state *comm
// tcpSupplementaryProbe TCP 补充探测
// 当 ICMP 响应率过低时(<10%),对未响应主机进行 TCP 探测
func tcpSupplementaryProbe(allHosts []string, aliveHosts []string, config *common.Config) []string {
func tcpSupplementaryProbe(ctx context.Context, allHosts []string, aliveHosts []string, session *common.ScanSession) []string {
if session.Config.DisableTcpProbe || session.Config.Mode == "icmp" {
return aliveHosts
}
totalHosts := len(allHosts)
if totalHosts == 0 {
return aliveHosts
@@ -99,15 +107,15 @@ func tcpSupplementaryProbe(allHosts []string, aliveHosts []string, config *commo
}
// 提示用户正在进行 TCP 补充探测
common.LogInfo(i18n.Tr("tcp_probe_low_icmp_rate", fmt.Sprintf("%.1f%%", responseRate*100), len(unrespondedHosts)))
session.LogInfo(i18n.Tr("tcp_probe_low_icmp_rate", fmt.Sprintf("%.1f%%", responseRate*100), len(unrespondedHosts)))
// 执行 TCP 补充探测
tcpAliveHosts := runTcpProbeForHosts(unrespondedHosts, config)
tcpAliveHosts := runTcpProbeForHosts(ctx, unrespondedHosts, session)
// 合并结果
if len(tcpAliveHosts) > 0 {
aliveHosts = append(aliveHosts, tcpAliveHosts...)
common.LogInfo(i18n.Tr("tcp_probe_found", len(tcpAliveHosts)))
session.LogInfo(i18n.Tr("tcp_probe_found", len(tcpAliveHosts)))
}
return aliveHosts
@@ -123,7 +131,7 @@ func IsContain(items []string, item string) bool {
return false
}
func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, aliveHosts *[]string, aliveHostsMu *sync.Mutex, existHosts map[string]struct{}, config *common.Config, livewg *sync.WaitGroup) {
func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, aliveHosts *[]string, aliveHostsMu *sync.Mutex, existHosts map[string]struct{}, config *common.Config, session *common.ScanSession, livewg *sync.WaitGroup) {
for ip := range chanHosts {
if _, ok := existHosts[ip]; !ok && IsContain(hostslist, ip) {
existHosts[ip] = struct{}{}
@@ -148,12 +156,9 @@ func handleAliveHosts(chanHosts chan string, hostslist []string, isPing bool, al
"protocol": protocol,
},
}
_ = common.SaveResult(result)
_ = session.SaveResult(result)
// 保留原有的控制台输出
if !config.Output.Silent {
common.LogInfo(i18n.Tr("host_alive", ip, protocol))
}
session.LogInfo(i18n.Tr("host_alive", ip, protocol))
}
livewg.Done()
}
@@ -178,9 +183,6 @@ func probeWithICMP(hostslist []string, chanHosts chan string, aliveHosts *[]stri
return
}
common.LogError(i18n.Tr("icmp_listen_failed", err))
common.LogInfo(i18n.GetText("trying_no_listen_icmp"))
// 尝试无监听ICMP探测
conn2, err := net.DialTimeout("ip4:icmp", "127.0.0.1", 3*time.Second)
if err == nil {
@@ -189,8 +191,6 @@ func probeWithICMP(hostslist []string, chanHosts chan string, aliveHosts *[]stri
return
}
common.LogError(i18n.Tr("icmp_connect_failed", err))
common.LogError(i18n.GetText("insufficient_privileges"))
common.LogInfo(i18n.GetText("switching_to_ping"))
// 降级使用ping探测
@@ -282,13 +282,13 @@ func waitAdaptive(hostslist []string, aliveHosts *[]string, aliveHostsMu *sync.M
// 条件1:所有主机都已响应,立即结束
if aliveCount >= totalHosts {
common.LogDebug(fmt.Sprintf("[ICMP] 全部响应,耗时 %v", elapsed.Round(time.Millisecond)))
common.LogDebug(i18n.Tr("icmp_debug_all_responded", elapsed.Round(time.Millisecond)))
break
}
// 条件2:超过最大等待时间,兜底结束
if elapsed >= maxWait {
common.LogDebug(fmt.Sprintf("[ICMP] 达到最大等待时间 %v,存活 %d/%d", maxWait, aliveCount, totalHosts))
common.LogDebug(i18n.Tr("icmp_debug_max_wait", maxWait, aliveCount, totalHosts))
break
}
@@ -301,8 +301,7 @@ func waitAdaptive(hostslist []string, aliveHosts *[]string, aliveHostsMu *sync.M
lastAliveCount = aliveCount
} else if time.Since(lastChangeTime) >= icmpStableThreshold {
// 连续 500ms 没有新响应,认为响应已稳定,提前结束
common.LogDebug(fmt.Sprintf("[ICMP] 响应稳定,提前结束,耗时 %v,存活 %d/%d",
elapsed.Round(time.Millisecond), aliveCount, totalHosts))
common.LogDebug(i18n.Tr("icmp_debug_stable_done", elapsed.Round(time.Millisecond), aliveCount, totalHosts))
break
}
} else {
@@ -321,8 +320,8 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
var endflag atomic.Bool
var listenerWg sync.WaitGroup
// 创建布隆过滤器用于去重(自动根据主机数量调整大小)
bloomFilter := NewBloomFilter(len(hostslist), 0.01)
// 去重集合:过滤重复的ICMP响应
seen := make(map[string]struct{}, len(hostslist))
// 启动监听协程
listenerWg.Add(1)
@@ -358,11 +357,10 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
if sourceIP != nil && !endflag.Load() {
ipStr := sourceIP.String()
// 使用布隆过滤器去重,过滤重复的ICMP响应和杂包
if bloomFilter.Contains(ipStr) {
if _, dup := seen[ipStr]; dup {
continue
}
bloomFilter.Add(ipStr)
seen[ipStr] = struct{}{}
livewg.Add(1)
select {
@@ -376,13 +374,22 @@ func RunIcmp1(hostslist []string, conn *icmp.PacketConn, chanHosts chan string,
}
}()
// 发送ICMP请求(应用令牌桶限速)
limiter := state.GetICMPLimiter(config.Network.ICMPRate)
// 发送ICMP请求(批量预构建 + 令牌桶限速)
// 预构建所有 ICMP 包和目标地址,减少发送循环中的开销
type icmpPacket struct {
data []byte
dst net.Addr
}
packets := make([]icmpPacket, 0, len(hostslist))
for _, host := range hostslist {
limiter.Wait(1) // 等待令牌,控制发包速率
dst, _ := net.ResolveIPAddr("ip", host)
IcmpByte := makemsg(host)
_, _ = conn.WriteTo(IcmpByte, dst)
dst, _ := common.DNSCache.ResolveIP(host)
packets = append(packets, icmpPacket{data: makemsg(host), dst: dst})
}
limiter := state.GetICMPLimiter(config.Network.ICMPRate)
for i := range packets {
limiter.Wait(1)
_, _ = conn.WriteTo(packets[i].data, packets[i].dst)
}
// 自适应等待响应
@@ -470,8 +477,12 @@ func icmpalive(host string) bool {
// RunPing 使用系统Ping命令并发探测主机存活
func RunPing(hostslist []string, chanHosts chan string, livewg *sync.WaitGroup) {
var wg sync.WaitGroup
// 限制并发数为50
limiter := make(chan struct{}, 50)
// 并发数根据主机数动态调整,上限 200
concurrency := len(hostslist)
if concurrency > 200 {
concurrency = 200
}
limiter := make(chan struct{}, concurrency)
// 并发探测
for _, host := range hostslist {
@@ -674,20 +685,34 @@ func ArrayCountValueTop(arrInit []string, length int, flag bool) (arrTop []strin
var tcpProbeCommonPorts = []int{80, 443, 22, 445}
// tcpProbeTimeout TCP 探测超时时间(较短,只做存活判断)
const tcpProbeTimeout = 2 * time.Second
const tcpProbeTimeout = 1 * time.Second
// tcpProbeThreshold TCP 补充探测触发阈值
// 当 ICMP 响应率低于此值时,自动启用 TCP 补充探测
const tcpProbeThreshold = 0.1 // 10%
// tcpProbeAlive 使用 TCP 探测主机是否存活
// 尝试连接常用端口,任一端口响应即认为存活
func tcpProbeAlive(host string) bool {
// tcpProbeAlive 使用 TCP 并行探测主机是否存活
// 同时连接所有常用端口,任一响应即返回
func tcpProbeAlive(ctx context.Context, session *common.ScanSession, host string) bool {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
result := make(chan bool, len(tcpProbeCommonPorts))
for _, port := range tcpProbeCommonPorts {
addr := fmt.Sprintf("%s:%d", host, port)
conn, err := common.WrapperTcpWithTimeout("tcp", addr, tcpProbeTimeout)
if err == nil {
_ = conn.Close()
go func(p int) {
addr := net.JoinHostPort(host, strconv.Itoa(p))
conn, err := session.DialTCP(ctx, "tcp", addr, tcpProbeTimeout)
if err == nil {
_ = conn.Close()
result <- true
return
}
result <- false
}(port)
}
for range tcpProbeCommonPorts {
if <-result {
return true
}
}
@@ -696,7 +721,7 @@ func tcpProbeAlive(host string) bool {
// runTcpProbeForHosts 对指定主机列表进行 TCP 补充探测
// 返回存活的主机列表
func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
func runTcpProbeForHosts(ctx context.Context, hosts []string, session *common.ScanSession) []string {
if len(hosts) == 0 {
return nil
}
@@ -705,10 +730,10 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
var mu sync.Mutex
aliveHosts := make([]string, 0)
// 并发控制,避免资源耗尽
concurrency := 50
if len(hosts) < concurrency {
concurrency = len(hosts)
// 并发控制,根据主机数动态调整,上限 200
concurrency := len(hosts)
if concurrency > 200 {
concurrency = 200
}
limiter := make(chan struct{}, concurrency)
@@ -722,7 +747,7 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
wg.Done()
}()
if tcpProbeAlive(h) {
if tcpProbeAlive(ctx, session, h) {
mu.Lock()
aliveHosts = append(aliveHosts, h)
mu.Unlock()
@@ -737,11 +762,9 @@ func runTcpProbeForHosts(hosts []string, config *common.Config) []string {
"protocol": "TCP",
},
}
_ = common.SaveResult(result)
_ = session.SaveResult(result)
if !config.Output.Silent {
common.LogInfo(i18n.Tr("host_alive", h, "TCP"))
}
session.LogInfo(i18n.Tr("host_alive", h, "TCP"))
}
}(host)
}
+545
View File
@@ -0,0 +1,545 @@
package core
import (
"sync"
"sync/atomic"
"testing"
"time"
)
// =============================================================================
// 集成测试 1:探测 → 参数调整 → 线程池创建 完整链路
// 验证从 NetworkProfile 到 TuneConfig 到 AdaptivePool 的端到端数据流
// =============================================================================
func TestIntegration_ProbeToPool_LAN(t *testing.T) {
// 模拟内网探测结果
profile := classifyNetwork(
makeDurations([]int{1, 1, 2, 2, 2, 3, 3, 3, 4, 5}), // ms
0, 10,
)
if profile.Env != EnvLAN {
t.Fatalf("探测环境 = %v, want LAN", profile.Env)
}
// 构建 Config + TuneConfig
config := makeDefaultConfig()
session := makeTestSession(config)
sys := ProbeSystem()
ep := &EnvironmentProfile{Net: *profile, System: sys}
ep.TuneConfig(config, session)
// 验证参数被合理调整
if config.Timeout > 3*time.Second {
t.Errorf("内网 Timeout = %v, 不应 > 3s", config.Timeout)
}
if config.MaxRetries != 1 {
t.Errorf("内网零丢包 MaxRetries = %d, want 1", config.MaxRetries)
}
// 用调整后的参数创建线程池
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if pool.Cap() <= 0 {
t.Errorf("池容量 = %d, 应该 > 0", pool.Cap())
}
t.Logf("内网完整链路: Timeout=%v MT=%d Retry=%d ICMP=%.2f target=%d ceiling=%d poolCap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries,
config.Network.ICMPRate, target, ceiling, pool.Cap())
}
func TestIntegration_ProbeToPool_Internet(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 130, 140, 150}),
0, 10,
)
if profile.Env != EnvInternet {
t.Fatalf("探测环境 = %v, want Internet", profile.Env)
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 4096, NumCPU: 4}}
ep.TuneConfig(config, session)
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 公网并发应该明显低于默认 600
if target >= 600 {
t.Errorf("公网 target = %d, 应该 < 600", target)
}
t.Logf("公网完整链路: Timeout=%v MT=%d Retry=%d target=%d ceiling=%d poolCap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, target, ceiling, pool.Cap())
}
// =============================================================================
// 集成测试 2AdaptivePool + ScanMetrics 联动
// 验证:任务执行 → metrics 记录 → 池读取 metrics → 做出调整决策
// =============================================================================
func TestIntegration_PoolMetrics_HealthyTraffic(t *testing.T) {
metrics := &ScanMetrics{}
var taskCount atomic.Int64
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {
taskCount.Add(1)
}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 注入健康 metrics
for i := 0; i < 200; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 运行任务
var wg sync.WaitGroup
for i := 0; i < 200; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_ = pool.Invoke(nil)
}()
}
wg.Wait()
pool.Wait()
// 触发调整
pool.lastCheck.Store(0)
pool.adjust()
if pool.Cap() < 90 {
t.Errorf("健康流量池容量不应大幅下降: cap = %d", pool.Cap())
}
t.Logf("健康流量: tasks=%d connects=%d cap=%d",
taskCount.Load(), metrics.Snapshot().Connects, pool.Cap())
}
func TestIntegration_PoolMetrics_ExhaustedTraffic(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 直接向 metrics 注入大量资源耗尽事件(模拟扫描过程中的 fd 不足)
for i := 0; i < 200; i++ {
metrics.RecordExhausted()
}
// 手动触发调整(清除时间守卫)
pool.lastCheck.Store(0)
pool.adjust()
// 资源耗尽率 100% → 应该降速
if pool.Cap() >= 100 {
t.Errorf("资源耗尽后池应该降速: cap = %d", pool.Cap())
}
t.Logf("资源耗尽: exhausted=%d cap=%d", metrics.Snapshot().Exhausted, pool.Cap())
}
// =============================================================================
// 集成测试 3:慢启动 → 稳态 AIMD 过渡
// 验证慢启动阶段的翻倍行为和过渡到稳态的时机
// =============================================================================
func TestIntegration_SlowStartToSteady(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {
metrics.RecordConnect(time.Millisecond)
}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if !pool.inSlowStart {
t.Fatal("初始应该在慢启动状态")
}
initialCap := pool.Cap()
t.Logf("慢启动初始: cap=%d", initialCap)
// 喂入足够的健康 metrics
for i := 0; i < 100; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 模拟多次调整周期
caps := []int{initialCap}
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0) // 强制触发检查
pool.adjust()
caps = append(caps, pool.Cap())
}
// 验证:容量应该逐步增长
growing := false
for i := 1; i < len(caps); i++ {
if caps[i] > caps[i-1] {
growing = true
break
}
}
if !growing {
t.Errorf("慢启动期间容量没有增长: %v", caps)
}
// 最终应该退出慢启动
finalCap := pool.Cap()
if finalCap < initialCap {
t.Errorf("最终容量 %d < 初始 %d, 不合理", finalCap, initialCap)
}
t.Logf("慢启动过渡: %v, inSlowStart=%v", caps, pool.inSlowStart)
}
// =============================================================================
// 集成测试 4:拥塞 → 降速 → 恢复 完整周期
// =============================================================================
func TestIntegration_CongestionRecovery(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 200, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 直接到稳态,满容量
pool.inSlowStart = false
pool.tune(200)
// === 阶段 1: 正常运行 ===
for i := 0; i < 100; i++ {
metrics.RecordConnect(time.Millisecond)
}
pool.lastCheck.Store(0)
pool.adjust()
normalCap := pool.Cap()
t.Logf("正常阶段: cap=%d", normalCap)
// === 阶段 2: 突发拥塞(大量资源耗尽)===
for i := 0; i < 200; i++ {
metrics.RecordExhausted()
}
pool.lastCheck.Store(0)
pool.adjust()
congestedCap := pool.Cap()
if congestedCap >= normalCap {
t.Errorf("拥塞后应降速: normal=%d congested=%d", normalCap, congestedCap)
}
t.Logf("拥塞阶段: cap=%d (降幅 %d%%)", congestedCap, (normalCap-congestedCap)*100/normalCap)
// === 阶段 3: 恢复(大量成功连接)===
for i := 0; i < 500; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 多次调整模拟恢复过程
for i := 0; i < 20; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
recoveredCap := pool.Cap()
if recoveredCap <= congestedCap {
t.Errorf("恢复后应提速: congested=%d recovered=%d", congestedCap, recoveredCap)
}
// 恢复后不应超过 ceiling
if recoveredCap > 200 {
t.Errorf("恢复后不应超过 ceiling: cap=%d ceiling=200", recoveredCap)
}
t.Logf("恢复阶段: cap=%d", recoveredCap)
}
// =============================================================================
// 集成测试 5:RTT 趋势检测 → 池调整
// 验证 ScanMetrics 的 RTT EMA 趋势信号能正确传导到池的健康判断
// =============================================================================
func TestIntegration_RTTTrend_DrivesPoolAdjustment(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(i interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 建立基线:100 个 5ms RTT
for i := 0; i < 200; i++ {
metrics.RecordConnect(5 * time.Millisecond)
}
pool.lastCheck.Store(0)
pool.adjust()
baselineCap := pool.Cap()
// RTT 突增到 100ms20 倍)
for i := 0; i < 100; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
ratio := metrics.RTTRatio()
if ratio <= 1.0 {
t.Logf("RTT ratio = %.2f, EMA 可能还没追上(正常)", ratio)
}
// 多次调整看池是否响应
for i := 0; i < 5; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
afterRTTSpike := pool.Cap()
t.Logf("RTT 趋势: baseline_cap=%d after_spike=%d rtt_ratio=%.2f",
baselineCap, afterRTTSpike, ratio)
// 如果 ratio 足够高,池应该降速
if ratio > 2.0 && afterRTTSpike >= baselineCap {
t.Errorf("RTT ratio=%.2f 但池没有降速: %d -> %d", ratio, baselineCap, afterRTTSpike)
}
}
// =============================================================================
// 集成测试 6:不同网络环境下的参数一致性
// 验证同一组目标在不同环境下参数调整的合理递进关系
// =============================================================================
func TestIntegration_ParameterProgression(t *testing.T) {
environments := []struct {
name string
rtts []int // ms
loss int // failures out of 10
wantEnv NetworkEnv
}{
{"内网", []int{1, 1, 2, 2, 3, 3, 4, 4, 5, 5}, 0, EnvLAN},
{"局域网", []int{10, 15, 20, 25, 30, 35, 40, 45, 48, 49}, 0, EnvWAN},
{"公网", []int{60, 70, 80, 90, 100, 120, 140, 160, 180, 195}, 0, EnvInternet},
{"慢速", []int{200, 300, 400, 500, 600, 700, 800, 900, 1000, 1500}, 0, EnvSlow},
}
type params struct {
timeout time.Duration
mt int
retry int
icmpRate float64
}
var results []params
for _, env := range environments {
profile := classifyNetwork(makeDurations(env.rtts), env.loss, 10)
if profile.Env != env.wantEnv {
t.Errorf("%s: env = %v, want %v", env.name, profile.Env, env.wantEnv)
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
results = append(results, params{
timeout: config.Timeout,
mt: config.ModuleThreadNum,
retry: config.MaxRetries,
icmpRate: config.Network.ICMPRate,
})
t.Logf("%s: Timeout=%v MT=%d Retry=%d ICMP=%.2f",
env.name, config.Timeout, config.ModuleThreadNum, config.MaxRetries, config.Network.ICMPRate)
}
// 验证递进关系:从内网到慢速,Timeout 应递增
for i := 1; i < len(results); i++ {
if results[i].timeout < results[i-1].timeout {
t.Errorf("Timeout 不递增: %v (env[%d]) < %v (env[%d])",
results[i].timeout, i, results[i-1].timeout, i-1)
}
}
// ICMPRate 应递减(内网最高,慢速最低)
for i := 1; i < len(results); i++ {
if results[i].icmpRate > results[i-1].icmpRate {
t.Errorf("ICMPRate 不递减: %.2f (env[%d]) > %.2f (env[%d])",
results[i].icmpRate, i, results[i-1].icmpRate, i-1)
}
}
}
// =============================================================================
// 集成测试 7:用户显式 -t + 网络探测 完整流程
// 验证用户指定值作为 ceiling 但探测仍然影响其他参数
// =============================================================================
func TestIntegration_ExplicitThreadNum_WithProbe(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{100, 120, 140, 160, 180, 200, 220, 240, 260, 300}),
2, 12, // 部分丢包
)
config := makeDefaultConfig()
config.ThreadNum = 200
config.ThreadNumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 4096, NumCPU: 4},
}
ep.TuneConfig(config, session)
// ThreadNum 不应被修改(fd limit 允许范围内)
// 但 Timeout、ModuleThreadNum 等应根据探测调整
if config.Timeout == 3*time.Second {
t.Error("即使 -t 显式,Timeout 仍应根据探测调整")
}
// 创建池
target, ceiling := profile.RecommendConcurrency(config.ThreadNum, config.ThreadNumExplicit)
if ceiling != 200 {
t.Errorf("显式 -t 200 的 ceiling = %d, want 200", ceiling)
}
if target > 200 {
t.Errorf("target = %d, 不应超过 ceiling 200", target)
}
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
t.Logf("显式 -t 200: Timeout=%v MT=%d Retry=%d target=%d ceiling=%d cap=%d",
config.Timeout, config.ModuleThreadNum, config.MaxRetries, target, ceiling, pool.Cap())
}
// =============================================================================
// 集成测试 8AdaptiveTimeout + ScanMetrics 双 RTT 追踪
// 验证两个 RTT 追踪器独立工作不干扰
// =============================================================================
func TestIntegration_DualRTTTracking(t *testing.T) {
adaptiveTO := NewAdaptiveTimeout(3 * time.Second)
metrics := &ScanMetrics{}
// 喂入相同的 RTT 数据到两个追踪器
for i := 0; i < 50; i++ {
rtt := 10 * time.Millisecond
adaptiveTO.Record(rtt)
metrics.RecordConnect(rtt)
}
// AdaptiveTimeout 用于连接超时
toValue := adaptiveTO.Timeout()
// ScanMetrics 用于池健康判断
rttFast := metrics.RTTFast()
ratio := metrics.RTTRatio()
if toValue > 3*time.Second {
t.Errorf("AdaptiveTimeout 应该 < 初始值: %v", toValue)
}
if rttFast < 8*time.Millisecond || rttFast > 12*time.Millisecond {
t.Errorf("ScanMetrics RTTFast 应接近 10ms: %v", rttFast)
}
if ratio < 0.8 || ratio > 1.2 {
t.Errorf("稳定 RTT 的 ratio 应接近 1.0: %.2f", ratio)
}
t.Logf("双追踪: AdaptiveTO=%v, MetricsFast=%v, Ratio=%.2f", toValue, rttFast, ratio)
}
// =============================================================================
// 集成测试 9:丢包环境下 Retry + ModuleThreadNum 联动
// 验证高丢包同时影响重试和并发
// =============================================================================
func TestIntegration_LossyNetwork_RetryAndConcurrency(t *testing.T) {
lossRates := []float64{0.0, 0.05, 0.10, 0.20, 0.40}
type result struct {
loss float64
retry int
mt int
}
var results []result
for _, loss := range lossRates {
profile := &NetworkProfile{
Env: EnvInternet,
RTTMedian: 80 * time.Millisecond,
RTTStddev: 20 * time.Millisecond,
LossRate: loss,
Samples: 20,
}
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: *profile,
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
results = append(results, result{loss, config.MaxRetries, config.ModuleThreadNum})
}
// 重试次数应随丢包率单调递增
for i := 1; i < len(results); i++ {
if results[i].retry < results[i-1].retry {
t.Errorf("Retry 不递增: loss=%.2f retry=%d < loss=%.2f retry=%d",
results[i].loss, results[i].retry, results[i-1].loss, results[i-1].retry)
}
}
// 高丢包时 ModuleThreadNum 应降低
if results[len(results)-1].mt >= results[0].mt {
t.Errorf("40%%丢包的 MT(%d) 应 < 0%%丢包的 MT(%d)",
results[len(results)-1].mt, results[0].mt)
}
for _, r := range results {
t.Logf("loss=%.0f%%: Retry=%d MT=%d", r.loss*100, r.retry, r.mt)
}
}
+13 -10
View File
@@ -1,6 +1,7 @@
package core
import (
"context"
"sync"
"github.com/shadow1ng/fscan/common"
@@ -16,17 +17,17 @@ type LocalScanStrategy struct {
// NewLocalScanStrategy 创建新的本地扫描策略
func NewLocalScanStrategy() *LocalScanStrategy {
return &LocalScanStrategy{
BaseScanStrategy: NewBaseScanStrategy("本地扫描", FilterLocal),
BaseScanStrategy: NewBaseScanStrategy(i18n.GetText("scan_strategy_local_name"), FilterLocal),
}
}
// LogPluginInfo 重写以只显示通过-local指定的插件
func (s *LocalScanStrategy) LogPluginInfo(config *common.Config) {
func (s *LocalScanStrategy) LogPluginInfo(config *common.Config, session *common.ScanSession) {
localPlugin := config.LocalPlugin
if localPlugin != "" {
common.LogInfo(i18n.Tr("local_plugin_info", localPlugin))
session.LogInfo(i18n.Tr("local_plugin_info", localPlugin))
} else {
common.LogError(i18n.GetText("local_plugin_not_specified"))
session.LogError(i18n.GetText("local_plugin_not_specified"))
}
}
@@ -41,32 +42,34 @@ func (s *LocalScanStrategy) Description() string {
}
// Execute 执行本地扫描策略
func (s *LocalScanStrategy) Execute(config *common.Config, state *common.State, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
func (s *LocalScanStrategy) Execute(ctx context.Context, session *common.ScanSession, info common.HostInfo, ch chan struct{}, wg *sync.WaitGroup) {
config := session.Config
// 输出扫描开始信息
s.LogScanStart()
s.LogScanStart(session)
// 验证插件配置
if err := s.ValidateConfiguration(); err != nil {
common.LogError(err.Error())
session.LogError(err.Error())
return
}
// 验证本地插件是否存在
if config.LocalPlugin != "" {
if !plugins.Exists(config.LocalPlugin) {
common.LogError(i18n.Tr("local_plugin_not_found", config.LocalPlugin))
session.LogError(i18n.Tr("local_plugin_not_found", config.LocalPlugin))
return
}
}
// 输出插件信息
s.LogPluginInfo(config)
s.LogPluginInfo(config, session)
// 准备目标(本地扫描通常只有一个目标,即本机)
targets := s.PrepareTargets(info)
// 执行扫描任务
ExecuteScanTasks(config, state, targets, s, ch, wg)
ExecuteScanTasks(ctx, session, targets, s, ch, wg)
}
// PrepareTargets 准备本地扫描目标
+281
View File
@@ -0,0 +1,281 @@
package core
import (
"context"
"math"
"net"
"sort"
"strconv"
"sync"
"time"
"github.com/shadow1ng/fscan/common"
"github.com/shadow1ng/fscan/common/i18n"
)
// NetworkEnv 网络环境分类
type NetworkEnv int
const (
EnvLAN NetworkEnv = iota // 内网: RTT < 5ms, 丢包 < 1%
EnvWAN // 局域网/专线: RTT 5~50ms, 丢包 < 5%
EnvInternet // 公网: RTT 50~200ms
EnvSlow // 慢速/高丢包: RTT > 200ms 或 丢包 > 10%
)
func (e NetworkEnv) String() string {
switch e {
case EnvLAN:
return i18n.GetText("net_env_lan")
case EnvWAN:
return i18n.GetText("net_env_wan")
case EnvInternet:
return i18n.GetText("net_env_internet")
default:
return i18n.GetText("net_env_slow")
}
}
// NetworkProfile 网络探测结果
type NetworkProfile struct {
Env NetworkEnv
RTTMin time.Duration
RTTMedian time.Duration
RTTP95 time.Duration
RTTStddev time.Duration
LossRate float64
Samples int
}
// RecommendConcurrency 根据探测结果推荐并发参数
// 返回 (target, ceiling)
// - target: 推荐的目标并发数
// - ceiling: 允许的最大并发数
//
// 如果用户显式指定了 -tceiling = 用户值,target 取 min(推荐值, 用户值)
// 如果用户未指定,target 和 ceiling 均为推荐值
func (p *NetworkProfile) RecommendConcurrency(userThreadNum int, explicit bool) (target, ceiling int) {
// 基于网络环境的缩放因子
var factor float64
switch p.Env {
case EnvLAN:
factor = 1.5
case EnvWAN:
factor = 1.0
case EnvInternet:
factor = 0.4
case EnvSlow:
factor = 0.15
}
recommended := int(float64(userThreadNum) * factor)
if recommended < 10 {
recommended = 10
}
// 丢包率高时进一步压缩
if p.LossRate > 0.05 {
recommended = int(float64(recommended) * (1.0 - p.LossRate))
if recommended < 10 {
recommended = 10
}
}
if explicit {
ceiling = userThreadNum
target = recommended
if target > ceiling {
target = ceiling
}
} else {
target = recommended
ceiling = recommended
}
return
}
// probePorts 探测用的端口列表(高响应率的常见端口)
var probePorts = []int{80, 443, 22, 445, 8080, 3389, 21, 8443}
func networkProbeAddress(host string, port int) string {
return net.JoinHostPort(host, strconv.Itoa(port))
}
// ProbeNetwork 探测目标网络环境
// 从 hosts 中抽样,用低并发 TCP 连接测量 RTT 和丢包率
// 整个过程控制在数秒内完成
func ProbeNetwork(ctx context.Context, hosts []string, session *common.ScanSession) *NetworkProfile {
if len(hosts) == 0 {
return defaultProfile()
}
// 抽样:均匀分布,最多 10 个
samples := pickSamples(hosts, 10)
probeTimeout := session.Config.Timeout
if probeTimeout > time.Second {
probeTimeout = time.Second
}
if probeTimeout < 500*time.Millisecond {
probeTimeout = 500 * time.Millisecond
}
var (
mu sync.Mutex
rtts []time.Duration
failures int
total int
)
sem := make(chan struct{}, 10)
var wg sync.WaitGroup
for _, host := range samples {
for _, port := range probePorts {
select {
case <-ctx.Done():
goto done
default:
}
total++
wg.Add(1)
sem <- struct{}{}
go func(h string, p int) {
defer func() { <-sem; wg.Done() }()
addr := networkProbeAddress(h, p)
start := time.Now()
conn, err := session.DialTCP(ctx, "tcp", addr, probeTimeout)
rtt := time.Since(start)
mu.Lock()
defer mu.Unlock()
if err != nil {
// 连接拒绝也是有效的 RTT 样本(说明对端可达)
if isConnectionRefused(err) {
rtts = append(rtts, rtt)
}
failures++
} else {
_ = conn.Close()
rtts = append(rtts, rtt)
}
}(host, port)
}
}
done:
wg.Wait()
return classifyNetwork(rtts, failures, total)
}
func classifyNetwork(rtts []time.Duration, failures, total int) *NetworkProfile {
if len(rtts) == 0 {
return defaultProfile()
}
sort.Slice(rtts, func(i, j int) bool { return rtts[i] < rtts[j] })
n := len(rtts)
median := rtts[n/2]
p95idx := int(float64(n) * 0.95)
if p95idx >= n {
p95idx = n - 1
}
p95 := rtts[p95idx]
// 标准差
var sum float64
for _, r := range rtts {
sum += float64(r)
}
mean := sum / float64(n)
var variance float64
for _, r := range rtts {
d := float64(r) - mean
variance += d * d
}
stddev := time.Duration(math.Sqrt(variance / float64(n)))
// 丢包率:只计算超时的(非 refused),但简化为 1 - 有效响应数/总数
lossRate := 1.0 - float64(n)/float64(total)
if lossRate < 0 {
lossRate = 0
}
// 分类
env := classifyEnv(median, lossRate)
return &NetworkProfile{
Env: env,
RTTMin: rtts[0],
RTTMedian: median,
RTTP95: p95,
RTTStddev: stddev,
LossRate: lossRate,
Samples: n,
}
}
func classifyEnv(median time.Duration, lossRate float64) NetworkEnv {
switch {
case lossRate > 0.10:
return EnvSlow
case median < 5*time.Millisecond && lossRate < 0.01:
return EnvLAN
case median < 50*time.Millisecond && lossRate < 0.05:
return EnvWAN
case median < 200*time.Millisecond:
return EnvInternet
default:
return EnvSlow
}
}
func defaultProfile() *NetworkProfile {
return &NetworkProfile{
Env: EnvWAN,
RTTMedian: 10 * time.Millisecond,
LossRate: 0,
Samples: 0,
}
}
// pickSamples 均匀抽样
func pickSamples(hosts []string, maxSamples int) []string {
if maxSamples <= 0 {
return nil
}
n := len(hosts)
if n <= maxSamples {
return hosts
}
step := n / maxSamples
samples := make([]string, 0, maxSamples)
for i := 0; i < n && len(samples) < maxSamples; i += step {
samples = append(samples, hosts[i])
}
return samples
}
func isConnectionRefused(err error) bool {
if err == nil {
return false
}
// connection refused 通常包含 "refused" 关键词
// 在不同 OS 上表现一致
return containsFold(err.Error(), "refused")
}
// isTimeoutError 判断是否为超时错误
func isTimeoutError(err error) bool {
if err == nil {
return false
}
if ne, ok := err.(net.Error); ok {
return ne.Timeout()
}
return containsFold(err.Error(), "timeout") || containsFold(err.Error(), "deadline")
}
+187
View File
@@ -0,0 +1,187 @@
package core
import (
"testing"
"time"
)
// =============================================================================
// 单元测试:classifyEnv — 网络环境分类
// =============================================================================
func TestClassifyEnv(t *testing.T) {
tests := []struct {
median time.Duration
lossRate float64
wantEnv NetworkEnv
desc string
}{
{1 * time.Millisecond, 0.0, EnvLAN, "1ms 零丢包 → 内网"},
{3 * time.Millisecond, 0.005, EnvLAN, "3ms 0.5%丢包 → 内网"},
{5 * time.Millisecond, 0.0, EnvWAN, "5ms 零丢包 → 局域网边界"},
{20 * time.Millisecond, 0.02, EnvWAN, "20ms 2%丢包 → 局域网"},
{50 * time.Millisecond, 0.03, EnvInternet, "50ms 3%丢包 → 公网边界"},
{100 * time.Millisecond, 0.05, EnvInternet, "100ms 5%丢包 → 公网"},
{300 * time.Millisecond, 0.05, EnvSlow, "300ms → 慢速"},
{50 * time.Millisecond, 0.15, EnvSlow, "50ms 15%丢包 → 高丢包归类慢速"},
{1 * time.Millisecond, 0.20, EnvSlow, "低延迟但高丢包 → 慢速"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
got := classifyEnv(tt.median, tt.lossRate)
if got != tt.wantEnv {
t.Errorf("classifyEnv(median=%v, loss=%.2f) = %v, want %v",
tt.median, tt.lossRate, got, tt.wantEnv)
}
})
}
}
// =============================================================================
// 单元测试:classifyNetwork — 从 RTT 样本推导 profile
// =============================================================================
func TestClassifyNetwork(t *testing.T) {
t.Run("内网 RTT 分布", func(t *testing.T) {
rtts := makeDurations([]int{1, 1, 1, 2, 2, 2, 3, 3, 4, 5}) // ms
p := classifyNetwork(rtts, 0, 10)
if p.Env != EnvLAN {
t.Errorf("env = %v, want LAN", p.Env)
}
if p.RTTMedian > 5*time.Millisecond {
t.Errorf("median = %v, want < 5ms", p.RTTMedian)
}
if p.LossRate != 0 {
t.Errorf("lossRate = %.2f, want 0", p.LossRate)
}
})
t.Run("公网 RTT 分布(低丢包)", func(t *testing.T) {
rtts := makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 150, 200, 300}) // ms
p := classifyNetwork(rtts, 0, 10) // 无丢包
if p.Env != EnvInternet {
t.Errorf("env = %v, want Internet", p.Env)
}
if p.LossRate != 0 {
t.Errorf("lossRate = %.2f, want 0", p.LossRate)
}
})
t.Run("高丢包归类为慢速", func(t *testing.T) {
rtts := makeDurations([]int{60, 70, 80, 90, 100}) // ms, 5 responded
p := classifyNetwork(rtts, 5, 10) // 50% loss
if p.Env != EnvSlow {
t.Errorf("env = %v, want Slow (高丢包)", p.Env)
}
})
t.Run("零样本降级", func(t *testing.T) {
p := classifyNetwork(nil, 5, 5)
if p.Env != EnvWAN {
t.Errorf("env = %v, want WAN (default)", p.Env)
}
if p.Samples != 0 {
t.Errorf("samples = %d, want 0", p.Samples)
}
})
}
// =============================================================================
// 单元测试:RecommendConcurrency
// =============================================================================
func TestRecommendConcurrency(t *testing.T) {
tests := []struct {
env NetworkEnv
lossRate float64
userT int
explicit bool
wantTMin int
wantTMax int
wantCeil int
desc string
}{
{EnvLAN, 0.0, 600, false, 800, 1000, -1, "内网自动: ×1.5"},
{EnvWAN, 0.0, 600, false, 550, 650, -1, "局域网自动: ×1.0"},
{EnvInternet, 0.0, 600, false, 200, 280, -1, "公网自动: ×0.4"},
{EnvSlow, 0.0, 600, false, 80, 100, -1, "慢速自动: ×0.15"},
{EnvInternet, 0.0, 200, true, 70, 100, 200, "公网显式: target<ceiling"},
{EnvLAN, 0.0, 100, true, 100, 160, 100, "内网显式: ceiling=用户值"},
{EnvInternet, 0.15, 600, false, 170, 240, -1, "公网高丢包: 进一步压缩"},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
p := &NetworkProfile{Env: tt.env, LossRate: tt.lossRate, Samples: 10}
target, ceiling := p.RecommendConcurrency(tt.userT, tt.explicit)
if target < tt.wantTMin || target > tt.wantTMax {
t.Errorf("target = %d, want [%d, %d]", target, tt.wantTMin, tt.wantTMax)
}
if tt.explicit && ceiling != tt.wantCeil {
t.Errorf("ceiling = %d, want %d", ceiling, tt.wantCeil)
}
})
}
}
// =============================================================================
// 单元测试:pickSamples
// =============================================================================
func TestPickSamples(t *testing.T) {
hosts := make([]string, 100)
for i := range hosts {
hosts[i] = "host"
}
s := pickSamples(hosts, 10)
if len(s) != 10 {
t.Errorf("pickSamples(100, 10) = %d items, want 10", len(s))
}
s = pickSamples(hosts[:5], 10)
if len(s) != 5 {
t.Errorf("pickSamples(5, 10) = %d items, want 5", len(s))
}
s = pickSamples(nil, 10)
if len(s) != 0 {
t.Errorf("pickSamples(nil, 10) = %d items, want 0", len(s))
}
}
func TestNetworkProbeAddressUsesJoinHostPort(t *testing.T) {
tests := []struct {
host string
port int
want string
}{
{"127.0.0.1", 80, "127.0.0.1:80"},
{"::1", 443, "[::1]:443"},
{"2001:db8::1", 22, "[2001:db8::1]:22"},
}
for _, tt := range tests {
if got := networkProbeAddress(tt.host, tt.port); got != tt.want {
t.Fatalf("networkProbeAddress(%q, %d) = %q, want %q", tt.host, tt.port, got, tt.want)
}
}
}
// =============================================================================
// 辅助
// =============================================================================
func makeDurations(ms []int) []time.Duration {
ds := make([]time.Duration, len(ms))
for i, m := range ms {
ds[i] = time.Duration(m) * time.Millisecond
}
return ds
}
+549
View File
@@ -0,0 +1,549 @@
package core
import (
"sync/atomic"
"testing"
"time"
)
// =============================================================================
// 优化 1target/ceiling 分离
// =============================================================================
func TestOpt1_TargetCeilingSeparation_TuneConfig(t *testing.T) {
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
if config.ThreadCeiling <= 0 {
t.Fatalf("ThreadCeiling 未被设置: %d", config.ThreadCeiling)
}
// 内网 factor=1.5,非显式 → target=ceiling=recommended
// 但 ceiling 应该 >= target
if config.ThreadCeiling < config.ThreadNum {
t.Errorf("Ceiling(%d) < ThreadNum(%d)", config.ThreadCeiling, config.ThreadNum)
}
t.Logf("target=%d, ceiling=%d", config.ThreadNum, config.ThreadCeiling)
}
func TestOpt1_TargetCeilingSeparation_ExplicitT(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 200
config.ThreadNumExplicit = true
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvInternet,
RTTMedian: 100 * time.Millisecond,
RTTStddev: 30 * time.Millisecond,
LossRate: 0.0,
Samples: 20,
},
System: SystemProfile{FDLimit: 65536, NumCPU: 8},
}
ep.TuneConfig(config, session)
// 用户显式指定 -t → ceiling = threadNum = 200
if config.ThreadCeiling != 200 {
t.Errorf("显式 -t 200: ceiling=%d, want 200", config.ThreadCeiling)
}
if config.ThreadNum != 200 {
t.Errorf("显式 -t 200: threadNum=%d, want 200", config.ThreadNum)
}
}
func TestOpt1_PoolUsesCeiling(t *testing.T) {
metrics := &ScanMetrics{}
target, ceiling := 50, 200
pool, err := NewAdaptivePool(target, ceiling, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(target)
// 注入健康 metrics 让池增长
for i := 0; i < 200; i++ {
metrics.RecordConnect(time.Millisecond)
}
// 多次 adjust,池应能增长超过 target 但不超过 ceiling
for i := 0; i < 30; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
finalCap := pool.Cap()
if finalCap <= target {
t.Errorf("池应能超过 target(%d): cap=%d", target, finalCap)
}
if finalCap > ceiling {
t.Errorf("池不应超过 ceiling(%d): cap=%d", ceiling, finalCap)
}
t.Logf("target=%d, ceiling=%d, finalCap=%d", target, ceiling, finalCap)
}
func TestOpt1_FDLimitConstraintsBothFields(t *testing.T) {
config := makeDefaultConfig()
config.ThreadNum = 1000
session := makeTestSession(config)
ep := &EnvironmentProfile{
Net: NetworkProfile{
Env: EnvLAN,
RTTMedian: 1 * time.Millisecond,
RTTStddev: 500 * time.Microsecond,
LossRate: 0.0,
Samples: 30,
},
System: SystemProfile{FDLimit: 256, NumCPU: 4},
}
ep.TuneConfig(config, session)
maxFD := 256 * 6 / 10
if config.ThreadNum > maxFD {
t.Errorf("ThreadNum(%d) 超过 fd 限制(%d)", config.ThreadNum, maxFD)
}
if config.ThreadCeiling > maxFD {
t.Errorf("ThreadCeiling(%d) 超过 fd 限制(%d)", config.ThreadCeiling, maxFD)
}
}
// =============================================================================
// 优化 2RTT 漂移微调 target
// =============================================================================
func TestOpt2_RTTDriftReducesTarget(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 400, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(200)
// 建立基线:slow EMA 锚定在 1ms 附近
for i := 0; i < 500; i++ {
metrics.RecordConnect(1 * time.Millisecond)
}
origTarget := atomic.LoadInt32(&pool.target)
// RTT 突增到 100ms100 倍),大量喂入让 fast EMA 拉开差距
for i := 0; i < 1000; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
ratio := metrics.RTTRatio()
t.Logf("RTT ratio after spike: %.2f", ratio)
if ratio <= 3.0 {
t.Skipf("RTT ratio=%.2fEMA 差距不够大,跳过", ratio)
}
// 需要足够的新 metrics 让 assessHealth 的 deltaTotal >= 30
for i := 0; i < 50; i++ {
metrics.RecordConnect(100 * time.Millisecond)
}
// 多次 adjust 触发 maybeReduceTarget
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0)
pool.prevSnapshot = MetricsSnapshot{} // 重置快照让 delta 足够
pool.adjust()
}
newTarget := atomic.LoadInt32(&pool.target)
if newTarget >= origTarget {
t.Errorf("RTT 漂移后 target 应降低: %d -> %d (ratio=%.2f)", origTarget, newTarget, ratio)
}
// 不应低于 ceiling/5
minTarget := atomic.LoadInt32(&pool.ceiling) / 5
if newTarget < minTarget {
t.Errorf("target(%d) 低于下限(%d)", newTarget, minTarget)
}
t.Logf("RTT drift: ratio=%.2f, target %d -> %d (min=%d)", ratio, origTarget, newTarget, minTarget)
}
func TestOpt2_NoReductionWhenStable(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(200, 400, func(interface{}) {}, metrics)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(200)
// 稳定 RTT
for i := 0; i < 200; i++ {
metrics.RecordConnect(10 * time.Millisecond)
}
origTarget := atomic.LoadInt32(&pool.target)
for i := 0; i < 10; i++ {
pool.lastCheck.Store(0)
pool.adjust()
}
newTarget := atomic.LoadInt32(&pool.target)
if newTarget != origTarget {
t.Errorf("稳定 RTT 不应改变 target: %d -> %d", origTarget, newTarget)
}
}
// =============================================================================
// 优化 3assessHealth 阈值跟 NetworkEnv 关联
// =============================================================================
func TestOpt3_LANTighterThresholds(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, EnvLAN)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 10% exhaust rate — 对 LAN 来说应该是 Congested(阈值 8%
for i := 0; i < 100; i++ {
if i < 10 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
if pool.Cap() >= 100 {
t.Errorf("LAN 10%% exhaust 应触发降速: cap=%d", pool.Cap())
}
t.Logf("LAN tight threshold: cap=%d (from 100)", pool.Cap())
}
func TestOpt3_InternetLooseThresholds(t *testing.T) {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, EnvInternet)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
pool.inSlowStart = false
pool.tune(100)
// 10% exhaust rate — 对 Internet 来说不算 Congested(阈值 25%),应是 Stressed
for i := 0; i < 100; i++ {
if i < 10 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(10 * time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
capAfter := pool.Cap()
// Internet 对 10% exhaust 只是 Stressed(×0.85),不是 Congested(×0.5
if capAfter < 80 {
t.Errorf("Internet 10%% exhaust 不应大幅降速: cap=%d", capAfter)
}
t.Logf("Internet loose threshold: cap=%d (from 100)", capAfter)
}
func TestOpt3_EnvAffectsHealthDecision(t *testing.T) {
envs := []struct {
env NetworkEnv
name string
}{
{EnvLAN, "LAN"},
{EnvWAN, "WAN"},
{EnvInternet, "Internet"},
}
var caps []int
for _, e := range envs {
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(100, 100, func(interface{}) {}, metrics, e.env)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
pool.inSlowStart = false
pool.tune(100)
// 相同的 12% exhaust rate
for i := 0; i < 100; i++ {
if i < 12 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
caps = append(caps, pool.Cap())
pool.Release()
t.Logf("%s: cap=%d (12%% exhaust)", e.name, caps[len(caps)-1])
}
// LAN 反应最激烈(cap 最低),Internet 最宽容(cap 最高)
if caps[0] >= caps[2] {
t.Errorf("LAN cap(%d) 应 < Internet cap(%d) for same exhaust rate", caps[0], caps[2])
}
}
// =============================================================================
// 优化 4:去掉 semaphoreants 池天然反压
// =============================================================================
func TestOpt4_SemaphoreRemoved(t *testing.T) {
// 验证 portScanTask 结构体不再有 semaphore 字段
// 如果 semaphore 被加回来,这段代码编译就会报 "unknown field"
_ = portScanTask{
host: "127.0.0.1",
port: 80,
addr: "127.0.0.1:80",
}
t.Log("portScanTask 无 semaphore 字段,反压由 ants pool 统一管理")
}
// =============================================================================
// 优化 5:扩充探测端口
// =============================================================================
func TestOpt5_ProbePortsExpanded(t *testing.T) {
if len(probePorts) < 5 {
t.Errorf("probePorts 只有 %d 个,应该扩充到至少 5 个", len(probePorts))
}
// 验证包含关键端口
required := map[int]bool{80: false, 443: false, 22: false}
for _, p := range probePorts {
if _, ok := required[p]; ok {
required[p] = true
}
}
for port, found := range required {
if !found {
t.Errorf("probePorts 缺少关键端口 %d", port)
}
}
// 验证没有重复
seen := make(map[int]bool)
for _, p := range probePorts {
if seen[p] {
t.Errorf("probePorts 有重复端口 %d", p)
}
seen[p] = true
}
t.Logf("probePorts = %v (%d 个)", probePorts, len(probePorts))
}
// =============================================================================
// 优化 6computeRetries 环境自适应
// =============================================================================
func TestOpt6_RetriesEnvAware(t *testing.T) {
lossRate := 0.3 // 30% 丢包
lanRetry := computeRetries(lossRate, EnvLAN)
wanRetry := computeRetries(lossRate, EnvWAN)
inetRetry := computeRetries(lossRate, EnvInternet)
// LAN 目标概率更严格(0.5%),应该重试更多;但上限更低(4)
// Internet 目标概率更宽松(2%),应该重试更少;但上限更高(6)
t.Logf("30%% loss: LAN=%d, WAN=%d, Internet=%d", lanRetry, wanRetry, inetRetry)
if lanRetry < 1 || lanRetry > 4 {
t.Errorf("LAN retry=%d, 应在 [1,4]", lanRetry)
}
if wanRetry < 1 || wanRetry > 5 {
t.Errorf("WAN retry=%d, 应在 [1,5]", wanRetry)
}
if inetRetry < 1 || inetRetry > 6 {
t.Errorf("Internet retry=%d, 应在 [1,6]", inetRetry)
}
}
func TestOpt6_RetriesMaxByEnv(t *testing.T) {
// 高丢包率,各环境应返回各自上限
lanMax := computeRetries(0.99, EnvLAN)
wanMax := computeRetries(0.99, EnvWAN)
inetMax := computeRetries(0.99, EnvInternet)
if lanMax != 4 {
t.Errorf("LAN max retry=%d, want 4", lanMax)
}
if wanMax != 5 {
t.Errorf("WAN max retry=%d, want 5", wanMax)
}
if inetMax != 6 {
t.Errorf("Internet max retry=%d, want 6", inetMax)
}
}
func TestOpt6_RetriesMathCorrectness(t *testing.T) {
envs := []struct {
env NetworkEnv
targetProb float64
name string
}{
{EnvLAN, 0.005, "LAN"},
{EnvWAN, 0.01, "WAN"},
{EnvInternet, 0.02, "Internet"},
}
for _, e := range envs {
for _, loss := range []float64{0.05, 0.10, 0.20, 0.30} {
retries := computeRetries(loss, e.env)
prob := 1.0
for i := 0; i < retries; i++ {
prob *= loss
}
// 重试后全失败概率应 < targetProb(除非被 clamp 了)
if prob >= e.targetProb && retries < 4 {
t.Errorf("%s loss=%.0f%% retries=%d: P=%.6f >= %.3f",
e.name, loss*100, retries, prob, e.targetProb)
}
}
}
}
// =============================================================================
// 端到端集成:全链路验证
// =============================================================================
func TestOptAll_EndToEnd_LANToPool(t *testing.T) {
// 模拟内网探测 → TuneConfig → 创建池 → 池根据 env 自适应
profile := classifyNetwork(
makeDurations([]int{1, 1, 2, 2, 2, 3, 3, 3, 4, 5}),
0, 10,
)
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 65536, NumCPU: 8}}
ep.TuneConfig(config, session)
// 验证 env 被存储
if config.DetectedNetworkEnv != int(EnvLAN) {
t.Errorf("DetectedNetworkEnv=%d, want %d(LAN)", config.DetectedNetworkEnv, int(EnvLAN))
}
// 验证 ceiling 合理
if config.ThreadCeiling < config.ThreadNum {
t.Errorf("ceiling(%d) < target(%d)", config.ThreadCeiling, config.ThreadNum)
}
// 创建池并验证 env 传递
netEnv := NetworkEnv(config.DetectedNetworkEnv)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(config.ThreadNum, config.ThreadCeiling, func(interface{}) {}, metrics, netEnv)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
if pool.networkEnv != EnvLAN {
t.Errorf("池的 networkEnv=%v, want LAN", pool.networkEnv)
}
t.Logf("端到端 LAN: target=%d ceiling=%d env=%v maxRetry=%d",
config.ThreadNum, config.ThreadCeiling, netEnv, config.MaxRetries)
}
func TestOptAll_EndToEnd_InternetToPool(t *testing.T) {
profile := classifyNetwork(
makeDurations([]int{60, 70, 80, 90, 100, 110, 120, 130, 140, 150}),
0, 10,
)
config := makeDefaultConfig()
session := makeTestSession(config)
ep := &EnvironmentProfile{Net: *profile, System: SystemProfile{FDLimit: 4096, NumCPU: 4}}
ep.TuneConfig(config, session)
if config.DetectedNetworkEnv != int(EnvInternet) {
t.Errorf("DetectedNetworkEnv=%d, want %d(Internet)", config.DetectedNetworkEnv, int(EnvInternet))
}
// 公网 target 应明显低于默认 600
if config.ThreadNum >= 600 {
t.Errorf("公网 threadNum=%d, 应 < 600", config.ThreadNum)
}
// ceiling 应 == target(非显式模式)
if config.ThreadCeiling != config.ThreadNum {
t.Errorf("非显式模式 ceiling(%d) != target(%d)", config.ThreadCeiling, config.ThreadNum)
}
netEnv := NetworkEnv(config.DetectedNetworkEnv)
metrics := &ScanMetrics{}
pool, err := NewAdaptivePool(config.ThreadNum, config.ThreadCeiling, func(interface{}) {}, metrics, netEnv)
if err != nil {
t.Fatalf("创建池失败: %v", err)
}
defer pool.Release()
// 注入 12% exhaustInternet 环境应只是 Stressed 而不是 Congested
pool.inSlowStart = false
pool.tune(config.ThreadNum)
for i := 0; i < 100; i++ {
if i < 12 {
metrics.RecordExhausted()
} else {
metrics.RecordConnect(80 * time.Millisecond)
}
}
pool.lastCheck.Store(0)
pool.adjust()
// cap 不应被砍到一半以下(Stressed 只降 15%
if pool.Cap() < config.ThreadNum*7/10 {
t.Errorf("Internet 12%% exhaust 降速过猛: %d -> %d", config.ThreadNum, pool.Cap())
}
t.Logf("端到端 Internet: target=%d ceiling=%d cap_after_stress=%d",
config.ThreadNum, config.ThreadCeiling, pool.Cap())
}
+102
View File
@@ -0,0 +1,102 @@
package core
import (
"errors"
"fmt"
"net"
"strings"
"testing"
)
// =============================================================================
// Benchmark: containsFold vs strings.ToLower + strings.Contains
// =============================================================================
func BenchmarkContainsFold(b *testing.B) {
err := errors.New("connection reset by peer: 192.168.1.1:445")
b.ResetTimer()
for i := 0; i < b.N; i++ {
containsFold(err.Error(), "connection reset")
}
}
func BenchmarkStringsToLowerContains(b *testing.B) {
err := errors.New("connection reset by peer: 192.168.1.1:445")
b.ResetTimer()
for i := 0; i < b.N; i++ {
strings.Contains(strings.ToLower(err.Error()), "connection reset")
}
}
// =============================================================================
// Benchmark: fmt.Sprintf vs net.JoinHostPort + fmtPort
// =============================================================================
func BenchmarkFmtSprintfAddr(b *testing.B) {
host := "192.168.1.1"
port := 445
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = fmt.Sprintf("%s:%d", host, port)
}
}
func BenchmarkJoinHostPortFmtPort(b *testing.B) {
host := "192.168.1.1"
port := 445
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = net.JoinHostPort(host, fmtPort(port))
}
}
func BenchmarkFmtPort(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = fmtPort(445)
}
}
// =============================================================================
// Benchmark: readFromConn buffer pre-allocation
// =============================================================================
func BenchmarkAppendFromNil(b *testing.B) {
data := []byte("HTTP/1.1 200 OK\r\nServer: nginx")
chunk := data[:10]
b.ResetTimer()
for i := 0; i < b.N; i++ {
var result []byte
result = append(result, chunk...)
result = append(result, chunk...)
_ = result
}
}
func BenchmarkAppendPreAllocated(b *testing.B) {
data := []byte("HTTP/1.1 200 OK\r\nServer: nginx")
chunk := data[:10]
b.ResetTimer()
for i := 0; i < b.N; i++ {
result := make([]byte, 0, 4096)
result = append(result, chunk...)
result = append(result, chunk...)
_ = result
}
}
// =============================================================================
// Benchmark: AdaptiveTimeout computation under lock vs outside lock
// =============================================================================
func BenchmarkAdaptiveTimeoutComputation(b *testing.B) {
at := NewAdaptiveTimeout(3000 * 1000000) // 3s in ns
// Warm up: add 64 samples
for i := 0; i < 64; i++ {
at.Record(10 * 1000000) // 10ms in ns
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = at.Timeout()
}
}

Some files were not shown because too many files have changed in this diff Show More