AI Evidence Collection - Technical Implementation

Framework-Specific Evidence Collection

SOC 2 Type II - Access Control Examples

Control CC6.1: Logical and Physical Access Controls

Requirement: The entity implements logical and physical access controls to protect against threats from sources outside its system boundaries.

Required Evidence Types:

User Access Lists Permission Matrices MFA Status Login Logs Failed Login Attempts Access Reviews

AI Collection Sources:

Microsoft Graph (Azure AD users, groups, MFA)
Okta (SSO access, authentication logs)
ConnectWise (technician access controls)
AWS IAM (cloud resource permissions)
Application logs (direct database queries)

AI Evidence Collection Implementation

controlId, $client);
        
        // 1. Collect Azure AD User Access Evidence
        $azureUsers = $this->collectAzureADUsers($client);
        $evidence->addEvidence('azure_ad_users', [
            'type' => 'user_access_list',
            'source' => 'Microsoft Graph API',
            'data' => $azureUsers,
            'collected_at' => now(),
            'ai_classification' => 'access_control',
            'control_mapping' => ['SOC2_CC6.1', 'ISO27001_A.9.2.1']
        ]);
        
        // 2. Collect MFA Status Evidence
        $mfaStatus = $this->collectMFAStatus($client);
        $evidence->addEvidence('mfa_enforcement', [
            'type' => 'authentication_control',
            'source' => 'Azure AD Conditional Access',
            'data' => $mfaStatus,
            'compliance_score' => $this->calculateMFACompliance($mfaStatus),
            'ai_analysis' => $this->analyzeMFAGaps($mfaStatus)
        ]);
        
        // 3. Collect Login Activity Evidence
        $loginLogs = $this->collectLoginLogs($client, $periodStart, $periodEnd);
        $evidence->addEvidence('login_activity', [
            'type' => 'access_logs',
            'source' => 'Azure AD Sign-in Logs',
            'data' => $loginLogs,
            'period' => ['start' => $periodStart, 'end' => $periodEnd],
            'anomalies' => $this->detectLoginAnomalies($loginLogs),
            'ai_insights' => $this->generateLoginInsights($loginLogs)
        ]);
        
        // 4. AI Validation and Gap Analysis
        $validation = $this->validateEvidence($evidence);
        $evidence->setValidation($validation);
        
        return $evidence;
    }
    
    private function collectAzureADUsers(Client $client): array 
    {
        $graphClient = $this->getGraphClient($client);
        
        // Get all users with their access details
        $users = $graphClient->users()
            ->select(['id', 'userPrincipalName', 'accountEnabled', 
                     'assignedLicenses', 'memberOf', 'signInActivity'])
            ->get();
            
        // AI Enhancement: Classify user types and risk levels
        return $users->map(function($user) {
            return [
                'id' => $user->getId(),
                'email' => $user->getUserPrincipalName(),
                'enabled' => $user->getAccountEnabled(),
                'groups' => $this->getUserGroups($user),
                'permissions' => $this->getUserPermissions($user),
                'last_signin' => $user->getSignInActivity(),
                'ai_risk_score' => $this->calculateUserRiskScore($user),
                'ai_classification' => $this->classifyUserType($user)
            ];
        })->toArray();
    }
    
    private function collectMFAStatus(Client $client): array 
    {
        $graphClient = $this->getGraphClient($client);
        
        // Get MFA registration details
        $mfaData = $graphClient->reports()
            ->authenticationMethods()
            ->userRegistrationDetails()
            ->get();
            
        // AI Analysis: Calculate compliance percentage and gaps
        $compliance = $this->analyzeMFACompliance($mfaData);
        
        return [
            'total_users' => count($mfaData),
            'mfa_enabled_users' => $compliance['enabled_count'],
            'mfa_compliance_percentage' => $compliance['percentage'],
            'non_compliant_users' => $compliance['gaps'],
            'recommended_actions' => $this->generateMFARecommendations($compliance),
            'ai_confidence' => 0.95
        ];
    }
    
    private function validateEvidence(EvidencePackage $evidence): ValidationResult 
    {
        $validator = new AIEvidenceValidator();
        
        return $validator->validate([
            'completeness' => $this->checkCompleteness($evidence),
            'accuracy' => $this->verifyAccuracy($evidence),
            'timeliness' => $this->checkTimeliness($evidence),
            'relevance' => $this->assessRelevance($evidence, $this->controlId),
            'ai_confidence' => $this->calculateConfidenceScore($evidence)
        ]);
    }
}

// Usage Example
$collector = new SOC2_CC6_1_EvidenceCollector();
$client = Client::find(123);
$evidence = $collector->collectEvidence($client, 
    Carbon::now()->subDays(30), 
    Carbon::now()
);

// AI automatically validates and flags any issues
if ($evidence->getValidation()->isComplete()) {
    $evidence->store();
    event(new EvidenceCollected($evidence));
} else {
    // Alert compliance team for manual review
    $this->alertComplianceTeam($evidence->getValidation()->getGaps());
}
?>

ISO 27001 - Asset Management Example

Control A.8.1.1: Inventory of Assets

Requirement: Information and other associated assets used by the organization shall be identified and an inventory of these assets shall be drawn up and maintained.

Required Evidence Types:

Hardware Inventory Software Inventory Network Assets Data Classification Asset Owners Update Records

AI Collection Sources:

ConnectWise Manage (device inventory)
Auvik (network device discovery)
Ninja RMM (endpoint management)
Hudu (asset documentation)
Azure/AWS (cloud resource inventory)

ISO 27001 Asset Inventory Collection

controlId, $client);
        
        // Parallel collection from multiple sources
        $collections = collect([
            'hardware' => $this->collectHardwareAssets($client),
            'software' => $this->collectSoftwareAssets($client),
            'network' => $this->collectNetworkAssets($client),
            'cloud' => $this->collectCloudAssets($client)
        ]);
        
        // AI Correlation and Deduplication
        $consolidatedAssets = $this->aiConsolidateAssets($collections);
        
        // Generate comprehensive asset inventory
        $evidence->addEvidence('complete_asset_inventory', [
            'type' => 'asset_inventory',
            'framework_control' => 'ISO27001_A.8.1.1',
            'total_assets' => $consolidatedAssets->count(),
            'categories' => [
                'hardware' => $consolidatedAssets->where('type', 'hardware')->count(),
                'software' => $consolidatedAssets->where('type', 'software')->count(),
                'network' => $consolidatedAssets->where('type', 'network')->count(),
                'cloud' => $consolidatedAssets->where('type', 'cloud')->count()
            ],
            'assets' => $consolidatedAssets->toArray(),
            'ai_analysis' => $this->generateAssetAnalysis($consolidatedAssets),
            'compliance_gaps' => $this->identifyComplianceGaps($consolidatedAssets)
        ]);
        
        return $evidence;
    }
    
    private function collectHardwareAssets(Client $client): Collection 
    {
        // ConnectWise Manage API
        $cwApi = $this->getConnectWiseClient($client);
        $configurations = $cwApi->get('/company/configurations', [
            'conditions' => 'company/id=' . $client->connectwise_company_id,
            'pageSize' => 1000
        ]);
        
        return collect($configurations)->map(function($config) {
            return [
                'id' => $config['id'],
                'name' => $config['name'],
                'type' => 'hardware',
                'category' => $this->aiClassifyHardware($config),
                'manufacturer' => $config['manufacturer'] ?? null,
                'model' => $config['modelNumber'] ?? null,
                'serial_number' => $config['serialNumber'] ?? null,
                'location' => $config['locationName'] ?? null,
                'status' => $config['status']['name'] ?? 'Unknown',
                'owner' => $this->extractAssetOwner($config),
                'last_updated' => $config['lastUpdated'] ?? null,
                'ai_risk_score' => $this->calculateAssetRiskScore($config),
                'source' => 'ConnectWise Manage'
            ];
        });
    }
    
    private function collectNetworkAssets(Client $client): Collection 
    {
        // Auvik API
        $auvikApi = $this->getAuvikClient($client);
        $devices = $auvikApi->get('/v1/inventory/device/info');
        
        return collect($devices['data'])->map(function($device) {
            return [
                'id' => $device['id'],
                'name' => $device['attributes']['deviceName'],
                'type' => 'network',
                'category' => $device['attributes']['deviceType'],
                'ip_address' => $device['attributes']['ipAddresses'][0] ?? null,
                'mac_address' => $device['attributes']['macAddress'] ?? null,
                'vendor' => $device['attributes']['vendorName'] ?? null,
                'model' => $device['attributes']['modelName'] ?? null,
                'firmware' => $device['attributes']['firmwareVersion'] ?? null,
                'status' => $device['attributes']['deviceStatus'],
                'last_seen' => $device['attributes']['lastSeenTime'],
                'vulnerabilities' => $this->checkNetworkVulnerabilities($device),
                'ai_classification' => $this->aiClassifyNetworkDevice($device),
                'source' => 'Auvik'
            ];
        });
    }
    
    private function aiConsolidateAssets(Collection $collections): Collection 
    {
        $allAssets = $collections->flatten(1);
        
        // AI-powered deduplication based on multiple factors
        $deduplicator = new AssetDeduplicationAI();
        
        return $deduplicator->deduplicate($allAssets, [
            'match_factors' => ['serial_number', 'mac_address', 'name', 'ip_address'],
            'confidence_threshold' => 0.85,
            'merge_strategy' => 'most_complete_record'
        ]);
    }
    
    private function generateAssetAnalysis(Collection $assets): array 
    {
        $analyzer = new AssetAnalysisAI();
        
        return [
            'coverage_analysis' => $analyzer->analyzeCoverage($assets),
            'risk_assessment' => $analyzer->assessRisks($assets),
            'compliance_score' => $analyzer->calculateComplianceScore($assets),
            'recommendations' => $analyzer->generateRecommendations($assets),
            'trends' => $analyzer->identifyTrends($assets),
            'anomalies' => $analyzer->detectAnomalies($assets)
        ];
    }
}
?>

NIST CSF - Incident Response Example

Function RS.RP: Response Planning

Requirement: Response processes and procedures are executed and maintained, to ensure response to detected cybersecurity incidents.

Required Evidence Types:

Incident Response Plans Response Team Records Training Records Incident Logs Response Times Lessons Learned

AI Collection Sources:

ServiceNow (incident tickets)
ConnectWise (service desk)
Sentry (application errors)
Security tools (SIEM alerts)
Training platforms (completion records)

NIST CSF Incident Response Evidence

// NIST CSF RS.RP - Incident Response Evidence Collection
class NIST_RSRP_IncidentResponseCollector {
    constructor(client, integrationManager) {
        this.client = client;
        this.integrationManager = integrationManager;
        this.controlId = 'NIST_CSF_RS.RP';
    }
    
    async collectEvidence(startDate, endDate) {
        const evidence = new EvidencePackage(this.controlId, this.client);
        
        // Collect incident data from multiple sources in parallel
        const [
            incidentTickets,
            securityAlerts,
            responseMetrics,
            trainingRecords
        ] = await Promise.all([
            this.collectIncidentTickets(startDate, endDate),
            this.collectSecurityAlerts(startDate, endDate),
            this.collectResponseMetrics(startDate, endDate),
            this.collectTrainingRecords()
        ]);
        
        // AI Analysis of incident response effectiveness
        const aiAnalysis = await this.analyzeIncidentResponse({
            incidents: incidentTickets,
            alerts: securityAlerts,
            metrics: responseMetrics,
            training: trainingRecords
        });
        
        evidence.addEvidence('incident_response_capability', {
            type: 'incident_response',
            framework_control: 'NIST_CSF_RS.RP',
            period: { start: startDate, end: endDate },
            data: {
                total_incidents: incidentTickets.length,
                security_incidents: incidentTickets.filter(i => i.category === 'security').length,
                average_response_time: this.calculateAverageResponseTime(incidentTickets),
                response_team_readiness: aiAnalysis.teamReadiness,
                process_maturity_score: aiAnalysis.processMaturity
            },
            incidents: incidentTickets,
            ai_insights: aiAnalysis,
            compliance_score: aiAnalysis.complianceScore,
            recommendations: aiAnalysis.recommendations
        });
        
        return evidence;
    }
    
    async collectIncidentTickets(startDate, endDate) {
        // ConnectWise Manage API for incident tickets
        const cwApi = this.integrationManager.getConnectWiseClient(this.client);
        
        const tickets = await cwApi.get('/service/tickets', {
            conditions: `company/id=${this.client.connectwise_company_id} and 
                        dateEntered >= [${startDate}] and dateEntered <= [${endDate}] and
                        (summary contains "security" or summary contains "incident" or 
                         summary contains "breach" or priority/name = "Critical")`,
            pageSize: 1000
        });
        
        // AI Enhancement: Classify incident types and severity
        return tickets.map(ticket => ({
            id: ticket.id,
            summary: ticket.summary,
            description: ticket.description,
            priority: ticket.priority?.name,
            status: ticket.status?.name,
            dateEntered: ticket.dateEntered,
            dateResolved: ticket.dateResolved,
            responseTime: this.calculateResponseTime(ticket),
            resolutionTime: this.calculateResolutionTime(ticket),
            assignedTeam: ticket.team?.name,
            aiClassification: this.classifyIncidentType(ticket),
            aiSeverity: this.assessIncidentSeverity(ticket),
            complianceRelevant: this.isComplianceRelevant(ticket)
        }));
    }
    
    async collectSecurityAlerts(startDate, endDate) {
        // Example: Sentry error tracking
        const sentryApi = this.integrationManager.getSentryClient(this.client);
        
        const alerts = await sentryApi.get('/issues/', {
            query: `is:unresolved environment:production level:error`,
            start: startDate,
            end: endDate,
            limit: 1000
        });
        
        // AI Classification of security-relevant alerts
        return alerts.filter(alert => 
            this.isSecurityRelevant(alert)
        ).map(alert => ({
            id: alert.id,
            title: alert.title,
            level: alert.level,
            firstSeen: alert.firstSeen,
            lastSeen: alert.lastSeen,
            count: alert.count,
            project: alert.project.name,
            aiThreatLevel: this.assessThreatLevel(alert),
            aiImpact: this.assessBusinessImpact(alert),
            responseRequired: this.requiresIncidentResponse(alert)
        }));
    }
    
    async analyzeIncidentResponse(data) {
        const analyzer = new IncidentResponseAI();
        
        return {
            teamReadiness: await analyzer.assessTeamReadiness(data.training),
            processMaturity: await analyzer.assessProcessMaturity(data.incidents),
            responseEffectiveness: await analyzer.analyzeResponseTimes(data.metrics),
            complianceScore: await analyzer.calculateNISTCompliance(data),
            recommendations: await analyzer.generateRecommendations(data),
            trends: await analyzer.identifyTrends(data),
            gaps: await analyzer.identifyGaps(data)
        };
    }
    
    classifyIncidentType(ticket) {
        // AI-powered incident classification
        const classifier = new IncidentClassificationAI();
        
        return classifier.classify({
            title: ticket.summary,
            description: ticket.description,
            priority: ticket.priority,
            keywords: this.extractKeywords(ticket.summary + ' ' + ticket.description)
        });
    }
}

// Usage Example
const collector = new NIST_RSRP_IncidentResponseCollector(client, integrationManager);
const evidence = await collector.collectEvidence('2025-01-01', '2025-01-31');

// AI automatically evaluates NIST CSF compliance
if (evidence.getComplianceScore() >= 0.85) {
    console.log('NIST CSF RS.RP compliance: SATISFACTORY');
} else {
    console.log('NIST CSF RS.RP compliance: NEEDS IMPROVEMENT');
    console.log('AI Recommendations:', evidence.getRecommendations());
}

Integration API Examples

Microsoft Graph API

GET /v1.0/users
Collect user access information for SOC 2 CC6.1

GET /v1.0/reports/authenticationMethods/userRegistrationDetails
MFA compliance status for access controls

GET /v1.0/auditLogs/signIns
Login activity logs for access monitoring

ConnectWise Manage API

GET /company/configurations
Hardware asset inventory for ISO 27001 A.8.1.1

GET /service/tickets
Incident tickets for NIST CSF response planning

GET /system/members
Technician access controls and permissions

Auvik Network API

GET /v1/inventory/device/info
Network device inventory and classification

GET /v1/stat/device/bandwidth
Network monitoring for availability controls

GET /v1/alert
Network security alerts and incidents

Security Tool APIs

Huntress: /api/v1/agents
Endpoint protection status and threats

CyberCNS: /api/vulnerabilities
Vulnerability scan results and remediation

Breach Secure Now: /api/training/status
Security awareness training completion

Database Schema Requirements

To create more accurate and specific examples, I need to understand your current database structure:

Control & Framework Tables

How are controls stored and mapped to frameworks?
What's the relationship between controls and evidence requirements?
How do you handle cross-framework control mapping?
What metadata do you store for each control?

Integration & Evidence Tables

How is evidence linked to specific controls?
What evidence metadata do you currently capture?
How are API integration credentials managed?
What's your current evidence validation process?

Client & Assessment Tables

How are client assessments structured?
What's the relationship between clients and frameworks?
How do you handle assessment scoring and results?
What assessment metadata is captured?

Configuration & Settings

How are integration settings per client managed?
What automation rules/schedules exist?
How are evidence collection preferences stored?
What AI/ML configuration options are available?

Database Dump Benefits

With your actual database schema, I can create:

Accurate table relationships and foreign key references
Real control mappings across your supported frameworks
Proper evidence types and validation rules
Realistic API examples using your actual data structures
Migration scripts for AI enhancement features