feat: async/parallel execution with configurable concurrency
Parallelize LLM calls across minibatches to reduce wall-clock time. All domain ports (LLMPort, JudgePort, ProposerPort) are now async. Adapter implementations wrap synchronous DSPy calls with asyncio.to_thread. Judge calls run in parallel within a batch using asyncio.gather + semaphore. Evaluator parallelizes minibatch execution with configurable concurrency. Evolution loop and use case are fully async. Proposer stays sequential. Added --max-concurrency CLI flag and max_concurrency YAML config field. Added async_retry_with_backoff for async error handling. All 139 unit tests pass. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
"""Shared test fixtures."""
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -66,17 +66,17 @@ def mock_eval_result() -> EvalResult:
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_llm_port() -> MagicMock:
|
||||
def mock_llm_port() -> AsyncMock:
|
||||
"""Mock LLMPort that returns canned responses."""
|
||||
port = MagicMock()
|
||||
port = AsyncMock()
|
||||
port.execute.return_value = "This is a mock response."
|
||||
return port
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_judge_port() -> MagicMock:
|
||||
def mock_judge_port() -> AsyncMock:
|
||||
"""Mock JudgePort that returns moderate scores."""
|
||||
port = MagicMock()
|
||||
port = AsyncMock()
|
||||
port.judge_batch.return_value = [
|
||||
(0.5, "Moderate quality, needs improvement."),
|
||||
] * 5
|
||||
@@ -84,9 +84,9 @@ def mock_judge_port() -> MagicMock:
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_proposer_port() -> MagicMock:
|
||||
def mock_proposer_port() -> AsyncMock:
|
||||
"""Mock ProposerPort that returns a slightly modified prompt."""
|
||||
port = MagicMock()
|
||||
port = AsyncMock()
|
||||
port.propose.return_value = Prompt(
|
||||
text="You are a very helpful assistant. Answer the question precisely."
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user